DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 1, 7, and 11 and claims dependent thereon rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
Regarding Claim 1, the terms “an external environment/activity estimation neural network configured to detect at least one facial region as a region of interest" in lines 5-6 and “the at least one facial region includes inner sides of eyes, a nose, and a cheek” in line 17 render the claim indefinite because it is unclear if the claim is intending to be read as 1) the at least one facial region chosen for analysis is one of the inner sides of eyes, the nose, and the cheek or 2) the at least one facial region must include “inner sides of eyes, a nose, and a cheek” with other facial regions (e.g. a forehead) being potentially detected and analyzed in different steps. Examiner suggests amending the claim to read “an external environment/activity estimation neural network configured to detect a facial region as a region of interest" in lines 5-6 and “the facial region includes inner sides of eyes, a nose, and a cheek” in line 17 for claim clarity as this is the facial region relevant to the analysis and no claim refers to the detection and analysis of a second facial region. Mirrored changes should be made for independent Claims 7 and 11 and dependent claims 6, 10, and 14.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 5-7, 9-11, and 13-14 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
Regarding Claim 1, the claim(s) recites “an external environment/activity estimation neural network configured to detect at least one facial region as a region of interest from an input thermal image of a target person to be measured, and estimate an environmental type including an external temperature and participation in physical activity based on a temperature of the at least one region of interest; and
a body temperature prediction neural network configured to predict a body temperature of the target person based on the environmental type estimated by the external environment/activity estimation neural network and the temperature of the at least one region of interest.;
wherein:
the at least one facial region includes inner sides of eyes, a nose, and a cheek,
the external environment/activity estimation neural network is trained by using, as a first input data for training, a temperature of the at least one region of interest for each of face thermal images of a plurality of training targets, and by using, as label data, an environmental type including an external temperature and participation in physical activity according to the temperature of the at least one region of interest when measuring the temperature for each of the plurality of training targets, and
the body temperature prediction neural network is trained by using, as a second input data for training, a temperature of the at least one region of interest for each of the face thermal images of the plurality of training targets and an environmental type for training including an external temperature and participation in physical activity according to the temperature of the at least one region of interest for each of the plurality of training targets, and by using, as a label data, a body temperature obtained when measuring the temperature for each of the training targets; and
the first input data for training or the second input data for training includes:
a temperature of the inner sides of the eves, which is determined as a highest temperature among pixel temperatures within a region of the inner sides of the eves,
a temperature of the nose, which is determined as an average temperature of all pixels within a region of the nose, and
a temperature of the cheek, which is determined as an average temperature of all pixels within a region of the cheek.” which amounts to an abstract idea (mathematical concepts; see also claim 2 of example 47 from the 2024 AI SME Update1, which establishes that a neural network corresponds to a series of mathematical calculations). Furthermore, the limitations also recite a mental process as nothing from the claims suggest that a skilled artisan would not be able to practically perform these steps (clinician can predict body temperature based on mental analysis of thermal image, or using simple pen/paper).
This judicial exception is not integrated into a practical application because:
- The claims fail to outline an improvement to the technical field.
- The claims fail to apply the judicial exception to effect a particular treatment.
- The claims fail to apply the judicial exception with a particular machine.
- The claims fail to effect a transformation or reduction of a particular article to a different state or thing.
Next, the claim as a whole is analyzed to determine whether any element or a combination of elements, integrates judicial exception into a practical application.
For this part of the 101 analysis, the following additional limitations are considered:
“an input interface device configured to receive an input thermal image of a target person to be measured;”
“an output interface device configured to output the body temperature of the target person”
The additional elements are insufficient to amount to significantly more than the judicial exception because they seem to merely generally link the use of the judicial exception to a particular technological environment.
Moreover, the claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because they pertain merely to insignificant extrasolution data gathering activities and generic postsolution activities.
Furthermore, input interface devices and output interface devices are general fields of use.
Dependent claims 5-6 also do not recite patent eligible subject matter as they merely further limit the abstract idea, recite limitations that do not integrate the claims into a practical application for similar reasons as set forth above, and/or do not recite significantly more than the identified abstract idea for substantially similar reasons as set forth above.
Regarding Claim 7, the claim(s) recites “estimating an environmental type including an external temperature and participation in physical activity based on a temperature of the at least one region of interest; and
predicting a body temperature of the target person based on the estimated environmental type and the temperature of the at least one region of interest for the input thermal image;
…wherein:
the at least one facial region includes inner sides of eyes, a nose, and a cheek,
the external environment/activity estimation neural network is trained by using, as a first input data for training, a temperature of the at least one region of interest for each of face thermal images of a plurality of training targets, and by using, as label data, an environmental type including an external temperature and participation in physical activity according to the temperature of the at least one region of interest when measuring the temperature for each of the plurality of training targets, and
the body temperature prediction neural network is trained by using, as a second input data for training, a temperature of the at least one region of interest for each of the face thermal images of the plurality of training targets and an environmental type for training including an external temperature and participation in physical activity according to the temperature of the at least one region of interest for each of the plurality of training targets, and by using, as a label data, a body temperature obtained when measuring the temperature for each of the training targets; and
the first input data for training or the second input data for training includes:
a temperature of the inner sides of the eves, which is determined as a highest temperature among pixel temperatures within a region of the inner sides of the eves,
a temperature of the nose, which is determined as an average temperature of all pixels within a region of the nose, and
a temperature of the cheek, which is determined as an average temperature of all pixels within a region of the cheek.” which amounts to an abstract idea (mathematical concepts, under the broadest reasonable interpretation when read in light of the Specification, a prediction of the numerical value of body temperature from input data would corresponds to an algorithm, i.e. a mathematical calculation. Further, the limitations also recite a mental process, i.e. clinician can predict body temperature based on mental analysis of thermal image, or using simple pen/paper).
This judicial exception is not integrated into a practical application because:
- The claims fail to outline an improvement to the technical field.
- The claims fail to apply the judicial exception to effect a particular treatment.
- The claims fail to apply the judicial exception with a particular machine.
- The claims fail to effect a transformation or reduction of a particular article to a different state or thing.
Next, the claim as a whole is analyzed to determine whether any element or a combination of elements, integrates judicial exception into a practical application.
For this part of the 101 analysis, the following additional limitations are considered:
“detecting at least one facial region as a region of interest from an input thermal image of a target person to be measured received via an input interface device,”
“and outputting the body temperature of the target person via an output interface device,”
The additional elements are insufficient to amount to significantly more than the judicial exception because they seem to merely generally link the use of the judicial exception to a particular technological environment.
Moreover, the claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because they pertain merely to insignificant extrasolution data gathering activities and generic postsolution activities.
None of these limitations, considered as an ordered combination provide eligibility because the claim taken as a whole, does not amount to significantly more than the underlying abstract idea of making a prediction on body temperature based on a facial region temperature and secondary contextual parameter (e.g. environmental temperature and physical activity context) and does not purport to improve the functioning of the signal processing, or to improve any other technology or technical field. Use of a generic signal processing does not amount to significantly more than the abstract idea itself.
Dependent claims 9-10 do not recite patent eligible subject matter as they merely further limit the abstract idea, recite limitations that do not integrate the claims into a practical application for similar reasons as set forth above, and/or do not recite significantly more than the identified abstract idea for substantially similar reasons as set forth above.
Regarding Claim 11, the claim(s) recites “training an external environment/activity estimation neural network by using, as first training input data, a plurality of face thermal images for training and by using, as first label data, an environmental type including an external temperature and participation in physical activity according to a temperature of at least one facial region, which is a region of interest, of each of the plurality of face thermal images for training to detect the at least one region of interest of a target person from an input thermal image of the target person, and estimate the environmental type for the target person based on the temperature of the at least one region of interest for the target person; and
training a body temperature prediction neural network by using, as second training input data, a plurality of face thermal images for training and a plurality of estimated environmental types for training including an external temperature and participation in physical activity and by using, as second label data, body temperatures obtained based on a temperature of at least one facial region, which is a region of interest, of each of the plurality of face thermal images for training and the plurality of estimated environmental types for training to predict a body temperature of the target person based on the temperature of the at least one region of interest of the target person and the estimated environmental type for the target person; wherein
the at least one facial region includes inner sides of eyes, a nose, and a cheek, and
the first training input data or the second training input data includes:
a temperature of the inner sides of the eves, which is determined as a highest temperature among pixel temperatures within a region of the inner sides of the eves,
a temperature of the nose, which is determined as an average temperature of all pixels within a region of the nose, and
a temperature of the cheek, which is determined as an average temperature of all pixels within a region of the cheek.” which amounts to an abstract idea (mathematical concepts; see also claim 2 of example 47 from the 2024 AI SME Update2, which establishes that training a neural network corresponds to a series of mathematical calculations). Furthermore, the limitations also recite a mental process as nothing from the claims suggest that a skilled artisan would not be able to practically perform these steps (clinician can test and optimize a body temperature prediction based on mental analysis of training thermal images, or using simple pen/paper).
This judicial exception is not integrated into a practical application because:
- The claims fail to outline an improvement to the technical field.
- The claims fail to apply the judicial exception to effect a particular treatment.
- The claims fail to apply the judicial exception with a particular machine.
- The claims fail to effect a transformation or reduction of a particular article to a different state or thing.
Next, the claim as a whole is analyzed to determine whether any element or a combination of elements, integrates judicial exception into a practical application.
For this part of the 101 analysis, there are no additional limitations to consider.
Dependent claims 13-14 do not recite patent eligible subject matter as they merely further limit the abstract idea, recite limitations that do not integrate the claims into a practical application for similar reasons as set forth above, and/or do not recite significantly more than the identified abstract idea for substantially similar reasons as set forth above.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 5-7, 9-10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Frank et al (US 2020/0397306) (“Frank”) in view of Beall (US 2021/0302238) and further in view of Putterman et al (US 2022/0157146) (“Putterman”) and further in view of LaBelle et al (US 2015/0238140) (“LaBelle”).
Regarding Claim 1, while Frank teaches an apparatus for predicting a body temperature (Abstract, Figs. 1, 17A, [0114]-[0116], [0160], [0177]-[0179] system as a whole used to find body temperature, a value that can then be compared to a threshold to identify fever in a patient, [0182] and the system can measure for other conditions described within the specification such as stroke monitoring in Figs. 17a-17c), the apparatus comprising:
an input interface device configured to receive an input thermal image of a target person to be measured (Fig. 1d, [0194] smartglasses 370 uses detection components to receive data of a target person to be measured, thermal data of a target person can be captured from head-mounted temperature sensor 372, thermal data of the target person’s environment can be captured from head-mounted temperature sensor 374, [0093] where the thermal data captured may be a thermal image, [0217] environmental thermal images may be derived from thermal data of a target person’s face by an inward-facing temperature sensor);
detect at least one facial region from input images of the target person ([0202]-[0203] inward-facing head-mounted camera 376 provides images 377 that can be processed to detect desired facial regions, [0199] where desired regions of interest include nose, temple, forehead, and/or cheekbone);
an external environment/activity estimation neural network configured to estimate an environmental type, including at least one of an external temperature and participation in physical activity based on a temperature of the at least one region of interest ([0217] environmental / external temperature may be estimated from a temperature measurement of a region of interest of a patient, the environmental temperature reflecting an environmental type, [0081] additional data that may characterize the environmental data, [0148], [0150] hemoglobin concentration measurements can act as environmental data reflective of environmental temperature, [0178]-[0181] a model 346 is trained to generate feature values to identify fever from input data, the model is trained with input data representing various external environmental temperatures and activity states, the model 346 can be a neural network. Thus, the environmental type and its effect on the skin temperature must be estimated and accounted for in the final fever determination);
a body temperature prediction neural network configured to predict a body temperature of the target person based on the environmental type estimated by the external environment/activity estimation neural network and the temperature of the at least one region of interest ([0178]-[0181], [0217] model 346 acts as both external environmental/activity estimation neural network and body temperature prediction neural network to predict a body temperature as an intermediate value in determining fever),
the at least one facial region includes inner sides of eyes, a nose, and a cheek ([0199] the facial regions for skin temperature can include nose and cheek, [0173] a stress-specific inward-facing thermal camera can be used, measuring the periorbital region which includes the area around a target person’s eye, [0222] where stress may be used as input to the fever detection model),
the body temperature prediction neural network is trained by using, as a input data for training, a temperature of the at least one region of interest for each of face thermal images of a plurality of training targets, an external temperature, according to the temperature of the at least one region of interest when measuring the temperature for each of the plurality of training targets, and additional input data of participation in physical activity ([0172], [0178]-[0180]) and by using, as label data, a body temperature obtained when measuring the temperature for each of the training targets ([0184]-[0185]); and
and Frank teaches an output interface device configured to output the calculated values of the target person ([0187] a calculated value would include the predicted internal body temperature), the calculated values including body temperature.
And Frank further teaches in a second embodiment determining congestive heart failure that imaging-related values may be captured as averages of pixels in a region ([0314]) and that teachings from separate embodiments may be combined ([0540]-[0541])
Frank fails to teach the first input data for training or the second input data for training includes:
a temperature of the inner side of the eyes, which is determined as a representative temperature among pixel temperatures within a region of the inner side of the eyes, and
a temperature of the nose, which is determined as an average temperature of all pixels within a region of the nose, and
a temperature of the cheek, which is determined as an average temperature of all pixels within a region of the cheek.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to that the thermal image of the fever evaluating embodiment of Frank may additionally utilize the imaging processing step of averaging pixel values from the CHF evaluating embodiment of Frank to ensure a representative feature value is provided while reducing the amount of facial data that must be processed.
Yet Frank fails to teach
an external environment/activity estimation configured to detect at least one facial region as a region of interest from the input thermal image of the target person; and
the representative temperature of the inner side of the eyes is determined as a highest temperature.
However Beall teaches a method for predicting a body temperature (Abstract, [0046], [0078]), the method comprising:
detecting at least one facial region as a region of interest from an input thermal image of a target person to be measured ([0078] “Upon a subject entering the FOV, the device 100 detects face and eye regions directly in the thermal image and a predetermined ROI shape is applied at each visible canthus location and the maxima or other summary statistic is taken as the canthi temperature estimate.” Subject’s face is the region of interest [0081]-[0082]), and the representative temperature of the inner side of the eyes is determined as a highest temperature ([0078] “More specifically, thermal imaging of key locations on the body (e.g., the inner canthus) typically have few confounds and provide the closest correspondence to a subject's core body temperature, suitable for applying a correction for physiology to arrive at a reliable estimate of core body temperature…Upon a subject entering the FOV, the device 100 detects face and eye regions directly in the thermal image and a predetermined ROI shape is applied at each visible canthus location and the maxima or other summary statistic is taken as the canthi temperature estimate.” Where an inner canthi is a subregion within the periorbital region).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to detect the facial region of interest in Frank from the thermal image as taught by Beall as this identification ensures the appropriate facial regions are being measured, and thus the measured temperatures are serving their intended purpose of identifying body temperature. Furthermore, it would be obvious for the representative temperature of the inner eye taught in Frank to be substituted from an average temperature to a max temperature as the max temperature is a recognized equivalent summary statistic by Beall.
Yet their combined efforts fail to teach
the external environment/activity estimation neural network configured to detect at least one facial region as a region of interest from the input thermal image of the target person
the external environment/activity estimation neural network is trained by using, as a first input data for training, a temperature of the at least one region of interest for each of face thermal images of a plurality of training targets, and by using, as label data, an environmental type including an external temperature and participation in physical activity according to the temperature of the at least one region of interest when measuring the temperature for each of the plurality of training targets, and
the body temperature prediction neural network is trained by using, as a second input data for training, a temperature of the at least one region of interest for each of the face thermal images of the plurality of training targets and an environmental type for training including an external temperature and participation in physical activity according to the temperature of the at least one region of interest for each of the plurality of training targets, and by using, as a label data, a body temperature obtained when measuring the temperature for each of the training targets.
However Putterman teaches an imaging-based body temperature prediction device (Abstract) where facial skin temperature is used as an input to predict body temperature and fever (Abstract, [0013]-[0014]) comprising
Processing by using of more or more neural networks ([0013]),
a categorizing clustering step related to contextual environment data of a target person ([0028]-[0029]),
the categorizing clustering step performed with machine learning techniques ([0028]); and
a representative temperature value may be either a mean temperature or a maximum temperature ([0054]).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, that Frank’s neural network monitoring can be performed with an intermediate patient categorization step by clustering as taught by Putterman as a way to process the same information of environmental temperature and skin temperature in Frank, while also enabling a caregiver to review the machine learning’s categorization outcome. In doing so, one can verify that the environmental data is being appropriately considered. And Frank teaches that machine learning steps of clustering and neural networks may both be applied in processing information ([0077]). Consequently, it would be obvious that that adding an external environment/activity estimation neural network utilizing a clustering technique to Frank can support the body temperature identification of Frank, by considering how the baseline temperature of individual target person should be corrected for in a final body temperature/fever calculation. And applying this to Frank would require facial skin temperature and environmental data as the input and a label of environmental/cluster category as the output to train this neural network. Correspondingly, the original body temperature prediction neural network of Frank can utilize the skin temperature and environmental/cluster category as the input and a label of body temperature as the output for training Frank’s neural network. Finally, it would be obvious that the external environment/activity estimation neural network step would incorporate Beall’s facial region detection to facilitate a streamlined, automated processing by first identifying the appropriate measuring location before measuring input data and establishing a relationship with labeled output data.
Yet their combined efforts fail to teach the body temperature prediction neural network is trained by using, as a second input data for training, an environmental type for training including participation in physical activity according to the temperature of the at least one region of interest for each of the plurality of training targets.
However LaBelle teaches a physiological monitoring device (Abstract) and teaches that changes in surface temperature due to stress and physical activity are related ([0008]-[0010] physical exercise is a stress state of the body, [0030]-[0032] changes in surface temperature and body temperature due to stress and exercise are equivalent).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, that the stress evaluation included in the fever prediction of Frank can alternatively be considered as a measure of exercise / participation in physical activity as LaBelle teaches that exercise is a stress state and exercise and stress cause related changes in surface temperature and body temperature of a patient. Thus, the included stress evaluation of facial regions of interest through neural networks of Frank ([0173], [0222]) would also act as physical activity evaluations and would be trained by the same mechanisms.
Regarding Claim 5, Frank, Beall, Putterman, and LaBelle teach the apparatus of claim 1, wherein the environmental type includes at least one of a hot environment, an environment with exercise, a normal environment without exercise, and a cold environment (See Claim 1 Rejection, Frank [0179] describes training conditions for different temperature environment and different patient movement states that fulfill at least one of a hot environment, an environment with exercise, a normal environment without exercise, and a cold environment).
Regarding Claim 6, Frank, Beall, Putterman, and LaBelle teach the apparatus of claim 1, and Beall further teaches wherein in the detected region of interest, a face region is detected from the input thermal image of the target person based on a first object detection algorithm ([0107] described facial recognition algorithm), and the at least one facial region is detected as the region of interest within the detected face region based on a second object detection algorithm ([0078] predetermined region of interest shape, relative to a face shape used to identify location of canthi).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, that the facial recognition of Beall utilize multiple detection algorithms as this enables the detection algorithms to be specialized to their relative location.
Regarding Claim 7, while Frank teaches a method for predicting a body temperature to be performed by a body temperature apparatus including an external environment/activity estimation neural network and a body temperature prediction neural network (Abstract, Figs. 1, 17A, [0114]-[0116], [0160], [0177]-[0181] system as a whole used to find body temperature, a value that can then be compared to a threshold to identify fever in a patient, a model 346 is trained to generate feature values to identify fever from input data, the model is trained with input data representing various external environmental temperatures and activity states, the model 346 can be a neural network. Thus, model 346 acts as both external environmental/activity estimation neural network and body temperature prediction neural network to predict a body temperature as an intermediate value in determining fever, [0182] and the system can measure for other conditions described within the specification such as stroke monitoring in Figs. 17a-17c), the method comprising:
detect at least one facial region from input images of the target person ([0202]-[0203] inward-facing head-mounted camera 376 provides images 377 that can be processed to detect desired facial regions, [0199] where desired regions of interest include nose, temple, forehead, and/or cheekbone, Fig. 1d, [0194] smartglasses 370 uses detection components to receive data of a target person to be measured) and receiving an input thermal input image of a target person via an input interface device (Fig. 1d, [0194] thermal data of a target person can be captured from head-mounted temperature sensor 372, thermal data of the target person’s environment can be captured from head-mounted temperature sensor 374, [0093] where the thermal data captured may be a thermal image, [0217] environmental thermal images may be derived from thermal data of a target person’s face by an inward-facing temperature sensor);
estimating an environmental type, including at least one of an external temperature and participation in physical activity based on a temperature of the at least one region of interest ([0217] environmental / external temperature may be estimated from a temperature measurement of a region of interest of a patient, the environmental temperature reflecting an environmental type, [0081] additional data that may characterize the environmental data, [0148], [0150] hemoglobin concentration measurements can act as environmental data reflective of environmental temperature, [0178]-[0181] a model 346 is trained to generate feature values to identify fever from input data, the model is trained with input data representing various external environmental temperatures and activity states, thus, the environmental type and its effect on the skin temperature must be estimated and accounted for in the final fever determination);
predicting a body temperature of the target person based on the estimated environmental type and the temperature of the at least one region of interest ([0178]-[0181], [0217] model 346 acts as both external environmental/activity estimation neural network and body temperature prediction neural network to predict a body temperature as an intermediate value in determining fever),
the at least one facial region includes inner sides of eyes, a nose, and a cheek ([0199] the facial regions for skin temperature can include nose and cheek, [0173] a stress-specific inward-facing thermal camera can be used, measuring the periorbital region which includes the area around a target person’s eye, [0222] where stress may be used as input to the fever detection model),
the body temperature prediction neural network is trained by using, as a input data for training, a temperature of the at least one region of interest for each of face thermal images of a plurality of training targets, an external temperature, according to the temperature of the at least one region of interest when measuring the temperature for each of the plurality of training targets, and additional input data of participation in physical activity ([0172], [0178]-[0180]) and by using, as label data, a body temperature obtained when measuring the temperature for each of the training targets ([0184]-[0185]);
Frank teaches outputting the calculated values of the target person via an output interface device ([0187] a calculated value would include the predicted internal body temperature), the calculated values including body temperature; and
Frank further teaches in a second embodiment determining congestive heart failure that imaging-related values may be captured as averages of pixels in a region ([0314]) and that teachings from separate embodiments may be combined ([0540]-[0541]),
Frank fails to teach the first input data for training or the second input data for training includes:
a temperature of the inner side of the eyes, which is determined as a representative temperature among pixel temperatures within a region of the inner side of the eyes, and
a temperature of the nose, which is determined as an average temperature of all pixels within a region of the nose, and
a temperature of the cheek, which is determined as an average temperature of all pixels within a region of the cheek.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to that the thermal image of the fever evaluating embodiment of Frank may additionally utilize the imaging processing step of averaging pixel values from the CHF evaluating embodiment of Frank to ensure a representative feature value is provided while reducing the amount of facial data that must be processed.
Yet Frank fails to teach
detecting at least one facial region as a region of interest from the input thermal image of the target person; and
the representative temperature of the inner side of the eyes is determined as a highest temperature.
However Beall teaches a method for predicting a body temperature (Abstract, [0046], [0078]), the method comprising:
detecting at least one facial region as a region of interest from an input thermal image of a target person to be measured ([0078] “Upon a subject entering the FOV, the device 100 detects face and eye regions directly in the thermal image and a predetermined ROI shape is applied at each visible canthus location and the maxima or other summary statistic is taken as the canthi temperature estimate.” Subject’s face is the region of interest [0081]-[0082]), and the representative temperature of the inner side of the eyes is determined as a highest temperature ([0078] “More specifically, thermal imaging of key locations on the body (e.g., the inner canthus) typically have few confounds and provide the closest correspondence to a subject's core body temperature, suitable for applying a correction for physiology to arrive at a reliable estimate of core body temperature…Upon a subject entering the FOV, the device 100 detects face and eye regions directly in the thermal image and a predetermined ROI shape is applied at each visible canthus location and the maxima or other summary statistic is taken as the canthi temperature estimate.” Where an inner canthi is a subregion within the periorbital region).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to detect the facial region of interest in Frank from the thermal image as taught by Beall as this identification ensures the appropriate facial regions are being measured, and thus the measured temperatures are serving their intended purpose of identifying body temperature. Furthermore, it would be obvious for the representative temperature of the inner eye taught in Frank to be substituted from an average temperature to a max temperature as the max temperature is a recognized equivalent summary statistic by Beall.
Yet their combined efforts fail to teach
the external environment/activity estimation neural network is trained by using, as a first input data for training, a temperature of the at least one region of interest for each of face thermal images of a plurality of training targets, and by using, as label data, an environmental type including an external temperature and participation in physical activity according to the temperature of the at least one region of interest when measuring the temperature for each of the plurality of training targets, and
the body temperature prediction neural network is trained by using, as a second input data for training, a temperature of the at least one region of interest for each of the face thermal images of the plurality of training targets and an environmental type for training including an external temperature and participation in physical activity according to the temperature of the at least one region of interest for each of the plurality of training targets, and by using, as a label data, a body temperature obtained when measuring the temperature for each of the training targets.
However Putterman teaches an imaging-based body temperature prediction device (Abstract) where facial skin temperature is used as an input to predict body temperature and fever (Abstract, [0013]-[0014]) comprising
Processing by using of more or more neural networks ([0013]),
a categorizing clustering step related to contextual environment data of a target person ([0028]-[0029]),
the categorizing clustering step performed with machine learning techniques ([0028]); and
a representative temperature value may be either a mean temperature or a maximum temperature ([0054]).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, that Frank’s neural network monitoring can be performed with an intermediate patient categorization step by clustering as taught by Putterman as a way to process the same information of environmental temperature and skin temperature in Frank, while also enabling a caregiver to review the machine learning’s categorization outcome. In doing so, one can verify that the environmental data is being appropriately considered. And Frank teaches that machine learning steps of clustering and neural networks may both be applied in processing information ([0077]). Consequently, it would be obvious that that adding an external environment/activity estimation neural network utilizing a clustering technique to Frank can support the body temperature identification of Frank, by considering how the baseline temperature of individual target person should be corrected for in a final body temperature/fever calculation. And applying this to Frank would require facial skin temperature and environmental data as the input and a label of environmental/cluster category as the output to train this neural network. Correspondingly, the original body temperature prediction neural network of Frank can utilize the skin temperature and environmental/cluster category as the input and a label of body temperature as the output for training Frank’s neural network. Finally, it would be obvious that Frank’s neural network step would incorporate Beall’s facial region detection to facilitate a streamlined, automated processing by first identifying the appropriate measuring location before measuring input data and establishing a relationship with labeled output data.
Yet their combined efforts fail to teach the body temperature prediction neural network is trained by using, as a second input data for training, an environmental type for training including participation in physical activity according to the temperature of the at least one region of interest for each of the plurality of training targets.
However LaBelle teaches a physiological monitoring device (Abstract) and teaches that changes in surface temperature due to stress and physical activity are related ([0008]-[0010] physical exercise is a stress state of the body, [0030]-[0032] changes in surface temperature and body temperature due to stress and exercise are equivalent).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, that the stress evaluation included in the fever prediction of Frank can alternatively be considered as a measure of exercise / participation in physical activity as LaBelle teaches that exercise is a stress state and exercise and stress cause related changes in surface temperature and body temperature of a patient. Thus, the included stress evaluation of facial regions of interest through neural networks of Frank ([0173], [0222]) would also act as physical activity evaluations and would be trained by the same mechanisms.
Regarding Claim 9, Frank, Beall, Putterman, and LaBelle teach the method of claim 7, wherein the environmental type includes at least one of a hot environment, an environment with exercise, a normal environment without exercise, and a cold environment (See Claim 7 Rejection, Frank [0179] describes training conditions for different temperature environment and different patient movement states that fulfill at least one of a hot environment, an environment with exercise, a normal environment without exercise, and a cold environment).
Regarding Claim 10, Frank, Beall, Putterman, and LaBelle teach the method of claim 7, and Beall further teaches wherein in the detected region of interest, a face region is detected from the input thermal image of the target person based on a first object detection algorithm ([0107] described facial recognition algorithm), and the at least one facial region is detected as the region of interest within the detected face region based on a second object detection algorithm ([0078] predetermined region of interest shape, relative to a face shape used to identify location of canthi).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, that the facial recognition of Beall utilize multiple detection algorithms as this enables the detection algorithms to be specialized to their relative location.
Regarding Claim 11, while Frank teaches a method for training a body temperature prediction apparatus (Abstract, Figs. 1, 17A, [0114]-[0116], [0160], [0177]-[0181] system as a whole used to find body temperature, a value that can then be compared to a threshold to identify fever in a patient, a model 346 is trained to generate feature values to identify fever from input data, the model is trained with input data representing various external environmental temperatures and activity states, the model 346 can be a neural network. Thus, model 346 acts as both external environmental/activity estimation neural network and body temperature prediction neural network to predict a body temperature as an intermediate value in determining fever), the method comprising:
detect at least one facial region from input images of the target person ([0202]-[0203] inward-facing head-mounted camera 376 provides images 377 that can be processed to detect desired facial regions, [0199] where desired regions of interest include nose, temple, forehead, and/or cheekbone, Fig. 1d, [0194] smartglasses 370 uses detection components to receive data of a target person to be measured) and receiving an input thermal input image of a target person via an input interface device (Fig. 1d, [0194] thermal data of a target person can be captured from head-mounted temperature sensor 372, thermal data of the target person’s environment can be captured from head-mounted temperature sensor 374, [0093] where the thermal data captured may be a thermal image, [0217] environmental thermal images may be derived from thermal data of a target person’s face by an inward-facing temperature sensor);
estimating an environmental type, including at least one of an external temperature and participation in physical activity based on a temperature of the at least one region of interest ([0217] environmental / external temperature may be estimated from a temperature measurement of a region of interest of a patient, the environmental temperature reflecting an environmental type, [0081] additional data that may characterize the environmental data, [0148], [0150] hemoglobin concentration measurements can act as environmental data reflective of environmental temperature, [0178]-[0181] a model 346 is trained to generate feature values to identify fever from input data, the model is trained with input data representing various external environmental temperatures and activity states, thus, the environmental type and its effect on the skin temperature must be estimated and accounted for in the final fever determination);
predicting a body temperature of the target person based on the estimated environmental type and the temperature of the at least one region of interest ([0178]-[0181], [0217] model 346 acts as both external environmental/activity estimation neural network and body temperature prediction neural network to predict a body temperature as an intermediate value in determining fever),
the at least one facial region includes inner sides of eyes, a nose, and a cheek ([0199] the facial regions for skin temperature can include nose and cheek, [0173] a stress-specific inward-facing thermal camera can be used, measuring the periorbital region which includes the area around a target person’s eye, [0222] where stress may be used as input to the fever detection model),
the body temperature prediction neural network is trained by using, as a input data for training, a temperature of the at least one region of interest for each of face thermal images of a plurality of training targets, an external temperature, according to the temperature of the at least one region of interest when measuring the temperature for each of the plurality of training targets, and additional input data of participation in physical activity ([0172], [0178]-[0180]) and by using, as label data, a body temperature obtained when measuring the temperature for each of the training targets ([0184]-[0185]);
Yet Frank fails to teach
Training an external environmental/activity estimation neural network includes detecting at least one facial region as a region of interest from the input thermal image of the target person; and
the representative temperature of the inner side of the eyes is determined as a highest temperature.
However Beall teaches a method for predicting a body temperature (Abstract, [0046], [0078]), the method comprising:
detecting at least one facial region as a region of interest from an input thermal image of a target person to be measured ([0078] “Upon a subject entering the FOV, the device 100 detects face and eye regions directly in the thermal image and a predetermined ROI shape is applied at each visible canthus location and the maxima or other summary statistic is taken as the canthi temperature estimate.” Subject’s face is the region of interest [0081]-[0082]), and the representative temperature of the inner side of the eyes is determined as a highest temperature ([0078] “More specifically, thermal imaging of key locations on the body (e.g., the inner canthus) typically have few confounds and provide the closest correspondence to a subject's core body temperature, suitable for applying a correction for physiology to arrive at a reliable estimate of core body temperature…Upon a subject entering the FOV, the device 100 detects face and eye regions directly in the thermal image and a predetermined ROI shape is applied at each visible canthus location and the maxima or other summary statistic is taken as the canthi temperature estimate.” Where an inner canthi is a subregion within the periorbital region).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to detect the facial region of interest in Frank from the thermal image as taught by Beall as this identification ensures the appropriate facial regions are being measured, and thus the measured temperatures are serving their intended purpose of identifying body temperature. Furthermore, it would be obvious for the representative temperature of the inner eye taught in Frank to be substituted from an average temperature to a max temperature as the max temperature is a recognized equivalent summary statistic by Beall.
Yet their combined efforts fail to teach
Training the external environment/activity estimation neural network by using, as a first training input data, a plurality of face thermal images for training, and by using, as first label data, an environmental type including an external temperature and participation in physical activity according to the temperature of the at least one facial region, and
Training the body temperature prediction neural network by using, as a second training input data, a plurality of face thermal images for training and a plurality of estimated environmental types for training including an external temperature and participation in physical activity and by using, as second label data, body temperature obtained based on a temperature of at least one facial region, which is a region of interest, of each of the plurality of face thermal images for training.
However Putterman teaches an imaging-based body temperature prediction device (Abstract) where facial skin temperature is used as an input to predict body temperature and fever (Abstract, [0013]-[0014]) comprising
Processing by using of more or more neural networks ([0013]),
a categorizing clustering step related to contextual environment data of a target person ([0028]-[0029]),
the categorizing clustering step performed with machine learning techniques ([0028]); and
a representative temperature value may be either a mean temperature or a maximum temperature ([0054]).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, that Frank’s neural network monitoring can be performed with an intermediate patient categorization step by clustering as taught by Putterman as a way to process the same information of environmental temperature and skin temperature in Frank, while also enabling a caregiver to review the machine learning’s categorization outcome. In doing so, one can verify that the environmental data is being appropriately considered. And Frank teaches that machine learning steps of clustering and neural networks may both be applied in processing information ([0077]). Consequently, it would be obvious that that adding an external environment/activity estimation neural network utilizing a clustering technique to Frank can support the body temperature identification of Frank, by considering how the baseline temperature of individual target person should be corrected for in a final body temperature/fever calculation. And applying this to Frank would require facial skin temperature and environmental data as the input and a label of environmental/cluster category as the output to train this neural network. Correspondingly, the original body temperature prediction neural network of Frank can utilize the skin temperature and environmental/cluster category as the input and a label of body temperature as the output for training Frank’s neural network. Finally, it would be obvious that Frank’s neural network step would incorporate Beall’s facial region detection to facilitate a streamlined, automated processing by first identifying the appropriate measuring location before measuring input data and establishing a relationship with labeled output data.
Yet their combined efforts fail to teach the body temperature prediction neural network is trained by using, as a second input data for training, an environmental type for training including participation in physical activity according to the temperature of the at least one region of interest for each of the plurality of training targets.
However LaBelle teaches a physiological monitoring device (Abstract) and teaches that changes in surface temperature due to stress and physical activity are related ([0008]-[0010] physical exercise is a stress state of the body, [0030]-[0032] changes in surface temperature and body temperature due to stress and exercise are equivalent).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, that the stress evaluation included in the fever prediction of Frank can alternatively be considered as a measure of exercise / participation in physical activity as LaBelle teaches that exercise is a stress state and exercise and stress cause related changes in surface temperature and body temperature of a patient. Thus, the included stress evaluation of facial regions of interest through neural networks of Frank ([0173], [0222]) would also act as physical activity evaluations and would be trained by the same mechanisms.
Regarding Claim 13, Frank, Beall, Putterman, and LaBelle teach the method of claim 11, wherein the environmental type includes at least one of a hot environment, an environment with exercise, a normal environment without exercise, and a cold environment (See Claim 11 Rejection, Frank [0179] describes training conditions for different temperature environment and different patient movement states that fulfill at least one of a hot environment, an environment with exercise, a normal environment without exercise, and a cold environment).
Regarding Claim 14, Frank, Beall, Putterman, and LaBelle teach the method of claim 11, and Beall teaches wherein in the detected region of interest, a face region is detected from the input thermal image of the target person based on a first object detection algorithm ([0107] described facial recognition algorithm) and the at least one facial region is detected as the region of interest within the detected face region based on a second object detection algorithm ([0078] predetermined region of interest shape, relative to a face shape used to identify location of canthi).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, that the facial recognition of Beall utilize multiple detection algorithms as this enables the detection algorithms to be specialized to their relative location.
Response to Arguments
Applicant’s amendments and arguments filed 7/23/2025 with respect to the claim objections have been fully considered and are persuasive. The objection(s) is/are withdrawn.
Applicant’s amendments and arguments filed 7/23/2025 with respect to the 35 USC 112(a) rejections have been fully considered and are persuasive. The rejection(s) is/are withdrawn.
Applicant’s amendments and arguments filed 7/23/2025 with respect to the 35 USC 112(b) rejections have been fully considered and are persuasive. The rejection(s) is/are withdrawn.
Applicant’s amendments and arguments filed 7/23/2025 with respect to the 35 USC 101 rejections have been fully considered, but are not persuasive. Applicant argues on page 8 of the Remarks that the claims are not directed to an abstract idea because the configuration is not a generic idea and solves a technical problem. Examiner notes that step 1 is whether the claim is directed to an abstract idea. Examiner asserts the claim is directed to the use of neural networks to output a body temperature prediction from various inputs. This can be considered a combination of mental processes and mathematical concepts. A practitioner in the art may receive labeled output data and input data and use pen and paper to derive a relationship between the input and output, thus mapping onto a mental process. And a neural network is considered a series of mathematical calculations as noted in claim 2 of example 47 from the 2024 AI SME update. The genericness of the idea and problems in the art are not pertinent at this step.
Applicant argues on page 8 of the Remarks that the claims contain significantly more because the claim implements the prediction with a structured neural network based on real-world sensor input. Further the claims do not recite mere generic computer functions. Examiner respectfully disagrees. In step 2, prong 2A, the pertinent question is whether the claims recite additional elements that integrate the judicial exception into a practical application – not whether the claim language as a whole represents generic computer function. No structure is given for the neural networks, the input interface devices are not limited to sensors and a singular processor can be considered the input interface and thus act as the input interface device. The claims thus can be accomplished by a single processor.
Applicant argues on page 9 of the Remarks that the claims contain significantly more because they are an improvement in the prior art as traditional infrared systems fail to account for external temperature and activity, leading to inaccuracy. Examiner respectfully disagrees. While noting an improvement in the art is a viable path to overcoming 101, Applicant should take care to consider that the improvement cannot be limited to the abstract idea (i.e. the neural network and body temperature prediction). Examiner notes also that a practitioner may receive thermal images as input, make predictions of environmental type and physical activity of the target person in accordance to labeled body temperature data, and establish two mathematical models between the inputs and body temperature using three input thermal input images and labeled output body temperature. If the mathematical model was a neural network, this mental process would align with the claim language. While such a method may not meet the expected accuracy of Applicant, the actual claim language is what is examined, not the intended, real-world processing. Further, systems such as Frank and Putterman above use training of machine learning and environmental contextual data to predict body temperature from thermal images. It is unclear, at this point in prosecution, if the claim language represents an improvement in the art if these references account for external temperature and user activity. Correspondingly, the argument that the claims use machine learning models to improve the functioning of a specific technical process is unpersuasive. Examiner notes that an argument can be made that claims as a whole, with judicial exceptions and additional elements, represent more than what is well-understood, routine, and conventional in the art and thus overcome 35 USC 101 under step 2, prong 2B.
Applicant argues on page 9 of the Remarks that the claims are specific implementations of a novel and practical process, consistent prior rulings. Examiner respectfully disagrees. In view of Frank and Putterman, the novelty of the claim language remains unclear as the main idea of processing input thermal imaging data and contextual data to identify body temperature in a target person is not novel. Applicant may choose to argue the specifics, such as the monitored locations, the processing of the monitoring locations, and include positively claimed hardware to distinguish the claims from an abstract idea and as an improvement in the prior art. However, at this point, the rejection stands.
Applicant’s amendments and arguments filed 7/23/2025 with respect to the 35 USC 103 rejections have been fully considered and are persuasive. The rejection(s) is/are withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Frank, Beall, Putterman, and LaBelle.
Consequently, claims 5-6, 9-10, and 13-14 remain rejected due to their dependency on rejected independent claims 1, 7, and 11.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JAIRO H PORTILLO whose telephone number is (571)272-1073. The examiner can normally be reached M-F 9:00 am - 5:15 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jacqueline Cheng can be reached at (571)272-5596. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JAIRO H. PORTILLO/
Examiner
Art Unit 3791
/JACQUELINE CHENG/Supervisory Patent Examiner, Art Unit 3791
1 2024 AI SME Update, see claim 2 of example 47: neural network… a series of mathematical calculations.
https://www.uspto.gov/sites/default/files/documents/2024-AI-SMEUpdateExamples47-49.pdf
2 2024 AI SME Update, see claim 2 of example 47: neural network… a series of mathematical calculations.
https://www.uspto.gov/sites/default/files/documents/2024-AI-SMEUpdateExamples47-49.pdf
Prosecution Insights