Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
2. Applicant has amended the claims and argues that the amendments apply a trained deep learning model to thermal image data. Also, “A human cannot perform this process because a human cannot see in the thermal spectrum, isolate minute temperature fluctuations around the nostrils caused by breathing and apply a complex computational model to this data to predict a precise respiration rate” and improves the field of physiological monitoring by enabling non-contact, automated, and potentially more accurate respiration analysis. Examiner disagrees. In making this decision to maintain the rejection of the claims, Examiner believes more likely than not that the claim is ineligible (See Reminders on evaluating subject matter eligibility of claims under 35 USC § 101, USPTO Memorandum August, 4, 2025.). In the memo, the Examiner is reminded to consult the specification to determine whether the disclosed invention improves technology or a technical field and review the claim to ensure it reflects the disclosed improvement. Further, the Examiner should consider whether the technological limitations are being used as a tool to improve the recited judicial exception (e.g., automating a manual business process) or whether the claim provides an improvement to technology or a technical field citing “Compare Recentive Analytics, Inc. v. Fox Corp., 134 F.4th 1205 (Fed. Cir. 2025) (steps incidental to automating an abstract idea were not sufficient to confer eligibility), with USPTO Example 47, claim 3 (the claim as a whole improved the technical field of network intrusion detection).”
Here, Applicant’s specification makes several references to the use of deep learning models that “can be used to identify different behavioral cues and/or predict the likelihood of deception associated with a detected behavioral cue.” (Spec. [0024 and similarly in “applying deep learning and/or computer vision techniques” [0031], “deep learning models used for predicting deception analysis are trained” and “a deep learning model can be trained for predicting respiration rate using thermal sensor data” [0044], and applying deep learning techniques…one of more metrics can be determined to identify relevant behavioral cues [0053], and “thermal sensor data prepared at 1003 is used as input to a machine learning model to predict respiration rate” [0092]. From this, Examiner notes the holding in Recentive where:
Machine learning is a burgeoning and increasingly important field and may lead to patent-eligible improvements in technology. Today, we hold only that patents that do no more than claim the application of generic machine learning to new data environments, without disclosing improvements to the machine learning models to be applied, are patent ineligible under § 101.
Conversely, Claim 3 of Example 47 is instructive where the Claim 3 not only recites training an artificial neural network (ANN) but also claims two algorithms which optimize it followed by employing the model to not only output anomaly data (as in Claim 2 of Example 47) but also detect anomalies, determine at least one detected anomaly is associated with one or more malicious network packets, detect a source address, drop the malicious packets in real time, and block future traffic from the source address.
Here, Examiner believes that Applicant’s specification employs deep learning models as a tool because several citations suggest expressly state “apply” or “can be used” and no details are disclosed as to specifically how deep models are trained or what specific deep models are actually employed. Likewise, Examiner believes on the continuum of ineligibility/eligibility suggested by Recentive vs. Example 47 Claim 3, Examiner leans heavily towards ineligibility because, akin to Recentive, Applicant’s specification provides several areas where deep learning can serve this technological field to identify behavioral cues. Conversely, Applicant’s Claim 1 does not go as far as Claim 3 to apply to a specific outcome or specify the particular model used (above) but merely recites how the output joins one or more other metrics help determine an indicator. Lastly, in light of the Applicant’s remarks that “[a] human cannot perform this process because a human cannot see in the thermal spectrum, isolate minute temperature fluctuations around the nostrils caused by breathing and apply a complex computational model to this data to predict a precise respiration rate” and improves the field of physiological monitoring by enabling non-contact, automated, and potentially more accurate respiration analysis,” in reminiscent of elements of McRO, Inc. v. Bandai Namco Games America Inc., 837 F.3d 1299 (Fed. Cir. 2016). McRO concerned automating part of a pre-existing 3-D automation method where animation of a character and lip synchronization was accomplished by an animator with the assistance of a computer to set appropriate morph weights at important times called keyframes instead of at every frame replacing a formerly manual, labor-intense process. The court recognized the computer automation is realized by improving the prior art through "the use of rules, rather than artists, to set the morph weights and transitions between phonemes." (Patentability Op., 55 F.Supp.3d at 1227). Applicant’s remarks are McRO-like when they allege that deep learning models potentially exceed human capabilities to predict a precise respiration rate and improve the field through automation and precision. However, Applicant’s specification and claims lack McRO’s use of claimed specific rules when generically recited as a trained model applied to sensor data.
Thus, on balance claims are deemed to be ineligible.
Applicant argues that Waldorf fails to teach applying learning models to thermal imaging data. Examiner agrees and directs Applicant to the teachings of Waldorf in view of U.S. Pat. Pub. No. 2018/0116578 to Tzvielli.
Claim Rejections - 35 USC § 101
4. 35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
5. Claims 1-20 are rejected under 35 U.S.C. § 101 because the claimed invention is directed to an abstract idea without significantly more.
6. Step 1
Claims 1-13 are directed to a method meeting the requirements for Step 1.
Claims 14-20 are directed to an apparatus/system meeting the requirements for Step 1.
7. Step 2A Prong 1
In independent Claim 1 and similarly in Claims 14 and 20, the following italicized step recite an abstract idea of abstract idea of determine an indicator associated with a likelihood the subject is being deceptive which is a mental process as the steps can be performed in the mind and/or with the aid of pencil and paper. For example, in many fields of endeavor such as an interviewing for a job, parenting a child, or interrogating a suspect, an interviewer/parent/interrogator looks at the eyes, the gestures, the body language used by the interviewee/child/suspect. From this data they determine metrics of how fast, how slow, gaze direction, gaze duration to quantify the character of the behavior. Finally, using all the data and metrics use an indicator of an overall assessment or conclusion for the job interviewee/child/suspect as to the chances they are lying/being deceptive.
Claim 1 is selected as representative of Claims 14 and 20:
A method, comprising:
receiving sensor data of a subject, wherein the sensor data includes eye tracking data and visible light image data; and thermal image data of aan area associated with nostrils of the subject;
using one or more processors to automatically analyze the sensor data to determine one or more metrics including by applying a trained deep learning model to the thermal image data to predict a respiration rate of the subject; and
using the one or more metrics including the predicted respiration rate to determine an indicator associated with a likelihood the subject is being deceptive.
The Courts have determined that methods which can be performed mentally, or which are the equivalent of human mental work, are unpatentable abstract ideas the "basic tools of scientific and technological work" that are open to all. Cybersource Corp. v. Retail Decisions, Inc. (Fed. Cir. 2011). Additionally, the claim can be viewed as also reciting abstract data collection, recognition, and storage found to be abstract in Electric Power Group, LLC v. Alstom S.A., 830 F.3d 1350 (Fed. Cir. 2016); In re TLI Communications LLC, 823 F.3d 607; and Content Extraction and Transmission. v. Wells Fargo Bank, 776 F.3d 1343 (Fed. Cir. 2014).
As explained in the MPEP and the October 2019 Update, in situations like this where a series of steps recite judicial exceptions, examiners should combine all recited judicial exceptions and treat the claim as containing a single abstract idea for purposes of further eligibility. See MPEP 2106.04 and 2106.05(II). Thus, for purposes of further discussion, the abstract idea is a single abstract idea of a mental process. The amended claims reciting thermal image data and trained deep learning models are deemed extra-solution data gathering and extra-solution application of learning models as tools which does not change the determination of the abstract mental process for the step of “determining an indicator associated with a likelihood the subject is being deceptive.
8. Step 2A Prong II
The recited claims fail to recite any elements that provide a practical application. Applicant's specification does not disclose new data sensor technology; new ways to communicate data; new metrics; new sources; or new indicators, but discloses only known technologies used in their customary ways. Here, the processor, memory, and instructions are recited so generically (no details whatsoever are provided other than in name only) that they represent no more than mere instructions to apply the judicial exception on a computer. Examiner finds in the specification, the computing environment to comprise a general purpose digital processor (Spec. 0119]) and “[a]s is well-known in the art, primary storage can be used as a general storage area: (Spec. [0120]). “Also as is well known in the art, primary storage typically includes basic operating instructions, program code, data and objects used by the processor 1702 to perform its functions (e.g., programmed instructions).” (Spec. [0120]) where “the computer-readable medium is any data storage device that can store data which can thereafter be read by a computer system.” (Spec. 0125]). Sensors comprise well-known eye tracking sensor 301, RGB camera sensor 311, thermal sensor 321, microphone 331, display 341, and audio output 351.” (Spec. [0039]). Where data is collected on a human subject, “the only physical contact that may be required is the use of a conventional input device such as a mouse, touchpad, and/or touchscreen.” (Spec. 0040]). Lastly, the claim recites metrics and indicators where “In some embodiments, one or more of the metrics are provided at least partially by the sensor equipment and/or by applying deep learning and/or computer vision techniques. For example, visible image data captured using an RGB camera can be fed into a trained deep learning model to predict the subject's heart rate. Similarly, cropped thermal image data of the area surrounding a subject's nostrils can be fed into a trained deep learning model to predict the subject's respiration rate. In some embodiments, computer vision techniques are applied, for example, as another technique for predicting the subject's respiration rate using captured thermal images.” (Spec. [0031]) where the metrics are of routine pause time, blink rate, pupil features, gaze fixation, gaze target, respiration rate (Spec. [0028]). “Indicators can display in real time the subject's response pause time, respiration rate, blink rate, pupil features, and/or gaze fixation duration, among other metrics. The determined metrics can be associated with responses and/or questions presented to the subject during an interview.” (Spec. [0025]). Examiner deems the sensors, sensor data, metrics, and indicators are the result of the use of pre-existing means of data collection subjected to established analytical technics used in their conventional ways, and as such, comprise extra-solution data gathering, processing, and storage activity. The amended claims reciting thermal image data and trained deep learning models are deemed extra-solution data gathering and extra-solution application of learning models as tools which does not change the determination of the abstract mental process for the step of “determining an indicator associated with a likelihood the subject is being deceptive. Thus, combined with the computing elements provide a generic computing environment to carry out the abstract idea.
Even when these limitations are viewed in combination, the additional elements in this claim do no more than automate the abstract data processing steps needed to be performed in using the one of more computer components as tools. While this type of automation is an improvement in a general sense as opposed to performance manually, there is no change to the computer and other technology that are recited in the claim as automating the abstract ideas, and thus this claim cannot improve computer functionality or other technology. Thus, Claim 1, and similarly Claim 14 and 20, lack the eligibility requirements of Step 2 Prong II.
9. Step 2B
According to the 2019 PEG, in addition to the considerations discussed in Step 2A, an additional consideration indicative of an inventive concept (aka “significantly more”) is the addition of a specific limitation other than what is well-understood, routine, conventional activity in the field (MPEP 2106.05(d)). Conversely, an additional consideration not indicative of an inventive concept is simply appending well-understood, conventional activities previously known to the industry, specified at a high level of generality, to the abstract idea (MPEP 2106.05(d) and Berkheimer Memo, April 20, 2018). Thus, the additional elements evaluated under Step 2A are re-evaluated in Step 2B to determine if they are more than what is well-understood, routine, conventional activity in the field.
Claim 1, and similarly for the processor and memory and medium of Claims 14 and 20, do not recite additional elements, individually or in combination, that amount to significantly more than the abstract idea. As discussed above with respect to the lack of a practical application, the additional elements in the claim amount to no more than mere instructions to apply the exception using a generic computer components. The same analysis applies here, i.e., mere instructions to apply an exception using generic computer component(s) cannot provide an inventive concept in Step 2B.
Further, under the 2019 PEG, a conclusion that an additional element is insignificant extra-solution activity in Step 2A should be reevaluated in Step 2B to determine if it is more than what is well-understood, routine, conventional activity in the field.
Examiner has identified the sensor data including eye tracking data and visible light image data, thermal data, and of one or more metrics as extra-solution data collection and data processing. This has been deemed to be well-known, routine, and conventional under Electric Power Group and in the MPEP (See receiving or transmitting data over a network (MPEP 2106.05(d)(II)(i), performing repetitive calculations (MPEP 2106.05(d)(II)(ii), and electronic recordkeeping (MPEP 2106.05(d)(II)(iii)). In Electric Power Group, looking at the claim as a whole, the court held the claim to be directed to abstract collecting and analyzing data even though the type of data collected was limited to data from an electric power grid. The claim was not patent-eligible, because it also failed to include an inventive concept. The court noted that the claims did not relate to a new source or type of information, or a new algorithm for analyzing the information. Likewise, here, while the data may pertain to the field of deception detecting, it utilized known sources, sensing data collection, and outcomes to provide an inventive concept. Thus, Claim 1 is ineligible. Independent Claims 14 and 20 inherit the same abstract idea as Claim 1 and are similarly ineligible.
10. Dependent Claims
Claims 2-5, 9-10, and 15-17 further recite additional extra-solution metrics or indicators. Claims 6, 8, 11, 12, and 18 further recite more abstract analysis. Claims 7 and 19 further recite additional extra-solution sensor data collection. Claim 13 recites extra-solution storing and retrieving information in memory (MPEP 2106.05(d)(II)(iv)).
Claim Rejections - 35 USC § 103
11. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
12. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
13. The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
14. Claims 1-3, 8-9, 11-12, 14-15, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Pat. Pub. No. 2009/0216092 to Waldorf in view of U.S. Pat. Pub. No. 2018/00992586 to Tzvieli.
In Reference to Claims 1, 14, and 20
Waldorf discloses one or more processors (Fig. 1 104 [0021]); and
a memory coupled to the one or more processors, wherein the memory is configured to provide the one or more processors with instructions {computer program} which when executed cause the one or more processors (database 103 [0031]) to or for:
receiv[ing] sensor data of a subject, wherein the sensor data includes eye tracking data and visible light image data (Fig. 2 205-206, see also pupil detection [0025] and eye gaze tracking [0026]);
us[ing] one or more processors to automatically analyze the sensor data to determine one or more metrics (Fig. 2 208 responses [0031] pupil size metric [0031] and respiration [0032, 0033]); and
us[ing] the one or more metrics to determine an indicator associated with a likelihood the subject is being deceptive (Fig. 2 211 indication of deception [0031]).
Waldorf discloses the invention substantially as claimed. However, the reference does not explicitly disclose thermal image data of an area associated with nostrils of the subject and of applying a trained deep learning model to the thermal image data. One of skill in the art would be aware of the teachings of Tzvieli.
Tzvieli teaches of selecting stress based on thermal measurements of the face ITitl.) wherein using a thermal camera [0037] to collect data for detection of physiological responses [0053]. Ways to accomplish detection is by the use of computers and performing calculation that involve use of a model trained using machine learning methods ([0056, 0062, 0065]). In some embodiments, the system selected a stressor to take measurements of one or more regions of the user’s forehead, nose and/or below the nostrils ([0135]) where exhail streams from the nose and/or mouth flow and it’s thermal measurements are indicative of the user’s breathing ([0232] see also [0070, 0221]). Tzvieli invents these methods in order to detect atypical behavior of the use based on the feature values ([Abstr.).
The Supreme Court in KSR Int'l Co. v. Teleflex Inc., 550 U.S. 398, 415-421, 82 USPQ2d 1385, 1395-97 (2007) identified a number of rationales to support a conclusion of obviousness
(A) Combining prior art elements according to known methods to yield predictable results;
(B) Simple substitution of one known element for another to obtain predictable results;
(C) Use of known technique to improve similar devices (methods, or products) in the same way; and
(D) Applying a known technique to a known device (method, or product) ready for improvement to yield predictable results.
Here, it would require only routine skill in the art to modify the estimation of any of the metrics of Waldorf to include respiration rate {breathing rate} with the thermal data and trained machine learning models of Tzvieli to achieve the predictable result of predicting a respiration rate of a subject. The Courts have held that combining prior art elements according to known methods to yield predictable results to be indicia of obviousness.
In Reference to Claim 2
Waldorf discloses wherein the one or more metrics include a pupil feature (pupil size [0013]).
In Reference to Claims 3 and 15
Waldorf discloses wherein the one or more metrics include a heart rate ([0015]).
In Reference to Claim 8
Waldorf discloses using the one or more processors to automatically analyze the thermal sensor data to determine a respiration rate (known breath rate, respiration rate ([0015, 0019, 0033]).
In Reference to Claim 9
Whether the indicator is a percentage value or a Boolean value is non-functional descriptive material as the form of the indicator does not have result in any outcome in the claim to operate whether the indicator is a percentage or Boolean or not. Further, the various rates and determinations of Waldorf would perform equally as well as indicators.
In Reference to Claims 11 and 12
Waldorf discloses automatically detecting a behavioral cue associated with the determined one or more metrics (pupil diameter [0033]) and automatically predicting an emotion response associated with the detected behavioral cue (sensing deep breaths associated with pupil diameter the system reminds subject to remain still or to breath normally [0033]).
15. Claim 4-5, 7, 16-17, and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Waldorf, Tzvieli further in view of U.S. Pat. Pub. No. U.S. Pat. Pub. No. 2008/0260212 to Moskal.
In Reference to Claims 4-5 and 16-17
Waldorf discloses the invention substantially as claimed. However, the reference does not explicitly disclose one or more micro-expressions that include a shoulder shrug.
One of skill in the art would be aware of the system for indicating deceive and verity of Moskal (Titl.). According to Moskal, known emblems of body movements have clearly defined meaning for example “raising the shoulders, turning palms upward, raising eyebrows, dropping the upper eyelid, making a U-shaped mouth, and tilting the head sideways.” ([0054]). Examiner deems the shoulder movement and mouth movement equivalent to micro-expressions of shoulder shrugs and lip shrugs.
The Supreme Court in KSR Int'l Co. v. Teleflex Inc., 550 U.S. 398, 415-421, 82 USPQ2d 1385, 1395-97 (2007) identified a number of rationales to support a conclusion of obviousness
(A) Combining prior art elements according to known methods to yield predictable results;
(B) Simple substitution of one known element for another to obtain predictable results;
(C) Use of known technique to improve similar devices (methods, or products) in the same way; and
(D) Applying a known technique to a known device (method, or product) ready for improvement to yield predictable results.
It would require only routine skill in the art to modify Waldorf with known additional metrics assessed from the face and body of a subject of Moskal to enhance the accuracy of the deception detection by the utilization of the additional data points. The Courts have held that combining prior art elements according to known methods to yield predictable results to be indicia of obviousness.
In Reference to Claims 7 and 19
Waldorf discloses wherein the sensor data includes thermal sensor data ([0032]). Moskal teaches a deceit indication is a function of an audio deceit indicator ([0018, 0066, 0104]).
16. Claims 6, 10, 13, and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Waldorf, Tzvieli further in view of U.S. Pat. Pub. No. 2021/0186395 to Zakariaie.
In Reference to Claims 6, 13, and 18
Waldorf discloses the invention substantially as claimed to include matching presently measured responses to retrieved normal test results ([0031]) yet Waldorf does not disclose that the normal test results are of the subject or of remote storage of metric and indicators. However, one of skill in the art would be aware of the ocular system and baseline metrics of subject in Zakariaie.
Zakariaie teaches of deception detection wherein the system attains baseline metrics (Fig. 3 [0073]) based on metrics of gaze location, gaze trajectory, pupil dilation duration [0067]) used in the determination of truthfulness or deceptiveness (Fig. 3). Further, Zakariaie teaches of a computing device may be a cloud-based computing device disposed remote from a standoff device located with the subject ([0011]).
It would require only routine skill in the art to modify the analysis of Waldorf with the baseline metrics and remote cloud storage of Zakariaie in order to compare changes in metrics to baseline values to achieve the predictable result of determining deception of truthfulness of a subject’s responses. The Courts have held that the use of a known technique to improve similar devices (methods, or products) in the same way to be indicia of obviousness.
In Reference to Claim 10
Examiner construes interface component in light of the Specification as the real-time display of the metric or micro-expression. Waldorf discloses display means for presenting an examiner with a visual representation of said dynamic response of said pupil size during said sequence of digital images (Waldorf Claim 4). As to a baseline metric, Zakariaie teaches of baseline metrics (Fig. 3 [0073]) that can be displayed by modifying Waldorf.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
18. Any inquiry concerning this communication or earlier communications from the examiner should be directed to Paul A. D’Agostino whose telephone number is (571) 270-1992.
19. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
20. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David Lewis can be reached on (571) 272-7673. The fax phone number for the organization where this application or proceeding is assigned is 571-270-2992.
/PAUL A D'AGOSTINO/Primary Examiner, Art Unit 3715