DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 04/06/2026 has been entered.
Response to Amendment
In the Reply filed 04/06/2026, Applicant has amended claims 1 and 9, to include that a subset of the plurality of biometrics determined by the rPPG extraction are “utilized for identifying deceit” and argues that this/those limitation(s) was/were not taught with the reference(s) cited in the previous action dated 11/06/2025. However, the Examiner respectfully disagrees for the reasons laid out below.
Response to Arguments
Applicant's arguments filed 04/06/2026 have been fully considered but they are not persuasive.
As noted above, Applicant has amended the independent claims to specify that a subset of the plurality of biometrics determined by the rPPG extraction are “utilized for identifying deceit” and argues that Tegreene and Yu separately do not teach this feature. In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). Tegreene in Figure 1 shows communications content capture device 101 which paragraph [0184] indicates is a video camera used to track heart rate for deception detection as shown in Figure 6, steps 602. Yu in Figure 1 specifically shows analyzing video content of a human face to track heart rate via rPPG. Therefore, while Yu does not specifically use this heart rate for deception detection, one of ordinary skill in the art could easily apply the method taught in Yu for tracking heart rate via rPPG for use with the deception indica related to heart rate in Tegreene as further detailed below.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3 and 9-11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tegreene, U.S. Publication No. 2013/0139259 in view of Yu et al, “Remote Photoplethysmograph Signal Measurement from Facial Videos Using Spatio-Temporal Networks” (published at https://arxiv.org/abs/1905.02419, July 2019).
Regarding claim 9, Tegreene teaches a system for detecting deception of a subject from a media stream (see Tegreene Figure 1), the system comprising:
a processor; and a memory storing one or more programs for execution by the processor, the one or more programs including instructions (see paragraph [0217]) for:
capturing a media stream of the subject, the media stream including a sequence of frames (see Figure 1, communications content capture device 101 and paragraphs [0027]);
processing each frame of the media stream to track a plurality of biometrics (see Figure 5, biometrics 502-508 and Figure 6, biometrics 602-608 and paragraphs [0175] and [0176]); and
determining whether the subject in the media stream is deceptive based upon changes to respective biometrics (see Figures 5 and 6, step 408 and paragraph [0178]).
Tegreene does not expressively teach utilizing a 3-dimensional convolutional neural network (3DCNN) to perform remote photoplethysmography (rPPG) extraction to determine at least a subset of the plurality of biometrics utilized for identifying deceit.
However, Yu in a similar invention in the same field of endeavor teaches a system for capturing a media stream of a subject, the media stream including a sequence of frames (see Yu Figure 2a, images 1 to T and page 4, “The overall architecture of PhysNet is shown in Figure 2. The input of the network is T-frame face images with RGB channels”) processing each frame of the media stream to track a subset of the biometrics (see Abstract) as taught in Tegreene (see Tegreene Figure 6, step 602) further configured for
utilizing a 3-dimensional convolutional neural network (3DCNN) to perform remote photoplethysmography (rPPG) extraction to determine the subset of the biometrics (see Yu, caption for Figure 2 and Abstract).
One of ordinary skill in the art before the effective filing date of the invention would have found it obvious to combine the teaching of using a 3DCNN to perform rPPG for tracking a biometric as taught in Yu with the system taught in Tegreene, the motivation being to achieve finer and therefore more accurate biometric measurements (see Yu section 2.2).
Tegreene in view of Yu teaches utilizing a 3-dimensional convolutional neural network (3DCNN) to perform remote photoplethysmography (rPPG) extraction to determine at least a subset of the plurality of biometrics utilized for identifying deceit (see Yu, caption for Figure 2 and Abstract as combined with Tegreene paragraph [0184]).
Method claim 1 recites similar limitations as claim 9, and is rejected under similar rationale.
Regarding claim 10, Tegreene in view of Yu teaches all the limitations of claim 9, and further teaches wherein the media stream includes one or more of a visible-light video stream, a near-infrared video stream, a longwave-infrared video stream, a thermal video stream, and an audio stream of the subject (see Tegreene paragraphs [0027] and [0031]).
Method claim 2 recites similar limitations as claim 10, and is rejected under similar rationale.
Regarding claim 11, Tegreene in view of Yu teaches all the limitations of claim 9, and further teaches wherein the plurality of biometrics includes two or more of pulse rate, eye gaze (see Tegreene Figure 5, step 506 and paragraph [0184]), eye blink rate, pupil diameter (see Tegreene Figure 5, step 504 and paragraph [0181]), face temperature, speech, and micro- expressions (see Tegreene Figure 5, step 502 and paragraph [0180]).
Method claim 3 recites similar limitations as claim 11, and is rejected under similar rationale.
Claims 4, 12, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Tegreene, U.S. Publication No. 2013/0139259 in view of Yu et al, “Remote Photoplethysmograph Signal Measurement from Facial Videos Using Spatio-Temporal Networks” (published at https://arxiv.org/abs/1905.02419, July 2019) and Cuestas Rodriguez, U.S. Publication No. 2020/0383621.
Regarding claim 12, Tegreene in view of Yu teaches all the limitations of claim 9, and further teaches wherein the plurality of biometrics includes heart rate (see Tegreene Figure 6, step 602 wherein “heart rate” is used in the art synonymously with “pulse rate”).
Tegreene in view of Yu does not expressively teach wherein the plurality of biometrics includes pupil diameter and face temperature.
However, Cuestas Rodriguez in a similar invention in the same field of endeavor teaches a system for detecting deception of a subject from a media stream comprising a processor (see Cuestas Rodriguez Figure 3, processing module 170) for: capturing a media stream of the subject, the media stream including a sequence of frames; processing each frame of the media stream to track a plurality of biometrics; and determining whether the subject in the media stream is deceptive based upon changes to respective biometrics, wherein the plurality of biometrics includes heart rate (see Figure 3, modules 110-150 and paragraph [0045]) as taught in Tegreene in view of Yu
wherein the plurality of biometrics includes pupil diameter and face temperature (see paragraph [0045]).
One of ordinary skill in the art before the effective filing date of the invention would have found it obvious as a matter of simple substitution to replace the biometrics taught in Tegreene in view of Yu with those taught in Cuestas Rodriguez, to yield the predictable results of successfully determining deception in a person.
Method claim 4 recites similar limitations as claim 12, and is rejected under similar rationale.
Regarding claim 18, Tegreene in view of Yu teaches all the limitations of claim 1, and further teaches wherein the plurality of biometrics consists of a specific subset comprising remote photoplethysmography (rPPG) (see Tegreene Figure 6, step 602 as combined with Yu Abstract) and other biometrics (see Tegreene Figure 6, steps 604-608), and wherein the determining of deception is based on a feature fusion of these biometrics (see Tegreeen paragraph [0044], “The visual indicator 119 may provide an indication of the level of indicia of deception on a aggregate basis (e.g. occurrence metrics for multiple indicia of deception types, such as eye movement, formal language, etc. are combined into a single indicator for an "overall" view of the indicia of deception)”).
Tegreene in view of Yu does not expressively teach wherein the plurality of biometrics consists of a specific subset comprising pupil diameter and face temperature.
However, Cuestas Rodriguez in a similar invention in the same field of endeavor teaches a system for detecting deception of a subject from a media stream comprising a processor (see Cuestas Rodriguez Figure 3, processing module 170) for: capturing a media stream of the subject, the media stream including a sequence of frames; processing each frame of the media stream to track a plurality of biometrics; and determining whether the subject in the media stream is deceptive based upon changes to respective biometrics, wherein the plurality of biometrics includes heart rate (see Figure 3, modules 110-150 and paragraph [0045]) as taught via an rPPG in Tegreene in view of Yu
wherein the plurality of biometrics includes pupil diameter and face temperature (see paragraph [0045]).
One of ordinary skill in the art before the effective filing date of the invention would have found it obvious as a matter of simple substitution to replace the biometrics taught in Tegreene in view of Yu with those taught in Cuestas Rodriguez, to yield the predictable results of successfully determining deception in a person.
Claims 5 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Tegreene, U.S. Publication No. 2013/0139259 in view of Yu et al, “Remote Photoplethysmograph Signal Measurement from Facial Videos Using Spatio-Temporal Networks” (published at https://arxiv.org/abs/1905.02419, July 2019) and Pavlidis, U.S. Publication No. 2003/0012253.
Regarding claim 13, Tegreene in view of Yu teaches all the limitations of claim 9, but does not expressively teach cropping each frame of the media stream to encapsulate a region of interest that includes one or more of a face, cheek, forehead, or an eye.
However, Pavlidis in a similar invention in the same field of endeavor teaches a system for detecting deception of a subject from a media stream (see Pavlidis Figure 1), the system comprising: a processor; and a memory storing one or more programs for execution by the processor, the one or more programs including instructions (see paragraph Figure 1, computing apparatus 14 and paragraph [0051]) for: capturing a media stream of the subject, the media stream including a sequence of frames (see Figure 1, thermal camera 12 and paragraph [0060]);processing each frame of the media stream to track a biometric (see Figure 4, steps 52 and 54); and determining whether the subject in the media stream is deceptive based upon changes to the biometric (see Figure 4, step 56 and paragraph [0063]) as taught in Tegreene in view of Yu and further teaches
cropping each frame of the media stream to encapsulate a region of interest that includes one or more of a face, cheek, forehead, or an eye (see paragraph [0120]).
One of ordinary skill in the art before the effective filing date of the invention would have found it obvious to combine the teaching of cropping each frame to include a region of interest as taught in Pavlidis with the system taught in Tegreene in view of Yu, the motivation being to eliminate background which will not skew the results.
Method claim 5 recites similar limitations as claim 13, and is rejected under similar rationale.
Claims 6 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Tegreene, U.S. Publication No. 2013/0139259 in view of Yu et al, “Remote Photoplethysmograph Signal Measurement from Facial Videos Using Spatio-Temporal Networks” (published at https://arxiv.org/abs/1905.02419, July 2019); Pavlidis, U.S. Publication No. 2003/0012253 and Farag et al, U.S. Publication No. 2008/0045847.
Regarding claim 14, Tegreene in view of Yu and Pavlidis teaches all the limitations of claim 13, but does not expressively teach wherein the region of interest includes two or more body parts.
However, Farag in a similar invention in the same field of endeavor teaches a system for detecting deception of a subject from a media stream (see Farag Figure 12 and Abstract), the system comprising: a processor; and a memory storing one or more programs for execution by the processor, the one or more programs including instructions (see paragraph [0035]) for: capturing a media stream of the subject, the media stream including a sequence of frames (see Figure 12, thermal IR camera and paragraph [0063]); processing each frame of the media stream to track a biometric; and determining whether the subject in the media stream is deceptive based upon changes to the biometric (see paragraph [0040]); and tracking an region of interest including a face (see Figure 12, step s5 and paragraph [0020]) as taught in Tegreene in view of Yu Pavlidis wherein
the region of interest includes two or more body parts (see Figure 7).
One of ordinary skill in the art before the effective filing date of the invention would have found it obvious to combine the teaching of including two body parts (i.e. head and neck) for calculating a biometric as taught in Farag with the system cropping a region of interest for measuring biometrics as taught in Tegreene in view of Yu and Pavlidis, the motivation being to have redundant measurements to improve accuracy for the biometric measurement.
Method claim 6 recites similar limitations as claim 14, and is rejected under similar rationale.
Claims 7, 8, 15, and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Tegreene, U.S. Publication No. 2013/0139259 in view of Yu et al, “Remote Photoplethysmograph Signal Measurement from Facial Videos Using Spatio-Temporal Networks” (published at https://arxiv.org/abs/1905.02419, July 2019) and Sebe et al, U.S. Publication No. 2017/0367590.
Regarding claim 15, Tegreene in view of Yu teaches all the limitations of claim 9, and further teaches using at least two of a visible-light video stream (see Tegreene paragraph [0027]), a near-infrared video stream (see Tegreene paragraph [0034]), and a thermal video stream.
Tegreeene in view of Yu does not expressively teach combining at least two of a visible-light video stream, a near-infrared video stream, and a thermal video stream into a fused video stream.
However, Sebe in a similar invention in the same field of endeavor teaches a system for detecting deception of a subject from a media stream (see Sebe paragraph [0136]), the system comprising: a processor; and a memory storing one or more programs for execution by the processor, the one or more programs including instructions (see paragraph [0139]) for: capturing a media stream of the subject, the media stream including a sequence of frames; processing each frame of the media stream to track a plurality of biometrics (see paragraph [0105]) as taught in Tegreene in view of Yu further configured for
combining at least two of a visible-light video stream, a near-infrared video stream, and a thermal video stream into a fused video stream (see paragraph [0105]).
One of ordinary skill in the art before the effective filing date of the invention would have found it obvious to combine the teaching of fusing streams as taught in Sebe with the system taught in Tegreene in view of Yu, the motivation being to increase the accuracy of the combined deception detection biometrics by ensuring there is no delay between them which would skew the results.
Method claim 7 recites similar limitations as claim 15, and is rejected under similar rationale.
Regarding claim 16, Tegreene in view of Yu and Sebe teaches all the limitations of claim 15, and further teaches wherein the visible-light video stream, the near-infrared video stream, and/or the thermal video stream are combined according to a synchronization device (see Sebe paragraph [0105]).
Method claim 8 recites similar limitations as claim 16, and is rejected under similar rationale.
Claim(s) 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tegreene, U.S. Publication No. 2013/0139259 in view of Yu et al, “Remote Photoplethysmograph Signal Measurement from Facial Videos Using Spatio-Temporal Networks” (published at https://arxiv.org/abs/1905.02419, July 2019) and Kado et al, “Spatial-Spectral-Temporal Fusion for Remote Heart Rate Estimation” (published in IEEE SENSORS JOURNAL, VOL. 20, NO. 19, pages 11688-11697, October 2020).
Regarding claim 17, Tegreene in view of Yu teaches all the limitations of claim 1, and further teaches wherein the 3DCNN processes a video stream (see Yu Abstract) to extract a pulse waveform (see Yu Figure 2(a) and caption).
Tegreene in view of Yu does not expressively teach combining at least two of a visible-light video stream, a near-infrared (NIR) video stream, and a thermal video stream into a fused video stream, and wherein the 3DCNN processes the fused video stream to extract a pulse waveform.
However, Kado in a similar invention in the same field of endeavor teaches a method of using rPPG to determine to extract a biometric (see Kado Figure 3 and Abstract) comprising
combining at least two of a visible-light video stream, a near-infrared (NIR) video stream, and a thermal video stream into a fused video stream (see Figure 2 which shows the spatial-spectral-temporal fusion as summarized in the Abstract) and processing the fused video stream to extract a pulse waveform (see Figure 2, (d) with extracted signals).
One of ordinary skill in the art before the effective filing date of the invention would have found it obvious to combine the teaching of fusing videos of different modalities as taught in Kado with the method taught in Tegreene in view of Yu, the motivation being to improve robustness in the biometric estimation against illumination fluctuations and head motions (see Kado Abstract).
Claim(s) 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tegreene, U.S. Publication No. 2013/0139259 in view of Yu et al, “Remote Photoplethysmograph Signal Measurement from Facial Videos Using Spatio-Temporal Networks” (published at https://arxiv.org/abs/1905.02419, July 2019) and de Haan et al, “Robust Pulse Rate From Chrominance-Based rPPG” (published in IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL.60, NO.10, pages 2878-2886, OCTOBER 2013).
Regarding claim 19, Tegreene in view of Yu teaches all the limitations of claim 1, but does not expressively teach further comprising partitioning the sequence of frames into partially overlapping subsequences, wherein the 3DCNN produces a pulse waveform for each subsequence, and wherein a pulse aggregation system applies a Hann function to combine the overlapping portions to generate a continuous aggregated pulse waveform.
However, de Haan in a similar invention in the same field of endeavor teaches a method of analyzing a sequence of frames (see de Haan section II, “For the following analysis, we assume that an area of skin is illuminated by a light source and registered with an RGB video camera”) for performing rPPG extraction (see section I, “In Section II of this paper, we shall analyze how motion enters the pulse signal. From this analysis, new techniques for rPPG emerge that shall be shown superior to all earlier methods both in signal-to-noise ratio (SNR) and motion robustness”) for determining a subset of biometrics (see section III, A, “To extract the pulse rate from the output signals of the different methods”) as taught for the 3DCNN analyzing a sequence of frames in Tegreene in view of Yu comprising
partitioning the sequence of frames into partially overlapping subsequences, wherein the analysis produces a pulse waveform for each subsequence (see section II, “To produce a pulse signal that is independent of the presumed stationary color of the light source and its brightness level, we can normalize each color channel C by dividing its samples by their mean over a temporal interval (2) where μ(Ci) can be a running average centered around image i, or an average of an overlap-add processing interval that includes image I” and Figure 5 which shows overlapped pulses), and wherein a pulse aggregation system applies a Hann function to combine the overlapping portions to generate a continuous aggregated pulse waveform (see caption for Figure 5).
One of ordinary skill in the art before the effective filing date of the invention would have found it obvious to combine the teaching of using overlapping subsequences and subsequently aggregating pulses from the subsequences as taught in de Haan with the method using a 3DCNN taught in Tegreene in view of Yu, the motivation being to increase signal-to-noise and in the output signal as well as making it robust to motion (see de Haan section I, “From this analysis, new techniques for rPPG emerge that shall be shown superior to all earlier methods both in signal-to-noise ratio (SNR) and motion robustness”).
Claim(s) 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tegreene, U.S. Publication No. 2013/0139259 in view of Yu et al, “Remote Photoplethysmograph Signal Measurement from Facial Videos Using Spatio-Temporal Networks” (published at https://arxiv.org/abs/1905.02419, July 2019); Gottemukkula et al, U.S. Publication No. 2015/0078629 and Kircher et al, U.S. Publication No. 2010/0324454.
Regarding claim 20, Tegreene in view of Yu teaches all the limitations of claim 1, and further teaches wherein the tracking the plurality of biometrics includes tracking features of eyes (see Tegreene Figure 5, steps 504 and 506).
Tegreene in view of Yu does not expressively teach wherein processing each frame comprises cropping the media stream to encapsulate an eye region, and wherein the plurality of biometrics includes simultaneous tracking of eye gaze, eye blink rate, and pupil diameter via the 3DCNN.
However, Gottemukkula in a similar invention in the same field of endeavor teaches a method of tracking a biometric (see Gottemukkula Figure 1A, step 102) in a media stream including sequence of frames (see paragraph [0005]) and processing each from for tracking feature of eyes (see paragraph [0052]) as taught in Tegreene in view of Yu wherein processing each frame comprises
cropping the media stream to encapsulate an eye region (see paragraph [0052]).
One of ordinary skill in the art before the effective filing date of the invention would have found it obvious to combine the teaching of cropping a media stream for eye analysis as taught in Gottemukkula with the method taught in Tegreene in view of Yu, the motivation being to not expend processing resources for eye analysis in non-eye regions.
Tegreene in view of Yu and Gottemukkula does not expressively teach wherein the plurality of biometrics includes simultaneous tracking of eye gaze, eye blink rate, and pupil diameter via the 3DCNN.
However, Kircher in a similar invention in the same field of endeavor teaches a method of processing a subject in a media stream (see Kircher Figure 1B, eye tracking device 142 and paragraph [0098], “An eye tracking device 142 (e.g., the Arrington ViewPoint Eye Tracker, discussed above; computer 101 of FIG. 1a executing eye tracking software; etc.), operably connected to the first computer, can monitor a plurality of eye movements as the test subject 150 views a test item from the plurality of ordered test items”) to track a plurality of biometrics (see paragraph [0072]) for deception detection (see Abstract) as taught for the 3DCNN analyzing frames of a media stream taught in Tegreene in view of Yu and Gottemukkula wherein
the plurality of biometrics includes simultaneous tracking of eye gaze, eye blink rate, and pupil diameter (see paragraph [0072]).
One of ordinary skill in the art before the effective filing date of the invention would have found it obvious to combine the teaching of tracking the various eye biometrics as taught in Kircher with the method of analzying images with a 3DCNN as taught in Tegreene in view of Yu and Gottemukkula, the motivation being to increase the accuracy of the deception detection through analyzing more biometrics.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Chen et al, “Video-Based Heart Rate Measurement: Recent Advances and Future Prospects” (published in IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 68, NO. 10, pages 3600-3615, October 2019) generally teaches using rPPG for deception detection (see Chen Figure 6, “polygraph” usage) that is done through multimodal fusion of visible and thermal camera videos (see section V, C).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CASEY L KRETZER whose telephone number is (571)272-5639. The examiner can normally be reached M-F 10:00-7:00 PM Pacific Time.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David Payne can be reached at (571)272-3024. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CASEY L KRETZER/Primary Examiner, Art Unit 2635
Prosecution Insights