DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This action is in response to the remarks filed on 12/30/2025. The amendments filed on 12/30/2025 are entered.
The previous objection of claim 9 has been withdrawn in light of the applicant’s remarks/amendments.
The previous rejections of claims 8 and 18 have been withdrawn in light of the applicant’s remarks/amendments.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 21 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 21 recites the limitation "the one or more quality criteria" in line 1. There is insufficient antecedent basis for this limitation in the claim.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 7-15, 17-18, and 21-23 are rejected under 35 U.S.C. 103 as being unpatentable over Kado, S., et al., (“Spatial-Spectral-Temporal Fusion for Remote Heart Rate Estimation,” IEEE Sensors Journal. Vol 20(19), 2020. P. 1-10) hereinafter Kado (see NPL reference of applicant’s submitted IDS of 10/04/2024 for citations), in view of Kaestle (U.S. Pub. No. 20170319114) hereinafter Kaestle.
Regarding claim 1, primary reference Kado teaches:
A method (abstract) comprising:
capturing a first set of image frames, wherein the first set of image frames includes a representation of a user's face (pages 3-5, III. Proposed HR Estimation Method, particularly NIR sequence acquisition of image frames of a user’s face as in figure 1; see also figures 2-4; page 5, IV Experimental Results, A. Setups and data collection);
identifying at least one skin patch of the user's face that is represented in the first set of frames (page 3, III Proposed HR Estimation Method, particularly Face landmark detection and tracking and Face patch sampling and signal extraction, for the identifying of a face path (analogous to skin patch on the user’s face) within the G channel of the RGB video frames; see also pages 4-5);
determining a light source configuration and transmitting the light source configuration to a first light source (page 5, IV Experimental Results, A. Setups and data collection, figure 3, lighting setup provides for a light source configuration determined, and provided to the light source; page 8, Scene 5, light source illumination based upon light fluctuations or low light);
illuminating, by the first light source, the at least one skin patch according to the light source configuration (page 5, IV Experimental Results, A. Setups and data collection, figure 3, lighting setup provides for a light source configuration determined, and provided to the light source;);
capturing a second set of image frames, wherein the second set of image frames includes a representation of the at least one skin patch illuminated by the first light source according to the light source configuration (pages 3-5, III. Proposed HR Estimation Method, particularly RGB sequence acquisition of image frames of a user’s face as in figure 1; see also figures 2-4, with figure 3 showing lighting setups for acquiring second set of image frames using the RGB-NIR camera; page 5, IV Experimental Results, A. Setups and data collection); and
processing one or more of the second set of image frames using remote photo- plethysmography, rPPG (page 3, Face Patch pair-based HR estimation, “Fast ICA [51] is then performed to estimate the PPG signal from the intensity traces”, this forms a remote PPG signal acquisition as it is estimated from the remotely acquired video signals; see also figure 1, abstract; pages 3-5, Proposed HR Estimation Method and Experimental Results; figures 2-4).
Primary reference Kado fails to teach:
determining a light source configuration and transmitting the light source configuration to a first light source based on the identifying
However, the analogous art of Kaestle of an oxygenation screening system using a remote PPG (abstract) teaches:
determining a light source configuration and transmitting the light source configuration to a first light source based on the identifying ([0043], light power is controlled by the control unit for operation of the detection unit in the optimum operating point; [0121], light source; [0122], “The light source 10 can be part of a control loop which sets an optimal operating point of the image sensor of the imaging unit 2. Optimal in this context refers to an output signal without signal clipping, no saturation of individual detectors of the image sensors and a good signal-to-noise ratio at least for the detector area corresponding to first and/or second marker area.” The control loop based upon the image sensor of the imaging unit forms a “based on the identifying” of the region of interest of the imaged skin patch in the combined prior art invention with Kado)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the remote PPG imaging acquisition and processing system of Kado to incorporate the determination of a light source configuration as taught by Kaestle because it enables configuration of illumination settings to provide an optimal output signal without signal clipping, no saturation of individual detectors of the image sensors and a good signal-to-noise ratio at least for the detector area (Kaestle, [0122]).
Regarding claim 2, the combined references of Kado and Kaestle teach all of the limitations of claim 1. Primary reference Kado further teaches:
wherein the capturing of the first and second sets of image frames is performed by a single camera (pages 3-5, III. Proposed HR Estimation Method, particularly RGB sequence acquisition of image frames of a user’s face as in figure 1; see also figures 2-4, with figure 3, showing RGB-NIR camera for capturing RGB sequence and is a single dual sensor camera; page 5, IV Experimental Results, A. Setups and data collection).
Regarding claim 3, the combined references of Kado and Kaestle teach all of the limitations of claim 2. Primary reference Kado further teaches:
wherein the camera comprises a red-green-blue, RGB, sensor, and wherein the capturing of the first and second sets of image frames comprises capturing images in the visible light spectrum using the RGB sensor (pages 3-5, III. Proposed HR Estimation Method, particularly RGB sequence acquisition of image frames of a user’s face as in figure 1; see also figures 2-4, with figure 3, showing RGB-NIR camera for capturing RGB sequence and is a single dual sensor camera; page 5, IV Experimental Results, A. Setups and data collection).
Regarding claim 4, the combined references of Kado and Kaestle teach all of the limitations of claim 2. Primary reference Kado further teaches:
wherein the camera comprises a near-infrared, NIR, sensor, and an RGB sensor, wherein the capturing of the first set of image fames comprises capturing images in the NIR spectrum using the NIR sensor, and wherein the capturing of the second set of image frames comprises capturing images in the visible light spectrum using the RGB sensor (pages 3-5, III. Proposed HR Estimation Method, particularly RGB and NIR sequence acquisition of image frames of a user’s face as in figure 1; see also figures 2-4, with figure 3, showing RGB-NIR camera for capturing RGB sequence and NIR sequence image frames using the RGB sensor and NIR sensor in the camera and is a single dual sensor camera; page 5, IV Experimental Results, A. Setups and data collection).
Regarding claim 7, the combined references of Kado and Kaestle teach all of the limitations of claim 4. Primary reference Kado further teaches:
further comprising: illuminating the user's face with a NIR light source to facilitate identifying the at least one skin patch (page 5, IV Experimental Results, A. Setups and data collection, figure 3, lighting setup provides for a light source of a NIR LED light; page 3, III Proposed HR Estimation Method, particularly Face landmark detection and tracking and Face patch sampling and signal extraction, for the identifying of a face path (analogous to skin patch on the user’s face)).
Regarding claim 8, the combined references of Kado and Kaestle teach all of the limitations of claim 1. Primary reference Kado further teaches:
further comprising: selecting at least some of the second set of image frames for rPPG processing, wherein the selecting is based on one or more quality criteria (figure 2, histogram estimation with reliability determined for histogram estimation for only suitable face patch pairs being used in HR estimation; see page 3, III. Proposed HR Estimation Method, including particularly 4) Histogram voting and fusion).
Regarding claim 9, the combined references of Kado and Kaestle teach all of the limitations of claim 1. Primary reference Kado further teaches:
wherein identifying the at least one skin patch comprises:
detecting one or more facial features represented in the first set of image frames (page 3, III Proposed HR Estimation Method, particularly Face landmark detection and tracking and Face patch sampling and signal extraction, for the identifying of a face path (analogous to skin patch on the user’s face) within the G channel of the RGB video frames; see also pages 4-5); and
identifying the at least one skin patch based on a location of the detected one or more facial features (page 3, III Proposed HR Estimation Method, particularly Face landmark detection and tracking and Face patch sampling and signal extraction, for the identifying of a face path (analogous to skin patch on the user’s face) within the G channel of the RGB video frames; see also pages 4-5).
Regarding claim 10, the combined references of Kado and Kaestle teach all of the limitations of claim 1. Primary reference Kado further teaches:
further comprising:
determining one or more heartbeat parameters based on the rPPG processing (page 3, Face Patch pair-based HR estimation, “Fast ICA [51] is then performed to estimate the PPG signal from the intensity traces”, this forms a remote PPG signal acquisition and an output is achieved of heart rate estimation; see also figure 1, abstract; pages 3-5, Proposed HR Estimation Method and Experimental Results; figures 2-4); and
generating a control signal based on the one or more heartbeat parameters to initiate an action associated with the one or more heartbeat parameters(page 3, Face Patch pair-based HR estimation, “Fast ICA [51] is then performed to estimate the PPG signal from the intensity traces”, this forms a remote PPG signal acquisition and an output is achieved of heart rate estimation. Pages 3-5, Histogram voting and fusion and Final HR estimate calculation include reliability measurements based on the heart rate parameter measurement and initiates further processing to achieve a final output associated with the parameter; see also figure 1, abstract; pages 3-5, Proposed HR Estimation Method and Experimental Results; figures 2-4; page 8, Scene 5, HR is measured over light fluctuations and an accuracy is associated with the heart rate measurement).
Regarding claim 11, primary reference Kado teaches:
A system (abstract) comprising:
at least one camera for capturing a first set of image frames, wherein the first set of image frames includes a representation of a user's face (pages 3-5, III. Proposed HR Estimation Method, particularly NIR sequence acquisition of image frames of a user’s face as in figure 1; see also figures 2-4, with figure 3, showing RGB-NIR camera for capturing NIR sequence; page 5, IV Experimental Results, A. Setups and data collection);
a facial feature detector for identifying at least one skin patch of the user's face that is represented in the first set of image frames (page 3, III Proposed HR Estimation Method, particularly Face landmark detection and tracking and Face patch sampling and signal extraction, for the identifying of a face path (analogous to skin patch on the user’s face) within the G channel of the RGB video frames; see also pages 4-5);
a light source configurator for determining a light source configuration and transmitting the light source configuration to a first light source (page 5, IV Experimental Results, A. Setups and data collection, figure 3, lighting setup provides for a light source configuration determined, and provided to the light source; page 8, Scene 5, light source illumination based upon light fluctuations or low light); and
the first light source for illuminating the at least one skin patch according to the light source configuration (page 5, IV Experimental Results, A. Setups and data collection, figure 3, lighting setup provides for a light source configuration determined, and provided to the light source;);
wherein the at least one camera is configured to capture a second set of image frames, wherein the second set of image frames includes a representation of the at least one skin patch illuminated by the first light source according to the light source configuration (pages 3-5, III. Proposed HR Estimation Method, particularly RGB sequence acquisition of image frames of a user’s face as in figure 1; see also figures 2-4, with figure 3 showing lighting setups for acquiring second set of image frames using the RGB-NIR camera; page 5, IV Experimental Results, A. Setups and data collection), and
wherein the system further comprises a remote photoplethysmography, rPPG, system for processing one or more of the second set of image frames using rPPG (page 3, Face Patch pair-based HR estimation, “Fast ICA [51] is then performed to estimate the PPG signal from the intensity traces”, this forms a remote PPG signal acquisition as it is estimated from the remotely acquired video signals; see also figure 1, abstract; pages 3-5, Proposed HR Estimation Method and Experimental Results; figures 2-4).
Primary reference Kado fails to teach:
a light source configurator for determining a light source configuration and transmitting the light source configuration to a first light source based on the identifying
However, the analogous art of Kaestle of an oxygenation screening system using a remote PPG (abstract) teaches:
a light source configurator for determining a light source configuration and transmitting the light source configuration to a first light source based on the identifying ([0043], light power is controlled by the control unit for operation of the detection unit in the optimum operating point; [0121], light source; [0122], “The light source 10 can be part of a control loop which sets an optimal operating point of the image sensor of the imaging unit 2. Optimal in this context refers to an output signal without signal clipping, no saturation of individual detectors of the image sensors and a good signal-to-noise ratio at least for the detector area corresponding to first and/or second marker area.” The control loop based upon the image sensor of the imaging unit forms a “based on the identifying” of the region of interest of the imaged skin patch in the combined prior art invention with Kado)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the remote PPG imaging acquisition and processing system of Kado to incorporate the determination of a light source configuration as taught by Kaestle because it enables configuration of illumination settings to provide an optimal output signal without signal clipping, no saturation of individual detectors of the image sensors and a good signal-to-noise ratio at least for the detector area (Kaestle, [0122]).
Regarding claim 12, the combined references of Kado and Kaestle teach all of the limitations of claim 11. Primary reference Kado further teaches:
wherein the at least one camera comprises a red-green-blue, RGB, sensor for capturing the first and second sets of image frames (pages 3-5, III. Proposed HR Estimation Method, particularly RGB sequence acquisition of image frames of a user’s face as in figure 1; see also figures 2-4, with figure 3, showing RGB-NIR camera for capturing RGB sequence and is a single dual sensor camera; page 5, IV Experimental Results, A. Setups and data collection).
Regarding claim 13, the combined references of Kado and Kaestle teach all of the limitations of claim 11. Primary reference Kado further teaches:
wherein the at least one camera comprises a near-infrared, NIR, sensor for capturing the first set of image frames in the NIR spectrum, and an RGB sensor for capturing the second set of image frames in the visible light spectrum (pages 3-5, III. Proposed HR Estimation Method, particularly RGB sequence acquisition of image frames of a user’s face as in figure 1; see also figures 2-4, with figure 3, showing RGB-NIR camera for capturing RGB sequence and is a single dual sensor camera; page 5, IV Experimental Results, A. Setups and data collection).
Regarding claim 14, the combined references of Kado and Kaestle teach all of the limitations of claim 11. Primary reference Kado further teaches:
wherein the at least one camera comprises a first camera for capturing the first set of image frames, and a second camera for capturing the second set of image frames (pages 3-5, III. Proposed HR Estimation Method, particularly RGB and NIR sequence acquisition of image frames of a user’s face as in figure 1; see also figures 2-4, with figure 3, showing RGB-NIR camera for capturing RGB sequence and NIR sequence image frames using the RGB sensor and NIR sensor in the camera and is a single dual sensor camera. As shown in figure 4, part (a) the dual-CCD camera setup forms a first camera and a second camera as claimed; page 5, IV Experimental Results, A. Setups and data collection).
Regarding claim 15, the combined references of Kado and Kaestle teach all of the limitations of claim 14. Primary reference Kado further teaches:
wherein the first camera comprises an NIR sensor for capturing the first set of image frames in the NIR spectrum, and wherein the second camera comprises an RGB sensor for capturing the second set of image frames in the visible light spectrum (pages 3-5, III. Proposed HR Estimation Method, particularly RGB and NIR sequence acquisition of image frames of a user’s face as in figure 1; see also figures 2-4, with figure 3, showing RGB-NIR camera for capturing RGB sequence and NIR sequence image frames using the RGB sensor and NIR sensor in the camera and is a single dual sensor camera. As shown in figure 4, part (a) the dual-CCD camera setup forms a first camera and a second camera as claimed; page 5, IV Experimental Results, A. Setups and data collection).
Regarding claim 17, the combined references of Kado and Kaestle teach all of the limitations of claim 11. Primary reference Kado further teaches:
wherein the first light source is controllable to illuminate more than one skin patch on the user's face separately and/or independently of one another (page 5, IV Experimental Results, A. Setups and data collection, figure 3, lighting setup provides for a light source configuration determined, and provided to the light source with capability of illuminated “selected patches” of figure 2 separately of one another and enables acquisition of extracted signals from the patches as in pages 3-5, III. Proposed HR Estimation Method. As “independently” is claimed in an alternative, independent adjustment of light to patches is not read into the claim broadest reasonable interpretation).
Regarding claim 18, the combined references of Kado and Kaestle teach all of the limitations of claim 11. Primary reference Kado further teaches:
wherein the first light source comprises a plurality of light emission sources, and wherein some or all of the plurality of light emission sources are individually configurable (page 5, IV Experimental Results, A. Setups and data collection, figure 3, lighting setup provides for :”NIR LEDs” which form a plurality of light emitting diodes).
Regarding claim 21, the combined references of Kado and Kaestle teach all of the limitations of claim 1. Primary reference Kado further fails to teach:
wherein the one or more quality criteria include a signal-to-noise ratio
However, the analogous art of Kaestle of an oxygenation screening system using a remote PPG (abstract) teaches:
wherein the one or more quality criteria include a signal-to-noise ratio ([0043]; [0121]; [0122], “a good signal-to-noise ratio”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the remote PPG imaging acquisition and processing system of Kado and Kaestle to incorporate the determination of a good signal to noise ratio as taught by Kaestle because it enables configuration of illumination settings to provide an optimal output signal without signal clipping, no saturation of individual detectors of the image sensors and a higher quality signal at least for the detector area (Kaestle, [0122]).
Regarding claim 22, the combined references of Kado and Kaestle teach all of the limitations of claim 1. Primary reference Kado further is not relied upon to teach:
wherein the illuminating includes illuminating the at least one skin patch at a first intensity or wavelength, while illuminating a second skin patch at a second intensity or wavelength
However, the analogous art of Kaestle of an oxygenation screening system using a remote PPG (abstract) teaches:
wherein the illuminating includes illuminating the at least one skin patch at a first intensity or wavelength, while illuminating a second skin patch at a second intensity or wavelength ([0060]-[0062], first and second groups or arrays of pixels (skin patches) as in figure 2, items 24, 25, which are illuminated at a first and second wavelength).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the remote PPG imaging acquisition and processing system of Kado and Kaestle to incorporate the first and second patch regions at different wavelengths as taught by Kaestle because it ensure that sufficient light at the respective wavelength is available (Kaestle, [0043]) and absorption and intensity modulation can be calculated across skin regions in different lighting conditions (Kaestle, [0060]-[0062]).
Regarding claim 23, the combined references of Kado and Kaestle teach all of the limitations of claim 1. Primary reference Kado further is not relied upon to teach:
further comprising taking into account relative locations of the first light source and at least one camera when determining the light source configuration, and precisely emitting light from the first light source onto the at least one skin patch
However, the analogous art of Kaestle of an oxygenation screening system using a remote PPG (abstract) teaches:
further comprising taking into account relative locations of the first light source and at least one camera when determining the light source configuration, and precisely emitting light from the first light source onto the at least one skin patch ([0121], “The optimal setup for best screening results would ensure little shadowing impact by operator, adequate light levels for each color channel, and relatively well-defined camera/light position with respect to the subject 100.” This well-defined camera/light position with respect to the subject forms a relative location account of the source and camera when determining light configuration and power; [0122]; see also [0043]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the remote PPG imaging acquisition and processing system of Kado and Kaestle to incorporate the relative positioning of the light source and light detector relative to the patient as taught by Kaestle because it enables configuration of illumination settings to provide an optimal output signal without signal clipping, no saturation of individual detectors of the image sensors and a higher quality signal at least for the detector area (Kaestle, [0122]).
Claims 19 and 24 are rejected under 35 U.S.C. 103 as being unpatentable over Kado, in view of Kaestle as applied to claims 11 or 1 above, and further in view of Bresch et al. (U.S. Pub. No. 20140253709) hereinafter Bresch.
Regarding claim 19, the combined references of Kado and Kaestle teach all of the limitations of claim 11. Primary reference Kado further fails to teach:
wherein the wherein the first light source comprises a liquid crystal display, LCD, projector or a digital light processing, DLP, projector
However, the analogous art of Bresch of a remote vital sign acquisition device with a controllable illumination system (abstract) teaches:
wherein the wherein the first light source comprises a liquid crystal display, LCD, projector or a digital light processing, DLP, projector ([0036], LCD projector).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the remote PPG imaging acquisition and processing system of Kado and Kaestle to incorporate the LCD illumination projector as taught by Bresch because it provides for controlled emission of light radiation in an appropriate frequency range allowing the extraction of vital sign information from detected light that has been reflected from the ROI (Bresch, [0036]). This leads to higher quality signals and improved clinical diagnostics.
Regarding claim 24, the combined references of Kado and Kaestle teach all of the limitations of claim 1. Primary reference Kado further fails to teach:
wherein the light source configuration is determined so as to focus light only onto the at least one skin patch identified
However, the analogous art of Bresch of a remote vital sign acquisition device with a controllable illumination system (abstract) teaches:
wherein the light source configuration is determined so as to focus light only onto the at least one skin patch identified ([0080], “The control unit 7 particularly controls said illumination device 11 to locally illuminate said region of interest 300 with focused radiation at an increased intensity level compared to the radiation used for illumination in the first phase.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the remote PPG imaging acquisition and processing system of Kado and Kaestle to incorporate the focused illumination on the region of interest as taught by Bresch because it can reduce disturbances to the subject in non-measured regions, and provides sufficient illumination of the region of interest for high quality signal reception (Bresch, [0080]). This leads to improved clinical diagnostics.
Response to Arguments
Applicant’s arguments with respect to claims 1-4, 7-15, 17-19, and 21-24 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SEAN A FRITH whose telephone number is (571)272-1292. The examiner can normally be reached M-Th 8:00-5:30 Second Fri 8:00-4:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Keith Raymond can be reached at 571-270-1790. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SEAN A FRITH/Primary Examiner, Art Unit 3798