DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Election/Restrictions
In response to election requirement of 05/21/2026, applicant elected species 1 with traverse. Applicant argues that:
“the subject matter of claims 3, 5, and 14, which specify one way of denoising the iPPG signals by solving the structured recovery problem of generic claim 1 (or claim 12 as the case may be) using unrolled gradient descent with the regularizer integrated as a pseudo- proximal operator (claims 3 and 14). The various ways of solving the structured recovery problem do not fall into separate classes since they are all solutions to a common problem.”
Examiner respectfully disagrees. The different solutions recites in the claims, require different field of searches as noted in the office action. There is nothing of record showing that one search would provide the results for another. Further, the applicant does not submit whether they are obvious variants of each other. In view of this, arguments are not persuasive and claims 3, 5, 14 remain withdrawn from consideration.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 4, 6-7, 9-10, 12-13, 15-17, 19-20 rejected under 35 U.S.C. 103 as being unpatentable over Liu [Adaptive-Weight Network for Imaging Photoplethysmography Signal Extraction and Heart Rate Estimation, IEEE Transactions On Instrumentation And Measurement, Vol. 71, 2022] in view of Yaman [Noise2Inpaint: Learning Referenceless Denoising by Inpainting Unrolling, arXiv.org 16 June 2020].
As per claim 1, Liu teaches a remote photoplethysmography (RPPG) system for estimating a vital sign signal of a subject, comprising:
a memory configured to store executable instructions; and a processor coupled with the memory (Liu Introduction page RHS, video-based noncontact HR measurements for rppg and ippg, requires processor and memory),
wherein the stored instructions, when executed by the processor, cause the RPPG system to:
collect a sequence of images of different regions of skin of the subject (Liu Fig 1 face video),
each region including pixels of different intensities indicative of variation of coloration of the skin (Liu section B. ROI Selection and IPPG Signal Extraction “the skin color and smoothness, the distance between face and camera, the facial micromovement, and the light intensity can also cause the quality differences of the IPPG signal among different ROIs.”);
transform the sequence of images into a sequence of imaging photoplethysmography (iPPG) signals indicative of variation of the vital signs of the subject in a time domain (Liu Fig 1 Fig 5, iPPg signal extraction from face video),
wherein the iPPG signals are subject to sparse non-Gaussian noise (In view of applicant spec. ¶0042, ¶0053, the non-gaussian noise occurs because the images are remotely collected. Hence, this claimed noise is present in rPPG /iPPG of Liu also);
denoise the iPPG signals (Liu section D. Effectiveness of IPPG Weight Network “a larger ROI is more likely to reduce noise, thus providing a better IPPG signal…The optimized signal has more stable periodic change and less noise…With the self-adaptive weights, more useful features contained in multiple ROIs are exploited; thereby, the noise is discarded.”)
by
output the vital sign signal corresponding to the denoised iPPG signals (Lliu Fig 1 HR estimate).
Liu does not expressly teach solving a structured recovery problem with a regularizer.
Yaman, in a related field of Deep learning based image denoising, solving a structured recovery problem with a regularizer (Yaman page 3 equation 8, solving function using regualrizer R).
Yaman teaches a self-supervised deep learning algorithm for image denoising, useful for colored Gaussian noise and non-Gaussian statistics, in biomedical and biological applications. The method in Yaman allows for denoising using only noisy images, i.e. no clean images required. Hence, Before the effective filing date of the claimed invention it would have been obvious to a person of ordinary skill in the art to modify the apparatus in Liu by utilizing Noise2Inpaint approach for denoising the ippg signals, since it requires only noisy images for deep learning (Yaman abstract, conclusion).
As per claim 2, Liu in view of Yaman further teaches wherein the processor is configured to solve the structured recovery problem using unrolled gradient descent (Yaman section 3.1 Algorithm Unrolling), wherein the regularizer is integrated as a fixed-point iteration of the unrolled gradient descent (Yaman section 3.1 …”solving the objective function in Equation (10) is unrolled for a fixed number of iterations.”).
As per claim 4, Liu in view of Yaman further teaches wherein the regularizer includes a learned regularization term implemented using a deep equilibrium model (DEQ), and wherein the processor is configured to solve the structured recovery problem using unrolled gradient descent (Yaman Fig 1, section 3.1. note the deep learning and algorithm features are similar to that recites. There is nothing differencing the claimed DEQ, from the learning algorithm of Yaman),
with a predetermined number of iterations (Yaman section 3.1),
wherein for each iteration of the predetermined number of iterations, the DEQ is executed multiple times to a fixed-point of interim outputs of the unrolled gradient descent (Yaman equation 13, section 3.1 “fixed number of iterations, with each iteration including a data fidelity and a regularization block… iteration k in the unrolled network, the denoised image is comprised of the CNN output at the masked locations and a weighted average for the non-masked locations.”).
As per claims 6-7, Liu in view of Yaman further teaches wherein the processor is further configured to estimate the vital sign signal by minimizing a difference between the sequence of iPPG signals and the denoised iPPG signals using a gradient descent minimization (Yaman section 3.1 “In these methods, an iterative optimization algorithm, such as proximal gradient descent … trained end-to-end by minimizing a loss function that characterizes the discrepancy between a reference and network output”)
wherein the processor is further configured to estimate the vital sign signal by minimizing a difference between the sequence of iPPG signals and the denoised iPPG signals using a proximal gradient descent minimization (Yaman section 3.1 “In these methods, an iterative optimization algorithm, such as proximal gradient descent”).
As per claim 9, Liu in view of Yaman further teaches wherein the processor is further configured to: estimate noise in the sequence of iPPG signals (Yaman section 3.1 “data fidelity term between the desired output and the noisy input”, section 3.2 “the loss is defined between noisy pixels excluded in the training and the network output at corresponding unseen locations”); and
modify the denoised iPPG signals with the estimated noise for minimizing a difference between the sequence of iPPG signals and the denoised iPPG signals (Yaman section 3.1 “trained end-to-end by minimizing a loss function that characterizes the discrepancy between a reference and network output”).
As per claim 10, Liu in view of Yaman further teaches the noise is processed with a noise neural network to enforce an implicit structure on the noise and generate a structured component of the noise (Yaman section 3.2 Noise2Inpaint Self-Supervised Training).
As per claims 12-13, 15-17, 19, they have directed to method of claims 1-2, 4, 6, 7, 9 and are rejected for same reasons as above.
As per claim 20, it is directed to embody method of claim 12 in a “non-transitory computer readable medium”. Before the effective filing date of the claimed invention it would have been obvious to a person of ordinary skill in the art to embody method of Liu in view of Yaman in a “non-transitory computer readable medium” so that the instructions can be automatically executed.
Claims 8, 18 rejected under 35 U.S.C. 103 as being unpatentable over Liu in view of Yaman as applied to claims 1, 12 above, and further in view of Agnew [US 20180268695 A1].
As per claims 8, 18, Liu in view of Yaman does not expressly teach further comprising a controller communicatively coupled to a machine and the processor, wherein the controller is configured to: receive the vital sign signal of the subject; and generate one or more control commands for controlling the machine, based on the received vital sign signal of the subject.
Agnew, in field of detecting a driver status teaches a controller communicatively coupled to a machine and the processor (Agnew Fig 1), wherein the controller is configured to: receive the vital sign signal of the subject (Agnew Fig 1 item 101, ¶0098); and generate one or more control commands for controlling the machine, based on the received vital sign signal of the subject (Agnew ¶0105-¶0106, Fig 1).
Agnew teaches driver monitoring including PPG monitoring, to estimated driver availability and traffic hazard, and in turn control a vehicle for same operation (Agnew ¶0009, ¶0086). Before the effective filing date of the claimed invention it would have been obvious to a person of ordinary skill in the art to modify Liu in view of Yaman by integrating application of ppg sensing as in Agnew for preventing accidents and improving vehicle safety.
Allowable Subject Matter
Claims 11 objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. Examiner does not find it obvious to modify Liu in view of Yaman, or any other references of record (See PTO 892) to show a noise neural network is trained with ground truth iPPG signals measured using contact sensing.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to OOMMEN JACOB whose telephone number is (571)270-5166. The examiner can normally be reached 8:00-4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ANNE M KOZAK can be reached at 571-270-0552. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Oommen Jacob/Primary Examiner, Art Unit 3797