DETAILED ACTION
Response to Arguments
Applicant's arguments filed with respect to claims 1-20 have been fully considered but are moot in view of the new ground(s) of rejection. The rejections are necessitated due to claim amendments.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine,
manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. When reviewing independent claim 1, and based upon consideration of all of the relevant factors with respect to the claim as a whole, claims 1-20 are held to claim an abstract idea without reciting elements that amount to significantly more than the abstract idea and is/are therefore rejected as ineligible subject matter under 35 U.S.C. 101.
The Examiner will analyze Claim 1, and similar rationale applies to independent Claims 12 and 13.
The rationale, under MPEP § 2106, for this finding is explained below. The claimed invention (1) must be directed to one of the four statutory categories, and (2) must not be wholly directed to subject matter encompassing a judicially recognized exception, as defined below. The following two step analysis is used to evaluate these criteria.
Step 1: Is the claim directed to one of the four patent-eligible subject matter categories: process, machine, manufacture, or composition of matter?
When examining the claim under 35 U.S.C. 101, the Examiner interprets that the claims is related to a machine since the claim is directed to information processing device.
Step 2a, Prong 1: Does the claim wholly embrace a judicially recognized exception, which includes laws of nature, physical phenomena, and abstract ideas, or is it a particular practical application of a judicial exception?
The Examiner interprets that the judicial exception applies since Claim 1 limitation of “calculate, in an image of a first frame of the moving image, a plurality of first representative values of pixel values of pixels in the image of the first frame”; “calculate a second representative value associated with the first frame by combining the plurality of first representative values”; and “calculate a pulse wave signal from a temporal change in the second representative value” are directed to abstract idea [mathematical concept].
If/when the claim recites a judicial exception (i.e., an abstract idea enumerated in MPEP § 2106.04(a), a law of nature, or a natural phenomenon), the claim requires further analysis in Prong Two.
Step 2a, Prong 2: Does the claim recite additional elements that integrate the judicial exception into a practical application? NO
“specify a location of a region of interest associated with an image of each frame of a moving image captured of a living organism” and “wherein locations of the pixels are in a plurality of regions of interest specified associated with images of a plurality of second frames including the image of the first frame”. In this case, the examiner is aware of cross frame relationship that specifies which regions supply pixel locations in the first frame; however, the claim doesn’t state any specific technique/method beyond calculating the pulse-wave signal from the selected values. These limitations do not integrate the mathematical concept into a practical application.
Step 2b: If a judicial exception into a practical application is not recited in the claim, the Examiner must interpret if the claim recites additional elements that amount to significantly more than the judicial exception.
The Examiner interprets that the Claims do not amount to significantly more since the claim state the information processing device processes images information without improving the function of the device.
Furthermore, the generic computer components or machine learning algorithm of the processor/memory/device recited as performing generic computer or machine learning functions that are well-understood, routine and conventional activities amount to no more than implementing the abstract idea with a computerized system.
The Examiner finds that Claims 2-11 and 14-20 does not state significantly more since the claim only recites additional steps for processing image signals.
Thus, claims 1-20 recite the same abstract idea and therefore are not drawn to the eligible subject matter as they are directed to the abstract idea without significantly more.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, 4-8, 12, 13, and 15-19 are rejected under 35 U.S.C. 103 as being unpatentable over Marks et al. (Pub. No. US 2021/0224983) in view of Varadarajan et al. (Pub. No. US 2019/0007690 hereinafter Var).
Regarding claims 1 and 12 Marks teaches at least one processor and at least one non-transitory computer-readable medium coupled to the at least one processor and storing one or more computer-executable instructions that, when executed by the at least one processor, cause the information processing device to: specify a location of a region of interest (facial regions of interest) associated with an image of each frame (each video frame) of a moving image (video) captured of a living organism (person) [Para. 24 “a set of iPPG signals measured from a video of the person are obtained by averaging pixel intensity over all pixels in each of a set of skin regions of the person at each time step (e.g., each video frame)” and “To obtain the locations of the facial regions, some embodiments first use a face alignment (i.e., facial landmark detection) method to detect a number of facial landmarks, then interpolate and extrapolate the detected landmarks to a number of interpolated locations that are used to subdivide the face into more regions”; Para. 89 “Some embodiments are based on recognition that when the facial landmarks are detected in each video frame independently, there is a high-frequency jitter in positions of the detected landmarks, even when the face is stationary”; Para. 86, 87, and 89].
calculate, in an image of a first frame (each video frame) of the moving image (video), a plurality of first representative values (IPPG signals) of pixel values of pixels (intensities of pixels) in the image of the first frame [Para. 24 “a set of iPPG signals measured from a video of the person are obtained by averaging pixel intensity over all pixels in each of a set of skin regions of the person at each time step (e.g., each video frame)”; Para. 89 “In some embodiments, the position of each facial landmark in frame t is estimated by averaging the detected positions of the landmark from frame t−5 to t+5”. It is clear that the prior art teaches computing the landmark position “in frame T” by averaging positions from a temporal neighborhood frame t-5 to t+5 and those landmarks are used to define/subdivide the face into “facial regions of interest” of frame t. Para. 86-88].
However, Maks doesn’t explicitly teach where locations of the pixels are in a plurality of regions of interest specified regarding images of a plurality of second frames including the image of the first frame.
Var teaches where locations of the pixels (pixels in the video frame) are in (corresponding to) a plurality of regions of interest (regions of interest in the ROI map) specified regarding images of a plurality of second frames including the image of the first frame (a current video frame; a previous frame) [para. 33, 35, 36, 38 and 40].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Marks’s RPPG method by incorporating Var’s teaching of generating an ROI map from regions detected using both a current video frame and a previous frame and then using the pixels corresponding to those mapped regions when calculating Marks’s regional ippg signals. This modification improves marks by making the sampled skin regions responsive to present and prior frame information, thereby reducing frame to frame ROI instability and improving the temporal reliability of the resulting pulse wave signal, with a reasonable expectation of success because both systems use frame indexed digital pixel regions.
Marks further teaches calculate a second representative value (a median) associated with the first frame (each video frame, which includes the first) by combining (a median across) the plurality of first representative values (ippg signals) [Para. 25 “A measured iPPG signal for each median region is obtained by computing for each time step a median across the iPPG signals from the mean regions that make up the median region”. Para. 86 and 98]; and
calculate a pulse wave signal (underlying ippg signal) from a temporal change (time series signal; measured intensity variations) in the second representative value (measured ippg signal; a median) [Para. 25 “A measured iPPG signal for each median region is obtained by computing for each time step a median across the iPPG signals from the mean regions that make up the median region” and Para. 88 “For each facial region j ∈{1, . . . , N}, the iPPG signal p.sub.j(t) obtained from the average pixel intensities is a one-dimensional time series signal, where t∈{1, . . . , T} is a video frame index within a time window of length T frames”; Para. 90 and Para. 98].
Regarding claims 2, Marks teaches an imaging unit (camera 1102) configured to acquire the moving image (video sequence) by capturing images of the living organism (patient 1104) [Para. 127].
Regarding claims 4 and 15, Marks teaches wherein the images of the plurality of second frames represent a plurality of images captured in a prescribed time (time windows of length ten seconds) [para. 88].
Regarding claims 5 and 16, Marks teaches wherein the images of the plurality of second frames represent a plurality of successively captured images (frames t-5 to 5+5) [Para. 89].
Regarding claims 6 and 17, Marks teaches wherein the images of the plurality of second frames represent the image of the first frame and an image of at least one frame captured before the first frame (frame t-5) [Para. 89].
Regarding claims 7 and 18, Marks teaches wherein the images of the plurality of second frames represent the image of the first frame and an image of at least one frame captured after the first frame (t+5) [Para. 89].
Regarding claims 8 and 19, Marks teaches wherein combining the plurality of first representative values comprises weighting the plurality of first representative values and combining ((weighted average) the plurality of weighted, first representative values [Para. 84-87 and 95].
Claim 13 is rejected for the same reason as claim 1. Furthermore, Marks teaches a computer-readable recording medium of a device coupled to at least one processor of the device and storing one or more computer-executable instructions that, when executed by the at least one processor, cause the device [Para. 135, fig. 12 and related description].
Claims 3, 9-11, 14 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Marks et al. (Pub. No. US 2021/0224983) in view of Varadarajan et al. (Pub. No. US 2019/0007690 hereinafter Var) in view of Kirenko et al. (Pub. No. US 2012/0195469).
Regarding claims 3, and 14, Mark teaches wherein the second representative value (final ippg signal) is calculated by the information processing device when the plurality of second frames is selected regarding the first frame [Para. 89 “weighted average”].
However, Mark in view of Var doesn’t explicitly teach the rest of claim limitations.
Kirenko teaches the second representative value is placed by a representative value (a single value) of pixel values of pixels in a location of a region of interest (sub-set) specified associated the image of the first frame when the plurality of second frames are not selected associated the first frame [Para. 71, 73-75].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Mark’s Pixel value combining unit , modified by Var, to include Kirenko’s per-frame sub-set/ROI representative value calculation when plural second frames are not selected, so Marks can still output a representative ROI value from the first frame alone. This modification would provide a fallback/single frame signal generation technique to improve Mark’s, by maintaining continues signal generation and reduce signal degradation when multi-frame data is unreliable due to noise.
Regarding claims 9 and 20, Mark in view of Var doesn’t explicitly teach the claim limitations.
However, Kirenko teaches wherein the one or more computer-executable instructions, when executed by the at least one processor, further cause the information processing device to: calculate a displacement between a first region of interest specified associated with the image of the first frame and a second region of interest specified associated with an image of one of the plurality of second frames other than the first frame, wherein weighting the plurality of first representative values comprises weighting the plurality of first representative values based on the calculated displacement (spatial distance) [para. 63, 68-70].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Mark in view of Var to include Kirrnko’s displacement calculation between a first sub-set/ROI and a corresponding sub-set/ROI in another frame to provide that displacement to Mark’s pixel value combining unit. The modified Marks system would weight the representative value according to the calculation displacement, thereby improving motion related corruption during iPPG signal generation.
Regarding claim 10, Mark in view of Var doesn’t explicitly teach the claim limitations.
However, Kirenko teaches wherein the displacement (motor vectors) is calculated based on at least one selected from the group consisting of a difference in location (distance) between the first region of interest and the second region of interest, a difference in area (enlargement or reduction in size) between the first region of interest and the second region of interest, and a rotation angle between the first region of interest and the second region of interest [Para. 27].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Mark in view of Var’s motion/noise processing module to calculate ROI displacement using Kirenko’s location and motion vector matching, size change, and rotation correspondence parameters. This medication would merely apply Kirenko’s known sub-set matching technique to marks’ facial region iPPG processing to improve correspondence tracking between frames.
Regarding claim 11, Mark in view of Var doesn’t explicitly teach the claim limitations.
However, Kirenko teaches wherein the images of the plurality of second frames (set 20) represent a plurality of images captured in a time (time interval) determined based on with the displacement (motion) [Para 74].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Mark in view of Var’s frame and window selection logic to use Kirenko’s variable time interval based on the selected image set and motion and displacement-based correspondence. This modification would allow Marks to dynamically select second frames over an appropriate time interval for more reliable ROI tracking and signal fusion which improves ROI tracking stability and reducing noise in the resulting time varying signals.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SOLOMON G BEZUAYEHU whose telephone number is (571)270-7452. The examiner can normally be reached on Monday-Friday 10 AM-8 PM.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Oneal Mistry can be reached on 313-446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 888-786-0101 (IN USA OR CANADA) or 571-272-4000.
/SOLOMON G BEZUAYEHU/
Primary Examiner, Art Unit 2666
