DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Applicant's arguments, filed 03/16/2026, have been fully considered. The following rejections and/or objections are either reiterated or newly applied. They constitute the complete set presently being applied to the instant application.
Applicants have amended their claims, filed 03/16/2026, and therefore rejections newly made in the instant office action have been necessitated by amendment.
Applicants have amended claims 1-5, 7, and 9-10.
Applicants have left claims 6 and 8 as originally filed/previously presented.
Claims 1-10 are the current claims hereby under examination.
Claim Rejections - 35 USC § 102 - Withdrawn
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Response to Arguments
Applicant’s arguments, see pages 7-8 of Remarks, filed 03/16/2026, with respect to claims 1-5 and 7-10 have been fully considered and are persuasive. Applicants have amended the claims, rendering the rejection moot. The 102(a)(2) rejection of claims 1-5 and 7-10 has been withdrawn.
Claim Rejections - 35 USC § 103 - Newly Applied Necessitated by Applicant’s Amendments
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-5 and 7-10 rejected under 35 U.S.C. 103 as being unpatentable over Wu et al. (US 20230000377 A1) (previously cited), hereinafter referred to as Wu, in view of Robert Newberry (US 20180214088 A1), hereinafter referred to as Newberry.
The claims are generally directed to a biological information acquisition device comprising: a memory configured to store instructions; and a processor configured to execute the instructions to: generate spatiotemporal information by accumulating information relating to a region of interest extracted from a plurality of time-series images included in a biological video, the biological video being obtained by shooting a skin of a subject for a predetermined period; extract AC components and DC components from the spatiotemporal information; and estimate blood oxygen saturation of the subject in the predetermined period, based on the AC components and the DC components, wherein the processor is further configured to execute the instructions to: input the AC components to a first convolutional neural network model to acquire a first set of features; input the DC components to a second convolutional neural network model to acquire a second set of features; and exchange information regarding a parameter between an intermediate layer of the first convolutional neural network model and an intermediate layer of the second convolutional neural network model in a process of acquiring the first set of features and the second set of features.
Regarding claim 1, Wu discloses a biological information acquisition device (Abstract, “systems … contactless image-based blood oxygen estimation”) comprising:
a memory configured to store instructions (Fig. 16, element 14, para. [0123], para. [0126], para. [0128]); and
a processor (Fig. 16, element 12, para. [0121-0124], para. [0126], para. [0128]) configured to execute the instructions to:
generate spatiotemporal information by accumulating information relating to a region of interest extracted from a plurality of time-series images included in a biological video, the biological video being obtained by shooting a skin of a subject for a predetermined period (Fig. 1(a), Fig. 1(b), Fig. 3, Fig. 8, Fig. 15, element 1500, para. [0051-0054], “smartphone captured hand videos … hand is detected as the region of interest … spatial averages of the red, green, and blue channels in the ROI may be calculated …”, para. [0086-0088], “contactless SpO2 monitoring from videos captured by cameras … spatial-temporal representation …”, para. [0115], “spatial and temporal data analysis …”);
extract AC components and DC components from the spatiotemporal information (para. [0057-0058], “the DC and AC components of the RGB channels may be calculated …”, para. [0086], “learn the features for SpO2 estimation”, para. [0117], “ calculating … DC and AC components of red, green, and blue color channels of the camera based on the spatial averaging …”); and
estimate blood oxygen saturation of the subject in the predetermined period, based on the AC components and the DC components (Fig. 1(a), Fig. 1(b), Fig. 8, Fig. 15, para. [0051], “extracted features may then be used for SpO2 prediction”, para. [0086], para. [0117]).
Wu teaches the use of a convolutional neural network model to acquire sets of features (para. [0086], para. [0089], para. [0116-0117]). However, Wu does not explicitly disclose wherein the processor is further configured to execute the instructions to: input the AC components to a first convolutional neural network model to acquire a first set of features; input the DC components to a second convolutional neural network model to acquire a second set of features; and exchange information regarding a parameter between an intermediate layer of the first convolutional neural network model and an intermediate layer of the second convolutional neural network model in a process of acquiring the first set of features and the second set of features.
Newberry teaches an analogous biological information acquisition device (Abstract, Fig. 2, para. [0021]) comprising a memory and processor configured to generate spatiotemporal information by accumulating information relating to a region of interest extracted from a plurality of time-series images included in a biological video, the biological video being obtained by shooting a skin of a subject for a predetermined period, extract AC components and DC components from the spatiotemporal information, and estimate blood oxygens saturation of the subject in the predetermined period, based on the AC components and the DC components (para. [0065-0072], para. [0092], para. [0211-0212], para. [0228], para. [0233]). Newberry further teaches the processor is further configured to execute instructions to: input the AC components to a first convolutional neural network model to acquire a first set of features (para. [0223-0224], para. [0232-0235]); input the DC components to a second convolutional neural network model to acquire a second set of features (para. [0223-0224], para. [0232-0235]); and exchange information regarding a parameter between an intermediate layer of the first convolutional neural network model and an intermediate layer of the second convolutional neural network model in a process of acquiring the first set of features and the second set of features (para. [0223-0224], para. [0230-0231]). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the processor disclosed by Wu to additionally input the AC components to a first convolutional neural network model to acquire a first set of features; input the DC components to a second convolutional neural network model to acquire a second set of features; and exchange information regarding a parameter between an intermediate layer of the first convolutional neural network model and an intermediate layer of the second convolutional neural network model in a process of acquiring the first set of features and the second set of features, as taught by Newberry. This is because Newberry teaches inputting AC and DC components into neural networks allows for multiple different health data points to be obtained, including oxygen saturation levels (para. [0233], para. [0268]).
Regarding claim 2, modified Wu discloses the biological information acquisition device according to claim 1, wherein the processor is further configured to execute the instructions to extract the AC components and the DC components by applying filtering processing on the spatiotemporal information (para. [0058], “DC component … second-order lowpass Butterworth filter … AC component … adaptive bandpass filters …”, para. [0094-0095]).
Regarding claim 3, modified Wu discloses the biological information acquisition device according to claim 1, wherein the processor is further configured to execute the instructions to extract the AC components and the DC components by inputting the spatiotemporal information into a deep learning model (para. [0086-0087], “convolutional neural networks … deep learning … HR may be directly inferred using a convolutional network with spatial temporal representation …”).
Regarding claim 4, modified Wu discloses the biological information acquisition device according to claim 3, wherein processor is further configured to execute the instructions to train the first convolutional neural network model and the second convolutional neural network model by using a loss function including an estimation error of the blood oxygen saturation, an estimation error of the AC components, and an estimation error of the DC components (para. [0059-0061], “DC component … AC component … loss function …”, para. [0095], “root-mean-squared-error … loss function …”).
Regarding claim 5, modified Wu discloses the biological information acquisition device according to claim 3, wherein the spatiotemporal information includes information in a temporal direction corresponding to an acquisition frequency or an acquisition count of the plurality of time-series images included in the biological video, and information in a spatial direction corresponding to intensity of each of red pixels, green pixels and blue pixels of the region of interest, and wherein the processor is further configured to execute instructions to extract the AC components and the DC components by inputting the spatiotemporal information into the deep learning model to perform convolution for each of red, green, and blue channels (para. [0038], “spatial and temporal data analysis …”, para. [0052], para. [0086], “spatially averaged to produce R, G, and B time series … three time series are fed into CNN …”, para. [0091-0095], “temporal convolutional and max pooling to extract the temporal information …”).
Regarding claim 7, modified Wu discloses the biological information acquisition device according to claim 1, wherein the processor further configured to execute the instructions to train the first convolutional neural network model and the second convolutional neural network model using a loss function representing an estimation error of the blood oxygen saturation (para. [0059-0061], “DC component … AC component … loss function …”, para. [0095], “root-mean-squared-error … loss function …”).
Regarding claim 8, modified Wu discloses the biological information acquisition device according to claim 1, wherein the biological video is a video obtained by shooting a surface of a hand or a face of the subject (Fig. 1(b), Fig. 8, para. [0047], “hand videos …”, para. [0051-0052], para. [0086]).
Regarding claim 9, Wu discloses a biological information acquisition method (Abstract, “methods … contactless image-based blood oxygen estimation”) comprising:
generating spatiotemporal information by accumulating information relating to a region of interest extracted from a plurality of time-series images included in a biological video, the biological video being obtained by shooting a skin of a subject for a predetermined period (Fig. 1(a), Fig. 1(b), Fig. 3, Fig. 8, Fig. 15, element 1500, para. [0051-0054], “smartphone captured hand videos … hand is detected as the region of interest … spatial averages of the red, green, and blue channels in the ROI may be calculated …”, para. [0086-0088], “contactless SpO2 monitoring from videos captured by cameras … spatial-temporal representation …”, para. [0115], “spatial and temporal data analysis …”);
extracting AC components and DC components from the spatiotemporal information (para. [0057-0058], “the DC and AC components of the RGB channels may be calculated …”, para. [0086], “learn the features for SpO2 estimation”, para. [0117], “ calculating … DC and AC components of red, green, and blue color channels of the camera based on the spatial averaging …”); and
estimating blood oxygen saturation of the subject in the predetermined period, based on the AC components and the DC components (Fig. 1(a), Fig. 1(b), Fig. 8, Fig. 15, para. [0051], “extracted features may then be used for SpO2 prediction”, para. [0086], para. [0117]).
Wu teaches the user of a convolutional neural network model to acquire sets of features (para. [0086], para. [0089], para. [0116-0117]). However, Wu does not explicitly disclose wherein the estimating comprises: inputting the AC components to a first convolutional neural network model to acquire a first set of features; inputting the DC components to a second convolutional neural network model to acquire a second set of features; and exchanging information regarding a parameter between an intermediate layer of the first convolutional neural network model and an intermediate layer of the second convolutional neural network model in a process of acquiring the first set of features and the second set of features.
Newberry teaches an analogous method (Abstract, para. [0021]), comprising generating spatiotemporal information by accumulating information relating to a region of interest extracted from a plurality of time-series images included in a biological video, the biological video being obtained by shooting a skin of a subject for a predetermined period; extracting AC components and DC components from the spatiotemporal information; and estimating blood oxygen saturation of the subject in the predetermined period, based on the AC components and the DC components (para. [0065-0072], para. [0092], para. [0211-0212], para. [0228], para. [0233]). Newberry further teaches the estimating comprises: inputting the AC components to a first convolutional neural network model to acquire a first set of features (para. [0223-0224], para. [0232-0235]); inputting the DC components to a second convolutional neural network model to acquire a second set of features (para. [0223-0224], para. [0232-0235]); and exchanging information regarding a parameter between an intermediate layer of the first convolutional neural network model and an intermediate layer of the second convolutional neural network model in a process of acquiring the first set of features and the second set of features (para. [0223-0224], para. [0230-0231]). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method disclosed by Wu to additionally input the AC components to a first convolutional neural network model to acquire a first set of features; input the DC components to a second convolutional neural network model to acquire a second set of features; and exchange information regarding a parameter between an intermediate layer of the first convolutional neural network model and an intermediate layer of the second convolutional neural network model in a process of acquiring the first set of features and the second set of features, as taught by Newberry. This is because Newberry teaches inputting AC and DC components into neural networks allows for multiple different health data points to be obtained, including oxygen saturation levels (para. [0233], para. [0268]).
Regarding claim 10, Wu discloses a non-transitory computer-readable recording medium storing a program, the program causing a computer to execute processing (Abstract, “computer program products … contactless image-based blood oxygen estimation”) comprising:
generating spatiotemporal information by accumulating information relating to a region of interest extracted from a plurality of time-series images included in a biological video, the biological video being obtained by shooting a skin of a subject for a predetermined period (Fig. 1(a), Fig. 1(b), Fig. 3, Fig. 8, Fig. 15, element 1500, para. [0051-0054], “smartphone captured hand videos … hand is detected as the region of interest … spatial averages of the red, green, and blue channels in the ROI may be calculated …”, para. [0086-0088], “contactless SpO2 monitoring from videos captured by cameras … spatial-temporal representation …”, para. [0115], “spatial and temporal data analysis …”);
extracting AC components and DC components from the spatiotemporal information (para. [0057-0058], “the DC and AC components of the RGB channels may be calculated …”, para. [0086], “learn the features for SpO2 estimation”, para. [0117], “ calculating … DC and AC components of red, green, and blue color channels of the camera based on the spatial averaging …”); and
estimating blood oxygen saturation of the subject in the predetermined period, based on the AC components and the DC components (Fig. 1(a), Fig. 1(b), Fig. 8, Fig. 15, para. [0051], “extracted features may then be used for SpO2 prediction”, para. [0086], para. [0117]).
Wu teaches the user of a convolutional neural network model to acquire sets of features (para. [0086], para. [0089], para. [0116-0117]). However, Wu does not explicitly disclose wherein the estimating comprises: inputting the AC components to a first convolutional neural network model to acquire a first set of features; inputting the DC components to a second convolutional neural network model to acquire a second set of features; and exchanging information regarding a parameter between an intermediate layer of the first convolutional neural network model and an intermediate layer of the second convolutional neural network model in a process of acquiring the first set of features and the second set of features.
Newberry teaches an analogous method and program (Abstract, para. [0021]), comprising generating spatiotemporal information by accumulating information relating to a region of interest extracted from a plurality of time-series images included in a biological video, the biological video being obtained by shooting a skin of a subject for a predetermined period; extracting AC components and DC components from the spatiotemporal information; and estimating blood oxygen saturation of the subject in the predetermined period, based on the AC components and the DC components (para. [0065-0072], para. [0092], para. [0211-0212], para. [0228], para. [0233]). Newberry further teaches the estimating comprises: inputting the AC components to a first convolutional neural network model to acquire a first set of features (para. [0223-0224], para. [0232-0235]); inputting the DC components to a second convolutional neural network model to acquire a second set of features (para. [0223-0224], para. [0232-0235]); and exchanging information regarding a parameter between an intermediate layer of the first convolutional neural network model and an intermediate layer of the second convolutional neural network model in a process of acquiring the first set of features and the second set of features (para. [0223-0224], para. [0230-0231]). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method and program disclosed by Wu to additionally input the AC components to a first convolutional neural network model to acquire a first set of features; input the DC components to a second convolutional neural network model to acquire a second set of features; and exchange information regarding a parameter between an intermediate layer of the first convolutional neural network model and an intermediate layer of the second convolutional neural network model in a process of acquiring the first set of features and the second set of features, as taught by Newberry. This is because Newberry teaches inputting AC and DC components into neural networks allows for multiple different health data points to be obtained, including oxygen saturation levels (para. [0233], para. [0268]).
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Wu et al. (US 20230000377 A1) (previously cited), hereinafter referred to as Wu, in view of Robert Newberry (US 20180214088 A1), hereinafter referred to as Newberry as applied to claim 5 above, and further in view of Tao et al. (US 11227161 B1) (previously cited), hereinafter referred to as Tao.
Regarding claim 6, modified Wu discloses the biological information acquisition device according to claim 5.
However, modified Wu does not explicitly disclose wherein the convolution is depth-wise convolution.
Tao teaches an analogous biological information acquisition device (Abstract, col. 1, lines 14-15). Tao teaches generating spatiotemporal information and estimating a physiological signal parameter based on the information (col. 2, lines 29-39). Tao further teaches the estimation involves using a convolutional neural network, including a depth-wise convolution (col. 7, lines 15-42, col. 10, lines 32-38). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the convolution taught by modified Wu to explicitly be a depth-wise convolution, as taught by Tao. This is because Tao teaches a depth-wise convolution is known CNN, and allows for the complexity of the model to be reduced, allowing for large channel values (col. 10, lines 32-38).
Response to Arguments
Applicant’s arguments filed 03/16/2026 did not directly address this rejection. The Examiner cannot find a reason to withdraw it.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KYLE W KRETZER whose telephone number is (571)272-1907. The examiner can normally be reached Monday through Friday 8:30 AM to 5:30 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jason M Sims can be reached at (571)272-7540. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/K.W.K./Examiner, Art Unit 3791
/JASON M SIMS/Supervisory Patent Examiner, Art Unit 3791
Prosecution Insights