DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “a camera shooting unit, configured to acquire a face video” in claim 17.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 17 and 18 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention. Claim 17 discloses the limitations “a detection unit, configured to”, “a first processing unit, configured to”, “a conversion unit, configured to” and “a second processing unit, configured to” and claim 18 discloses the limitations “a peak detection module, configured to”, “a sorting module, configured to”, “a confidence coefficient calculation module, configured to” and “an estimation module, configured to”. These are terms that invoke 112(f) as they include a nonce term “unit” or “module” modified by functional language linked by the transition word “configured to”. They do not have a known hardware or software embodiment in the art. The written description does not provide a corresponding structure or algorithm to transform a generic processor into a specialized computer.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 5, 7, 15, 17 and 18 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 17 discloses the limitations “a detection unit, configured to”, “a first processing unit, configured to”, “a conversion unit, configured to” and “a second processing unit, configured to” and claim 18 discloses the limitations “a peak detection module, configured to”, “a sorting module, configured to”, “a confidence coefficient calculation module, configured to” and “an estimation module, configured to”. These are terms that invoke 112(f) as they include a nonce term “unit” or “module” modified by functional language linked by the transition word “configured to”. The written description does not provide a corresponding structure or algorithm to transform a generic processor into a specialized computer. It is unclear as to what the bounds of the terms are as they do not have a known hardware or software embodiment in the art.
Claim 5 recites the limitation "the current frame" in line 5. There is insufficient antecedent basis for this limitation in the claim.
Claim 15 recites the limitations "the confidence coefficient" in line 4 and “the current result” in line 8. There is insufficient antecedent basis for this limitation in the claim.
The terms “seriously polluted” and “relatively small” in claim 15 are relative terms which render the claim indefinite. The terms “seriously” and “relatively” are not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. The terms are relative terms for which the written description has not defined upper and lower bounds.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 21 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter because a program without a physical embodiment, such as a non-transitory computer readable medium, is software per se. Thus, applying the broadest reasonable interpretation in light of the specification and taking into account the meaning of the words in their ordinary usage as they would be understood by one of ordinary skill in the art, the claimed system can be software per se. Therefore, the claimed subject matter as a whole fails to fall within the definition of a process, a machine, manufacture, or composition of matter.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-4, 8, 9, 11-18, 21 and 22 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Morris et al (2015/0302158).
Regarding claim 1, Morris et al disclose a heart rate estimation method, comprising:
acquiring a face video (a camera 102 captures signals – [0025]);
performing face detection on the face video to extract a local face area that is set as heart rate estimation (face tracking mechanism 224 locates and tracks one or more regions of interest, such as the face itself – [0028]);
performing first processing on values of pixel points in the local face area to obtain an initial heart rate signal (if the total head translation a(t) is greater than 400 pixels (over the 10 second window), the window is labeled as motion. These labels guide the processing and assist in heart rate estimation – [0074];[0076]; figs.5A-5C);
performing time-frequency domain conversion on the initial heart rate signal to obtain a frequency domain signal (time domain signals for a selected heart rate signal and motion component having a correlation – [0052];[0073]; figs.5A-5C); and
performing a second processing on the frequency domain signal within a heart rate estimation range to obtain a heart rate estimation value (picking the frequency that corresponds to heart rate – [0093]; chosen for further processing in the frequency domain – [0094]; fig.2).
Regarding claim 2, Morris et al disclose wherein the face video is collected by using an infrared camera (a camera 102 captures signals such as…infrared (IR) – [0025]).
Regarding claim 3, Morris et al disclose wherein the local face area comprises an area below the eyes (cheek regions ROI 2 and ROI 3 – [0030]; fig. 3).
Regarding claim 4, Morris et al disclose further comprising: combining face key point positioning with the face detection to extract the local face area that is set as heart rate estimation (cheeks tend to be useful because they are predominately soft tissue that exhibit significant pulsatile changes with blood flow – [0030]; fig.3).
Regarding claim 8, Morris et al disclose wherein before performing the time-frequency domain conversion on the initial heart rate signal, the heart rate estimation method further comprising: performing first denoising processing on the initial heart rate signal in the time domain, wherein the first denoising processing method includes bandpass filtering (this data may be band-pass smoothed – [0030]).
Regarding claim 9, Morris et al disclose wherein the step of performing second processing on the frequency domain signal within the heart rate estimation range to obtain the heart rate estimation value comprises: performing peak value detection on the frequency domain signal within the heart rate estimation range to obtain peak values (peak magnitudes – [0095]); sorting the peak values to obtain a sorting result (the nine components are sorted based on this value with the highest k chosen for further processing in the frequency domain – [0094]); obtaining a confidence coefficient according to the sorting result (spectral peak confidence is a good measure of the fitness of the component – [0096]); and obtaining the heart rate estimation value according to the confidence coefficient (outputs the heart rate as modified by any smoothing in this example – [0106]).
Regarding claim 11, Morris et al disclose wherein the heart rate estimation range is preset (range of frequencies corresponding to reasonable human heart rates – [0052]).
Regarding claim 12, Morris et al disclose wherein the heart rate estimation method further comprises: judging whether a detection time exceeds a second threshold value (heart rate value for a time window based upon confidence - abstract); and when the detection time exceeds the second threshold value, performing third processing on the frequency domain signal to obtain the heart rate estimation range (This is a contingent limitation as the claim scope is not limited by claim language that suggests or makes optional but does not require the steps to be performed MPEP 2111.04. The claim limitation does not provide any further patentable weight as it is only applicable when the condition is met).
Regarding claim 13, Morris et al disclose wherein the step of performing third processing on the frequency domain signal to obtain the heart rate estimation range comprises: continuously selecting consecutive X frames or X seconds of frequency domain signals through a sliding window to obtain a heart rate value, buffering the heart rate value, acquiring the heart rate estimation range by using a deep learning method, repeating action within M frames or M seconds, and averaging all acquired heart rate estimation ranges to obtain a final heart rate estimation range, wherein the action comprises the steps of continuously selecting X frames or X seconds of frequency domain signals selecting consecutive X frames or X seconds of frequency domain signals through a sliding window to obtain a heart rate value, buffering the heart rate value, acquiring the heart rate estimation range by using a deep learning method (This claim further defines a contingent limitation as the claim scope is not limited by claim language that suggests or makes optional but does not require the steps to be performed MPEP 2111.04. The claim limitation does not provide any further patentable weight as it is only applicable when the condition is met).
Regarding claim 14, Morris et al disclose wherein the peak values are sorted in a traversal manner in a descending order of the peak values, a highest peak value and a second peak values are selecting as the sorting result, and a ration of the highest peak value to the second peak value is taken as the confidence coefficient ([0096]; fig.7A).
Regarding claim 15, Morris et al disclose wherein the step of obtaining the heart rate estimation value according to the confidence coefficient comprises: comparing the confidence coefficient with a first threshold value to obtain a comparison result; wherein the confidence coefficient is less than the first threshold value, the comparison result indicates that the frequency domain signal is seriously polluted by noise, then discarding the current result, and detecting the next frame; and when the confidence coefficient is not less than the first threshold value, the comparison result indicates that the frequency domain signal is not polluted by noise or the noise pollution is relatively small, then acquiring the frequency corresponding to the highest peak value as the heart rate estimation value (This is a contingent limitation as the claim scope is not limited by claim language that suggests or makes optional but does not require the steps to be performed MPEP 2111.04. If the largest peak is much larger than the next-largest peak, this is indicative of high signal quality – [0039]. Only one of these conditions must be met. remove peak signals or lower confidence scores – [0102]).
Regarding claim 16, Morris et al disclose wherein before performing second processing on the frequency domain signal, the heart rate estimation method further comprises: performing second denoising processing on the frequency domain signal, wherein the second denoising processing method comprises at least one or more of the following: bandpass filtering (this data may be band-pass smoothed – [0030]).
Regarding claim 17, Morris et al disclose a heart rate estimation apparatus, comprising:
a camera shooting unit, configured to acquire a face video (a camera 102 captures signals – [0025]);
a detection unit, configured to perform face detection on the face video to extract a local face area that is set as heart rate estimation (face tracking mechanism 224 locates and tracks one or more regions of interest, such as the face itself – [0028]);
a first processing unit, configured to perform first processing on values of pixel points in the local face area to obtain an initial heart rate signal (if the total head translation a(t) is greater than 400 pixels (over the 10 second window), the window is labeled as motion. These labels guide the processing and assist in heart rate estimation – [0074];[0076]; figs.5A-5C; fig.10);
a conversion unit, configured to perform time-frequency domain conversion on the initial heart rate signal to obtain a frequency domain signal (time domain signals for a selected heart rate signal and motion component having a correlation – [0052];[0073]; figs.5A-5C; fig.10); and
a second processing unit, configured to perform second processing on the frequency domain signal within a heart rate estimation range to obtain a heart rate estimation value (picking the frequency that corresponds to heart rate – [0093]; chosen for further processing in the frequency domain – [0094]; figs.2 and 10).
Regarding claim 18, Morris et al disclose wherein the second processing unit comprises: a peak detection module, configured to perform peak value detection on the frequency domain signal within the heart rate estimation range to obtain peak values (peak magnitudes – [0095]); a sorting module, configured to sort the peak values to obtain a sorting result (the nine components are sorted based on this value with the highest k chosen for further processing in the frequency domain – [0094]); a confidence coefficient calculation module, configured to obtain a confidence coefficient according to the sorting result (spectral peak confidence is a good measure of the fitness of the component – [0096]); and an estimation module, configured to obtain the heart rate estimation value according to the confidence coefficient (outputs the heart rate as modified by any smoothing in this example – [0106]) (fig.10).
Regarding claim 21, Morris et al disclose a storage medium, wherein the storage medium comprises a stored program, and the program, when running controls a device (storage devices as executable code – [0116]) where the storage medium is located to execute the heart rate estimation method as claimed in claim 1 (see rejection of claim 1).
Regarding claim 22, Morris et al disclose an electronic device, comprising:
a processor (central processing unit (CPU 1002 – [0117]); and
a memory, configured to store executable instructions of the processor (various types of memory, including ROM, RAM, hard disk drive, and portable media drive – [0117]; an application comprising machine instructions is typically stored on the hard disk drive – [0125]);
wherein the processor is configured to execute the heart rate estimation method as claimed in claim 1 by executing the executable instructions (see rejection of claim 1).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 5 and 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Morris et al (2015/0302158) in view of Ahmad et al (9,962,095).
Regarding claim 5, Morris et al disclose wherein the step of performing first processing on the values of the pixel points in the local face area to obtain the initial heart rate signal comprises: performing an average on the values of the pixel points in the local face area, and taking the value of the average as a luminance signal of the current frame; and constituting the initial heart rate signal by the luminance signal of the current frame and luminance signals of previous N historical frames (Examiner notes N can be any number including zero. A local average may be computed from each ROI and each color channel, resulting in a total of nine intensity values – [0029]; fig.3).
Morris et al fail to explicitly disclose a weighted average.
However, Ahmad et al teach in the same medical field of endeavor, performing weighted average on values of pixel points in a local face area according to weights, and taking the value of weighted average as a luminance signal of a current frame (facial skin, apply at least one filter to the summed and averaged pixels, the filter being weighted by the noisiness measurement – claim 1).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the values of pixel points in the local face area and luminance signal of Morris et al with a weighted average of Ahmad et al as it would provide optimization of heart rate detection to account for noisiness.
Regarding claim 7, Morris et al as modified by Ahmad et al disclose the invention as claimed and discussed above. Ahmad et al further teach wherein the weight is set according to the position of the pixel point in the local face area, and the closer the pixel point is to an edge position in the local face area, the smaller the weight corresponding to the pixel point is (facial skin-only signal that removes the effects of eyelid, mouth and hair motion that would otherwise corrupt the signa intensity – col.1, ll.53-56. The hair is a pixel point closer to an edge and is removed, i.e. given no weight).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the pixels of the facial regions of interest of Morris et al with the weight is set according to the position of the pixel point in the local face area, and the closer the pixel point is to an edge position in the local face area, the smaller the weight corresponding to the pixel point is of Ahmad et al as it would provide the removal of pixels which can typically corrupt the heartbeat tracking results as set forth in Ahmad et al.
Claim(s) 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Morris et al (2015/0302158) in view of DiMaio et al (2017/0367580).
Regarding claim 6, Morris et al disclose the invention substantially as claimed, but fail to explicitly disclose converting the initial heart rate signal from a time domain into a frequency domain by using fast Fourier transform, to obtain the frequency domain signal.
However, DiMaio et al teach in the same medical field of endeavor, converting an initial heart rate signal from a time domain into a frequency domain by using fast Fourier transform, to obtain a frequency domain signal (heart rate signals may be transformed into the frequency domain (using for instance fast Fourier transforms) – [0046]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify performing time-frequency domain conversion on the initial heart rate signal to obtain the frequency domain signal of Morris et al with fast Fourier transforms of DiMaio et al as it would provide signal processing using a well-known and conventional technique.
Allowable Subject Matter
Claim 10 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ROCHELLE DEANNA TURCHEN whose telephone number is (571)270-7104. The examiner can normally be reached Mon - Fri 6:30-2:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Christopher Koharski can be reached on (571)272-7230. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ROCHELLE D TURCHEN/ Primary Examiner, Art Unit 3797