DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after 16 March 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 02 March 2026 has been entered.
Status of Claims
Claim(s) 1-2, 7-9, 21, 23-26 and 28-34 is/are currently amended. Claim(s) 3-5, 10-20 and 22 has/have been canceled. New claim(s) 35 has/have been added. Claim(s) 1-2, 6-9, 21 and 23-35 is/are pending.
Objections and/or Rejections Withdrawn
Objections to the specification, rejections under 35 U.S.C. 101, rejections under 35 U.S.C. 112(a) (pre-AIA 35 U.S.C. 112, first paragraph) and/or rejections under 35 U.S.C. 112(b) (pre-AIA 35 U.S.C. 112, second paragraph) not reproduced below has/have been withdrawn in view of Applicant's amendments to the claims and/or submitted remarks.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of pre-AIA 35 U.S.C. 112, second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim(s) 1-2, 6-9, 21 and 23-35 is/are rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
Regarding claim 1, claim 21, claim 33 and claims dependent thereon, the limitation processing the at least one edge-ROI to obtain a centroid, wherein the centroid includes an x-component and a y-component; determining a principal axis of movement of the centroid over the time period" is indefinite. It is unclear how an obtained centroid, which is merely a center point or location, includes "a principal axis of movement." To the best of the examiner's understanding of the specification as filed, and consistent with the limitations of, e.g., dependent claims 8 and 29, the "thermal signal associated with the FOV over a time period" received from the thermal camera is a video of the FOV over the time period, which inherently comprises a plurality of image frames; said video is processed to determine an edge-ROI(s) therein; a centroid of the edge-ROI is calculated for each frame of the video to obtain movement of the centroid as a function of time, and a principal axis of centroid movement over time is determined. Amendments commensurate in scope with the above-noted understanding would overcome this rejection.
Regarding claim 8, claim 29 and claims dependent thereon, there is insufficient antecedent basis for the limitation "each frame" in the claim.
Regarding claim 9, claim 26, claim 35 and claims dependent thereon, the limitation "using centroid processing for filtering out the at least one of the respiratory signal or the cardiac signal from the portion of the thermal signal" of claim 9 and the comparable limitations of claims 26 and 35 are indefinite. Applicant discloses (in the specification as filed):
[0056] Additionally, the waveforms from FIGs. 5A-C were generated by calculating the mean (aggregate) temperature of the ROI. Since movement is the instigator of the signals (except in the case of the nose measurement), exemplary embodiment described herein provide for algorithmically processing the ROI by calculating the centroid (or any other related method) of the ROI for each frame. The centroid movement (expressed in pixels) may then be plotted over time.
[0057] The centroid has an x and y component. In further exemplary embodiments, the centroid movement is calculated relative to an arbitrary point (i.e. the Euclidian distance) In further exemplary embodiments, the principal component is extracted, with determination of the movement relative to the mean centroid location and the direction of maximum variance. Additional exemplary embodiments also contemplate any other processing to enhance the signal quality.
[0058] In further exemplary embodiments, different centroid processing can also be applied to extract the respiration and cardiac signals since the forces causing the movements are different in nature. For example, respiration's principal axis of movement may be in direction A as shown in Figure 7, which generally shows at 700 a prone patient 710, whereas the cardiac movement may be in direction B. The breathing and cardiac signals will therefore be enhanced by considering the direction of each source's movement.
This appears to be the extent of the disclosure relevant to the claimed "centroid processing." In view of the above, Applicant appears to disclose "using centroid processing" includes determining or plotting centroid movement over time, calculating centroid movement relative to an arbitrary point, determining a principal component of centroid movement, etc. The independent claims require determining a principal axis of movement of the centroid, and based on the principal axis of movement of the centroid, filtering out the respiratory or cardiac signal. Accordingly, the independent claims already recite steps commensurate in scope with the disclosed "centroid processing," such that it is unclear what, if any, additional steps are required by "using centroid processing for filtering out…" of the limitations of claims 9, 26 and 35. If additional steps are required by said limitations, it is further unclear what those steps are, require and/or encompass.
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of pre-AIA 35 U.S.C. 112, first paragraph:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claim(s) 25, 30, 32, 34 and claims dependent thereon is/are rejected under 35 U.S.C. 112(a) or pre-AIA 35 U.S.C. 112, first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for pre-AIA the inventor(s), at the time the application was filed, had possession of the claimed invention.
Regarding claim 25 and claims dependent thereon, the limitation "aggregating thermal information of one or more edge pixels associated with the edge-ROI to obtain the portion of the thermal signal for the edge-ROI over the time period" lacks sufficient support in the application as filed. Applicant discloses a generating a signal by aggregating thermal information (temperature) of pixels in an ROI and bandpass filtering said signal to extract respiratory and/or cardiac signal components (¶¶ [0051]-[0052]). This appears to be disclosed as an alternative to "using centroid processing." Specifically, as noted above with respect to rejections under 35 U.S.C. 112(b), Applicant discloses, "Additionally, the waveforms from FIGs. 5A-C were generated by calculating the mean (aggregate) temperature of the ROI. Since movement is the instigator of the signals (except in the case of the nose measurement), exemplary embodiment described herein provide for algorithmically processing the ROI by calculating the centroid (or any other related method) of the ROI for each frame." Applicant fails to disclose a single embodiment in which thermal information of one or more edge pixels associated with the edge-ROI is aggregated to obtain the portion of the thermal signal for the edge-ROI over the time period (as required by claim 25) and the so-aggregated thermal signal is processed to obtain a centroid (as required by claim 21) and/or how a centroid of "aggregated thermal information" (i.e., average temperature within a ROI) could be used to calculate a centroid in said ROI. Therefore, the limitations of claim 25 lack sufficient support in the application as filed and are directed to and/or encompasses new matter.
Regarding claim 30, claim 32, claim 34 and claims dependent thereon, the limitation "plotting the centroid over the time period to determine the principal axis of movement over the time period" of claim 30 and the comparable limitations of claims 32 and 34 lack sufficient support in the application as filed. Firstly, Applicant fails to disclose "plotting the centroid over the time period." Rather, Applicant discloses "centroid movement," not the centroid itself, may be plotted over time. Secondly, Applicant fails to disclose that, or how, a plot of a centroid over a time period is used determining a principal axis of movement over the time period. Therefore, the limitations of claims 30, 32 and 34 lack sufficient support in the application as filed and are directed to and/or encompasses new matter.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
Determining the scope and contents of the prior art.
Ascertaining the differences between the prior art and the claims at issue.
Resolving the level of ordinary skill in the pertinent art.
Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1, 6-9, 21, 23 and 25-35 is/are rejected under 35 U.S.C. 103 as being unpatentable over "Estimation of Breathing Rate in Thermal Imaging Videos: A Pilot Study on Healthy Human Subjects" (previously cited, Pereira '16) in view of "A Novel Thermal Measurement for Heart Rate" (previously cited, Jing) and WO 2021/245787 A1 (references below made to English language equivalent, US 2023/0200684 A1, Kojima).
Regarding claims 1, 8-9, 21, 23, and 29-35, Pereira '15 discloses and/or suggests a system, comprising:
a thermal camera having a field of view (FOV) oriented towards a patient (pg. 1246, LWIR camera sat atop a tripod, placed approximately 2 m away from the subjects who sat on a chair);
a monitor (pg. 1247, monitor; and/or means by which detected BR (e.g., pg. 1246) is communicated to a user); and
at least one processor communicatively coupled to the monitor, the processor(s) configured for executing one or more computer instructions to cause the system to perform a plurality of operations (computing means by which steps b-d are executed), comprising:
receiving, from the thermal camera, a thermal signal associated with the FOV over a time period (pgs. 1246-1247, acquiring a thermal video sequence/recording using the camera; Fig. 1, data from step a received for further analysis (steps b-d));
processing the thermal signal associated with the FOV to determine at least one edge region of interest (edge-ROI) having a distinct temperature gradient, the at least one edge-ROI being associated with a portion of the thermal signal over the time period (Fig. 1, automatic detection and tracking of ROIs; pg. 1243, shoulder ROIs identified by edge detection in a binary image using a Sobel edge detector), the at least one edge-ROI being defined by an interface between at least a portion of the patient and at least a portion of a non-patient background (e.g., Fig. 1, step b, bottom image; Fig. 3, image b; etc.);
processing the at least one edge-ROI to obtain a signal indicative of movement over the time period substantially aligned with respiratory movement (pgs. 1243-1244, vertical projection PV give information about the position of the shoulders in the thermogram; pg. 1245, breathing waveform of the shoulders stands basically for upward and downward (i.e., vertical) movement);
filtering out, from the signal indicative of movement over the time period substantially aligned with respiratory movement, a respiratory signal (pg. 1245, bandpass range between 0.1 and 3 Hz);
analyzing the respiratory signal to determine a respiratory rate (pg. 1245, local maximum in the bandpass range corresponds to the BR at a given time point; pg. 1246, fusion of the shoulder BR with other BR estimates); and
displaying, via a monitor, the respiratory rate (pg. unobtrusive and contactless techniques for monitoring of vital signs for clinical purposes, thereby at least suggesting the calculated BR is and/or can be output for monitoring, e.g., by a clinician).
Pereira '15 does not disclose processing the edge-ROI(s) to obtain a signal indicative of movement over the time period comprises obtaining a centroid including an x-component and a y-component.
Jing discloses/suggests a comparable method comprising receiving image data including a plurality of frames; determining a ROI; processing the ROI to obtain a centroid over a time period by calculating the centroid of the ROI for each frame of the image data, wherein the centroid includes an x-component and a y-component (pg. 2164, centroid of the binary image is calculated in every frame, and every succeeding centroid is compared with the first frame centroid to obtain the horizontal (e.g., x-component) and vertical (e.g., y-component) variations of the centroids denoting the subject's front-back and up-down motions, respectively). Comparable to vertical movement of the edge ROI in Pereira '15, Jing discloses the centroid movement is associated with respiratory movement (pg. 2164).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Pereira '16 with processing the at least one edge-ROI to obtain a centroid having an x-component and a y-component as taught/suggested by Jing as a simple substitution of one suitable means/method for generating a motion signal indicative of respiratory movement for another to yield no more than predictable results. See MPEP 2143(I)(B).
Pereira '15 as modified does not disclose the method comprises determining a principal axis of movement of the centroid over the time period by calculating the centroid of the ROI for each frame of the portion of the thermal signal, wherein the principal axis of movement of the centroid over the time period comprises a first axis of movement aligned with respiratory movement; and based on the principal axis of movement of the centroid, filtering out, from the portion of the thermal signal, the respiratory signal associated with the first axis of movement.
Kojima discloses/suggests a method comprising receiving, from at least one camera having a FOV oriented towards a patient, image/video data associated with the FOV over a time (Fig. 15, S10; ¶ [0033]); processing the image/video data to determine at least one ROI (Fig. 15, S20); processing the ROI(s) to obtain a motion vector (Fig. 15, S30; ¶ [0044] obtaining motion vector fi(m)); determining a principal axis of movement of the motion vector over a time period by plotting the motion vector over time, the principal axis of movement comprising a first axis of movement aligned with respiratory movement (Fig. 15, S41; ¶ [0064] respiration direction axis Rd(m) is calculated by principal component analysis by using the motion vector fi(m) for 300 frames); based on the principal axis of movement of the motion vector, filtering out from the image/video data at least one of the respiratory signal associated with the first axis of movement (Fig. 15, S43-S50; ¶ [0050] respiration signal Ct(k) is calculated from a component of the corrected motion vector in the direction of the respiration reference axis and/or Fig. 15, S60, ¶ [0059] bandpass filtering the respiration signal corresponding to the respiration frequency range); and analyzing the at least one of the respiratory signal to determine a respiratory rate (Fig. 15, S60; ¶ [0059] respiration information estimation result B(k), such as breathing rate (e.g., ¶ [0053]) is calculated from the signal after undergoing the filtering).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Pereira '16 to comprise using centroid processing by determining a principal axis of movement of the centroid over the time period by plotting the centroid over the time period, wherein the principal axis of movement of the centroid over the time period comprises a first axis of movement aligned with respiratory movement; and based on the principal axis of movement of the centroid, filtering out, from the portion of the thermal signal, the respiratory signal associated with the first axis of movement as taught/suggested by Kojima, such that the quality of the respiratory signal is enhanced by reducing deviations due to individual differences and/or influence of body motion not cause by respiration, thereby enabling respiratory rate to be calculated with higher accuracy (Kojima, ¶ [0077]).
Regarding claims 6 and 27, Pereira '16 as modified discloses/suggests determining the at least one edge-ROI is based upon a threshold value for the distinct temperature gradient (pg. 1243, shoulder ROIs identified by edge detection in a binary image using a Sobel edge detector).
Regarding claims 7 and 28, Pereira '16 as modified discloses/suggests filtering out the respiratory signal includes bandpass filtering of the portion of the thermal signal (Pereira '15, pg. 1245, bandpass range between 0.1 and 3 Hz; Kojima, ¶ [0059]; etc.).
Regarding claims 25-26, Pereira '16 as modified discloses/suggests using centroid processing for filtering out the respiratory signal as discussed above, and further discloses/suggests aggregating thermal information of one or more edge pixels associated with the edge-ROI to obtain the portion of the thermal signal for the edge-ROI over the time period (pg. 1243, equation (2), vertical projection is calculated based on a sum within a 21-pixel wide sliding window).
Claim(s) 2 and 24 is/are rejected under 35 U.S.C. 103 as being unpatentable over Pereira '16 in view of Jing and Kojima as applied to claim(s) 1 and 21 above, and further in view of US 2016/0210747 A1 (previously cited, Hay).
Regarding claims 2 and 24, Pereira '16 as modified discloses/suggests the limitations of claims 1 and 21, as discussed above, but does not expressly disclose the method further comprises displaying, via the monitor, a respiratory waveform corresponding to the respiratory signal.
Hay discloses/suggests a comparable system comprising a monitor (Abstract, interface) communication coupled to a processor (Abstract, processor), wherein the processor is configured to filter out at least one respiratory signal from, e.g., thermal, image data (e.g., ¶ [0114]); and display, via the monitor, a respiratory rate and a respiratory waveform corresponding to the respiratory signal(s) (e.g., Fig. 17).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Pereira '16 to comprise displaying, via the monitor, a respiratory waveform corresponding to the respiratory signal as taught/suggested by Hay in order to provide additional respiration useful for monitoring (Hay, ¶ [0081]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to Applicant's disclosure: see attached PTO-892.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Meredith Weare whose telephone number is 571-270-3957. The examiner can normally be reached Monday - Friday, 9 AM - 5 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. Applicant is encouraged to use the USPTO Automated Interview Request at http://www.uspto.gov/interviewpractice to schedule an interview.
If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Tse Chen, can be reached on 571-272-3672. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Meredith Weare/Primary Examiner, Art Unit 3791
Prosecution Insights