DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Guazzi et al. (Non-contact measurement of oxygen saturation with an RGB camera. Biomedical Optics Express, August 11, 2015), in view of Panasyuk et al (US 20070024946).
Regarding claims 1, 13 and 17, Guazzi teaches a method comprising: accessing a sequence of images of a portion of a person's skin illuminated by ambient light ("2.4. Dataset The algorithm was tested using the video recordings of five volunteers" p. 9; "updated every frame" p. 13, [0001]; "Fig. 5. A comparison of different ROI selection methods and of the effect of the inclusion criteria on the ROI selection in Sophia, carried out on all subjects considered. From left to right: the method introduced by [0014], using a fixed region manually set onto the lower part of the face; the method introduced by [15], using a fixed region placed on the cheek of the subject;" p. 13; "The present work introduces a novel method for Skin - Oxygen Photoplethysmographic Image Analysis (Sophia), which allows oxygen saturation changes to be tracked accurately over time. Sophia uses broad-band lighting and an RGB camera"; p. 3); determining, from at least one of the images, a plurality of microregions in the portion of the person's skin ("Once a search area has been defined in which all the skin to be imaged has been included, the area is divided into N contiguous nxn pixel regions of interest i (n=40) and the algorithm is initiated (Step II). p. 6; "whereas the ROI-extraction method taken from [17] was updated every frame as it was understood to be dynamic. Once the raw red and blue signals were acquired by taking the average of the pixel intensities for these two color channels in the respective ROIs,"; p. 13,[0001]); determining, based on a similarity between particular microregions of the plurality of microregions, one or more regions of interest of the person's skin ("Once a search area has been defined in which all the skin to be imaged has been included, the area is divided into N contiguous nxn pixel regions of interest i (n=40) and the algorithm is initiated (Step II). p. 6; "whereas the ROI-extraction method taken from [17] was updated every frame as it was understood to be dynamic. Once the raw red and blue signals were acquired by taking the average of the pixel intensities for these two-color channels in the respective ROIs,"; p. 13 [0001]); Guazzi does not explicitly teach determining, for each of the regions of interest, a remote photoplethysmogram (rPPG) signal based on the sequence of images; and determining, based on one or more of the rPPG signals, an estimate of an oxygen saturation in the person's blood. However, in the medical imaging devices and methods field of endeavor, Panasyuk discloses hyperspectral/multispectral imaging in determination, assessment and monitoring of systemic physiology and shock, which is analogous art. Panasyuk teaches generating, based on the time-dependent signal ("near-real time information" [0080]. "By revealing changes in tissue oxygen delivery, oxygen extraction, S.sub.HSIO.sub.2 that correlate with adverse outcomes, the MHSI approach is additionally able to provide information about patient survivability" [0096]), a plurality of SpO2 estimates ("hyperspectral tissue oxygen saturation S.sub.HSIO.sub.2" [0051]; "an oxygen saturation value" Claim 62) for each of the plurality of skin pixels at each frame of the captured video ("changes in tissue S.sub.HSIO.sub.2" [0096]. "Oxygen saturation images in which the brightness of each pixel is proportionate to the intensity of the S.sub.HSIO.sub.2 for that pixel" [0196]; "62. calculating an oxygen saturation value for each pixel in the image."); and computing an overall SpO2 estimate from the plurality of SpO2 estimates ("Oxyhemoblobin (OxyHb), deoxyhemoglobin (DeoxyHb) can be presented as scalars reflecting a mean value across a region of interest (ROI), or the oxyhemoblobin and deoxyhemoglobin coefficients can be used to calculate hyperspectral tissue oxygen saturation. This information can be presented as black and white or false color images This presentation can be used to represent oxyhemoglobin and deoxyhemoglobin values for any pixel in the ROI, to present the average oxy hemoglobin and deoxyhemoglobin values over the entire ROI." [0051]). Therefore, based on Panasyuk's teachings, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Guazzi to have the step of determining, for each of the regions of interest, a remote photoplethysmogram (rPPG) signal based on the sequence of images; and determining, based on one or more of the rPPG signals, an estimate of an oxygen saturation in the person's blood as taught by Panasyuk, in order to facilitate the image analysis for contactless SpO2 tracking in tissue over time.
Regarding claims 2 and 14, 18, the portion of the person's skin comprises the person's face ("Fig. 5. A comparison of different ROI selection methods and of the effect of the inclusion criteria on the ROI selection in Sophia, carried out on all subjects considered. From left to right: the method introduced by [0014], using a fixed region manually set onto the lower part of the face; the method introduced by [15], using a fixed region placed on the cheek of the subject;" p. 13).
Regarding claims 3, 15, 19, the similarity between particular microregions of the plurality of microregions comprises a similarity in one or more skin properties corresponding to the respective particular microregions ("Once a search area has been defined in which all the skin to be imaged has been included, the area is divided into N contiguous nxn pixel regions of interest i (n=40) and the algorithm is initiated (Step II). p. 6; "whereas the ROI-extraction method taken from [17] was updated every frame as it was understood to be dynamic. Once the raw red and blue signals were acquired by taking the average of the pixel intensities for these two-color channels in the respective ROIs,"; p. 13 [0001]).
Regarding claims 4, 16, 20, the one or more skin properties of a microregion comprise an average intensity, for each of one or more-color channels, of pixels within that microregion ("2.4. Dataset The algorithm was tested using the video recordings of five volunteers" p. 9; "updated every frame" p. 13, [0001]; "Fig. 5. A comparison of different ROI selection methods and of the effect of the inclusion criteria on the ROI selection in Sophia, carried out on all subjects considered. From left to right: the method introduced by [0014], using a fixed region manually set onto the lower part of the face; the method introduced by [15], using a fixed region placed on the cheek of the subject;" p. 13; "The present work introduces a novel method for Skin - Oxygen Photoplethysmographic Image Analysis (Sophia), which allows oxygen saturation changes to be tracked accurately over time. Sophia uses broad-band lighting and an RGB camera"; p. 3).
Regarding claim 5, determining, based on a similarity between particular microregions of the plurality of microregions, one or more regions of interest of the person's skin comprises: determining, for a pair of particular microregions a difference between the skin properties of the pair of particular microregions; comparing the determining difference in skin properties to a first threshold ("Once a search area has been defined in which all the skin to be imaged has been included, the area is divided into N contiguous nxn pixel regions of interest i (n=40) and the algorithm is initiated (Step II). p. 6; "whereas the ROI-extraction method taken from [17] was updated every frame as it was understood to be dynamic. Once the raw red and blue signals were acquired by taking the average of the pixel intensities for these two-color channels in the respective ROIs,"; p. 13 [0001]); and in response to a determination that the determined difference is less than the first threshold, then combining the pair of particular microregions to form a region of interest ("Once a search area has been defined in which all the skin to be imaged has been included, the area is divided into N contiguous nxn pixel regions of interest i (n=40) and the algorithm is initiated (Step II). p. 6; "whereas the ROI-extraction method taken from [17] was updated every frame as it was understood to be dynamic. Once the raw red and blue signals were acquired by taking the average of the pixel intensities for these two-color channels in the respective ROIs,"; p. 13 [0001]).
Regarding claim 6, further comprising, in response to a determination that the determined difference is not less than first threshold, then: comparing the determined difference to a second threshold ("Once a search area has been defined in which all the skin to be imaged has been included, the area is divided into N contiguous nxn pixel regions of interest i (n=40) and the algorithm is initiated (Step II). p. 6; "whereas the ROI-extraction method taken from [17] was updated every frame as it was understood to be dynamic. Once the raw red and blue signals were acquired by taking the average of the pixel intensities for these two-color channels in the respective ROIs,"; p. 13 [0001]); in response to a determination that the determined difference is less than the second threshold, then determining whether the pair of particular microregions exhibit a spatial symmetry ("Once a search area has been defined in which all the skin to be imaged has been included, the area is divided into N contiguous nxn pixel regions of interest i (n=40) and the algorithm is initiated (Step II). p. 6; "whereas the ROI-extraction method taken from [17] was updated every frame as it was understood to be dynamic. Once the raw red and blue signals were acquired by taking the average of the pixel intensities for these two-color channels in the respective ROIs,"; p. 13 [0001]); and in response to a determination that the pair of particular microregions exhibit a spatial symmetry, then combing the pair of particular microregions to form a region of interest ("Once a search area has been defined in which all the skin to be imaged has been included, the area is divided into N contiguous nxn pixel regions of interest i (n=40) and the algorithm is initiated (Step II). p. 6; "whereas the ROI-extraction method taken from [17] was updated every frame as it was understood to be dynamic. Once the raw red and blue signals were acquired by taking the average of the pixel intensities for these two-color channels in the respective ROIs,"; p. 13 [0001]).
Regarding claim 7, a value of the first threshold is based at least in part on one or more of: a skin tone of the person; an intensity of the ambient light; or a spatially varying distribution of the intensity of the ambient light ("2.4. Dataset The algorithm was tested using the video recordings of five volunteers" p. 9; "updated every frame" p. 13, [0001]; "Fig. 5. A comparison of different ROI selection methods and of the effect of the inclusion criteria on the ROI selection in Sophia, carried out on all subjects considered. From left to right: the method introduced by [0014], using a fixed region manually set onto the lower part of the face; the method introduced by [15], using a fixed region placed on the cheek of the subject;" p. 13; "The present work introduces a novel method for Skin - Oxygen Photoplethysmographic Image Analysis (Sophia), which allows oxygen saturation changes to be tracked accurately over time. Sophia uses broad-band lighting and an RGB camera"; p. 3).
Regarding claim 8, further comprising calculating, for each region of interest, a plurality of first ratios, each first ratio corresponding to a particular color channel ("2.4. Dataset The algorithm was tested using the video recordings of five volunteers" p. 9; "updated every frame" p. 13, [0001]; "Fig. 5. A comparison of different ROI selection methods and of the effect of the inclusion criteria on the ROI selection in Sophia, carried out on all subjects considered. From left to right: the method introduced by [0014], using a fixed region manually set onto the lower part of the face; the method introduced by [15], using a fixed region placed on the cheek of the subject;" p. 13; "The present work introduces a novel method for Skin - Oxygen Photoplethysmographic Image Analysis (Sophia), which allows oxygen saturation changes to be tracked accurately over time. Sophia uses broad-band lighting and an RGB camera"; p. 3).
Regarding claim 9, determining, based on one or more of the rPPG signals, an estimate of an oxygen saturation in the person's blood further comprises estimating the oxygen saturation based on a weighted combination of each rPPG signal, wherein each weight is determined according to a corresponding pulse transit time (Panasyuk [0080], [0096]).
Regarding claim 10, further comprising estimating the oxygen saturation based on a ratio-of-ratios of a pair of color channels, wherein the pair of color channels is determined based on a color temperature of the ambient lighting (Panasyuk [0080], [0096]).
Regarding claim 11, the ambient light comprises light from a display screen of an electronic device ("2.4. Dataset The algorithm was tested using the video recordings of five volunteers" p. 9; "updated every frame" p. 13, [0001]; "Fig. 5. A comparison of different ROI selection methods and of the effect of the inclusion criteria on the ROI selection in Sophia, carried out on all subjects considered. From left to right: the method introduced by [0014], using a fixed region manually set onto the lower part of the face; the method introduced by [15], using a fixed region placed on the cheek of the subject;" p. 13; "The present work introduces a novel method for Skin - Oxygen Photoplethysmographic Image Analysis (Sophia), which allows oxygen saturation changes to be tracked accurately over time. Sophia uses broad-band lighting and an RGB camera"; p. 3).
Regarding claim 12, the light from the display screen consists of a particular color corresponding to one of a plurality of color channels ("2.4. Dataset The algorithm was tested using the video recordings of five volunteers" p. 9; "updated every frame" p. 13, [0001]; "Fig. 5. A comparison of different ROI selection methods and of the effect of the inclusion criteria on the ROI selection in Sophia, carried out on all subjects considered. From left to right: the method introduced by [0014], using a fixed region manually set onto the lower part of the face; the method introduced by [15], using a fixed region placed on the cheek of the subject;" p. 13; "The present work introduces a novel method for Skin - Oxygen Photoplethysmographic Image Analysis (Sophia), which allows oxygen saturation changes to be tracked accurately over time. Sophia uses broad-band lighting and an RGB camera"; p. 3).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MARJAN FARDANESH whose telephone number is (571)270-5508. The examiner can normally be reached Monday-Friday 9:00-17:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jacqueline Cheng can be reached at (571)272-5596. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MARJAN FARDANESH/Primary Examiner, Art Unit 3791