DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 14, 16-18, 23-25 and 33-34 are rejected under 35 U.S.C. 103 as being unpatentable over Chon et al. US2017/0055878 hereinafter referred to as Chon in view of Li et al. US2021/0290166 hereinafter referred to as Li.
As per Claim 1, Chon teaches a computer-implemented method comprising:
- displaying on a screen of a motion/position capture device and/or on a screen remotely connected to the device: (Chon, Paragraph [0068]-[0069], “The smartphone 214 may further include an integrated camera 206a, configurable to capture the first sequence of images and the second sequence of images. The smartphone 214 may further include a user interface 220a. The processor 218a may be further configured to determine the first time period and the second time period based on user interactions 203 with the user 202a via the user interface 220a. The processor 218a may be further configured to output a representation of the breathing information estimate 219a-1 via the user interface 220a. The representation of the breathing information estimate 219a-1 may, for example, be displayed on a screen on the user's smartphone 214. Alternatively or in combination, the representation of the breathing information estimate 219a-1 may be output in the form of audio from the user's smartphone 214.”)
(A) real-time depth video stream of the surface of the torso of a subject in a field of view of a camera of the motion/position capture device, and (B) real-time plot of motion of the surface; (Chon, Paragraph [0137], [0157], “FIG. 11D shows an example of calibrated V.sub.T estimates from the smartphone's chest movement signal, where the figure on top displays the processed waveform and detected breath phase onsets, and the figure at the bottom displays the corresponding V.sub.T estimates during inspiratory phases. The average RR and average V.sub.T are also displayed on this screen” and “In order to overcome the need for an external sensor for the task of RR estimation, i.e., the acoustical sensor, embodiments disclosed herein may take advantage of a smartphone's cameras. In particular, a method according to embodiments disclosed herein allows the real-time acquisition of a surrogate volumetric signal from breathing-related light intensity changes due to chest wall movements was implemented on a smartphone and its performance and was tested in healthy volunteers breathing at a metered pace and spontaneously, while seated. Under the paced breathing, it was found that the smartphone-based estimates of average RR were accurate when compared to those obtained from inductance plethysmography”)
- displaying (A) the real-time depth video stream of the ROI and an adjusted (B) real-time plot of motion for the ROI, wherein the (B) real-time plot of motion for the ROI displays motion of chest and/or abdomen of the subject associated with inhalation and exhalation. (Chon, Paragraph [0137], [0157], “FIG. 11D shows an example of calibrated V.sub.T estimates from the smartphone's chest movement signal, where the figure on top displays the processed waveform and detected breath phase onsets, and the figure at the bottom displays the corresponding V.sub.T estimates during inspiratory phases. The average RR and average V.sub.T are also displayed on this screen” and “In order to overcome the need for an external sensor for the task of RR estimation, i.e., the acoustical sensor, embodiments disclosed herein may take advantage of a smartphone's cameras. In particular, a method according to embodiments disclosed herein allows the real-time acquisition of a surrogate volumetric signal from breathing-related light intensity changes due to chest wall movements was implemented on a smartphone and its performance and was tested in healthy volunteers breathing at a metered pace and spontaneously, while seated. Under the paced breathing, it was found that the smartphone-based estimates of average RR were accurate when compared to those obtained from inductance plethysmography”)
Chon does not explicitly teach - focusing the field of view to capture motion over a region of interest (ROI) on the surface; - capturing motion in the ROI over a period of time; and
Li teaches - focusing the field of view to capture motion over a region of interest (ROI) on the surface; - capturing motion in the ROI over a period of time; and (Li, Paragraph [0148], “In some embodiments, the processing device 120A may obtain the breathing state and/or motion state of the subject through a camera (e.g., the image capturing device 160 or another image capturing device) and/or a sensor. For example, the processing device 120A may obtain a video stream of the chest and/or abdomen of the subject captured by the camera. The processing device 120A may determine the breathing state of the subject by analyzing the motion of the chest and/or abdomen of the subject based on the video stream. As another example, the processing device 120A may monitor the motion of one or more preset points on straps tied to the chest and/or abdomen of the subject based on the video stream to obtain the breathing state of the subject. As still another example, the processing device 120A may monitor airflow in the oral cavity and/or the nasal cavity of the subject through one or more sensors to obtain the breathing state of the subject. As still another example, the processing device 120A may obtain a video stream of the subject through the camera, and determine the motion state of the subject based on the video stream, for example, by tracking the motion of feature point(s) and/or region(s) of interest of the subject based on the video stream of the subject”)
Thus it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to implement the teachings of Li into Chon because by capturing the ROI and not the entire subject will focus on the important regions in order to monitor breathing/motion of the patient.
Therefore it would have been obvious to one of ordinary skill to combine the two references to obtain the invention in Claim 1.
As per Claim 2, Chon in view of Li teaches the method of claim 1, wherein the capturing motion in the ROI over a period of time comprises simultaneously updating (B) real-time plot of motion for the ROI by applying an artificial intelligence (Al) to update motion metrics. (Li, Paragraph [0083] and Chon, Paragraph [0137], [0157])
The rationale applied to the rejection of claim 1 has been incorporated herein.
As per Claim 3, Chon in view of Li teaches the method of claim 1, comprising simultaneously transmitting data comprising (A) the real-time depth video stream of the ROI and the adjusted (B) real-time plot of motion for the ROI to a remote server or to one or more devices for radiation treatment. (Chon, Paragraph [0137], [0157] and Li, Paragraph [0044], [0148])
The rationale applied to the rejection of claim 1 has been incorporated herein.
As per Claim 4, Chon in view of Li teaches the method of claim 1, wherein the screen remotely connected to the device is the screen of a remote monitoring device, a data storage device, a radiation treatment device, and/or an imaging device. (Li, Figure 1, Paragraph [0044], “the medical device 110 may include a radiotherapy (RT) device, such as a conformal radiation therapy device, an image-guided radiation therapy (IGRT) device, an intensity-modulated radiation therapy (IMRT) device, an intensity-modulated arc therapy (IMAT) device, an emission guided radiation therapy (EGRT), or the like. The RT device may be configured to deliver a radiotherapy treatment to a subject”)
The rationale applied to the rejection of claim 1 has been incorporated herein.
As per Claim 14, Chon in view of Li teaches the method of claim 1, wherein the subject has a tumor located in the in the torso, neck or head. (Li, Paragraph [0166], “A region of a subject to be scanned refers to a region of the subject on which a scan is to be performed. For example, the region of the subject to be scanned may include the abdomen, the chest, the brain, the heart, a lung, a region infected by a tumor, or the like, or any combination thereof, of the subject”)
The rationale applied to the rejection of claim 1 has been incorporated herein.
As per Claim 16, Chon in view of Li teaches the method of claim 1, wherein the device is connected to a data storage system, a remote display device, a radiation device, and/or a medical imaging device. (Li, Figure 1, Paragraph [0044], “the medical device 110 may include a radiotherapy (RT) device, such as a conformal radiation therapy device, an image-guided radiation therapy (IGRT) device, an intensity-modulated radiation therapy (IMRT) device, an intensity-modulated arc therapy (IMAT) device, an emission guided radiation therapy (EGRT), or the like. The RT device may be configured to deliver a radiotherapy treatment to a subject”)
The rationale applied to the rejection of claim 1 has been incorporated herein.
As per Claim 17, Chon in view of Li teaches the method of claim 16, wherein the data storage system comprises cloud storage, the remote display device comprises a computer monitor, laptop, smartphone, or another handheld device comprising a screen, and medical imaging device comprises a computer- assisted tomography (CAT) scanner, a magnetic resonance imager, or positron emission tomography (PET) scanner. (Li, Paragraph [0041], “In some embodiments, the medical device 110 may be a non-invasive biomedical medical imaging device for disease diagnostic or research purposes. The medical device 110 may include a single modality scanner and/or a multi-modality scanner. The single modality scanner may include, for example, an ultrasound scanner, an X-ray scanner, an computed tomography (CT) scanner, a magnetic resonance imaging (MRI) scanner, an ultrasonography scanner, a positron emission tomography (PET) scanner”)
The rationale applied to the rejection of claim 16 has been incorporated herein.
As per Claim 18, Chon in view of Li teaches the method of claim 16, wherein the data storage system comprises a medical device, wherein the medical device comprises a radiation therapy device for treatment with breathing maneuvers. (LI, Paragraph [0044], [0148])
The rationale applied to the rejection of claim 16 has been incorporated herein.
As per Claim 22, Chon in view of Li teaches the method of claim 1, further comprising prompting the subject to perform breathing maneuvers comprising holding breath prior to start of the capturing and/or after the start of the focusing. (Chon, Paragraph [0072], [0101]-[0106] and [0137])
The rationale applied to the rejection of claim 1 has been incorporated herein.
As per Claim 23, Chon in view of Li teaches the method of claim 22, further comprising indicating visually or audibly to the subject or a healthcare provider a first period of time the subject performed breath hold based on the analysis of the (B) real-time plot of motion of the surface. (Chon, Paragraph [0072], [0101]-[0106] and [0137])
The rationale applied to the rejection of claim 22 has been incorporated herein.
As per Claim 24, Chon in view of Li teaches the method of claim 23, comprising further prompting the subject to perform breath hold and indicating visually or audibly to the subject a second period of time the subject performed breath hold. (Chon, Paragraph [0072], [0101]-[0106] and [0137])
The rationale applied to the rejection of claim 23 has been incorporated herein.
As per Claim 25, Chon in view of Li teaches the method of claim 22, wherein the steps of prompting are repeated till the subject performs breath hold for a period of time that is determined clinically acceptable. (Chon, Paragraph [0072], [0101]-[0106] and [0137])
The rationale applied to the rejection of claim 22 has been incorporated herein.
As per Claim 33, Chon in view of Li teaches a non-transitory computer-readable medium comprising instructions stored thereon for causing a computer system to implement the methods of 1. (Li, Paragraph[0047] and [0051])
The rationale applied to the rejection of claim 1 has been incorporated herein.
As per Claim 34, Chon in view of Li teaches a computer system comprising the non-transitory computer-readable medium of Claim 33. (Li, Paragraph[0047] and [0051])
The rationale applied to the rejection of claim 1 has been incorporated herein.
Claims 6, 8-9 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over Chon et al. US2017/0055878 hereinafter referred to as Chon in view of Li et al. US2021/0290166 hereinafter referred to as Li as applied to Claim 1 and further in view of Akdemir et al. US2021/0166411 hereinafter referred to as Akedemir.
As per Claim 6, Chon in view of Li teaches the method of claim 1,
Chon in view of Li does not explicitly teach wherein the motion/position capture device comprises a mobile device or a smartphone with a depth sensor.
Akdemir teaches wherein the motion/position capture device comprises a mobile device or a smartphone with a depth sensor. (Akdemir, Paragraph [0089], [0096], “The mobile device 130 may collect depth images from a camera 142a, in which case a front camera and/or a back camera, for example, may be employed”)
Thus it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to implement the teachings of Akdemir into Chon in view of Li because by utilizing a mobile device with two cameras, one front and one back will allow images to be taken by both cameras.
Therefore it would have been obvious to one of ordinary skill to combine the three references to obtain the invention in Claim 6.
As per Claim 8, Chon in view of Li and Akdemir teaches the method of claim 6, wherein the smartphone operates on an operating system that is configured for sensing depth. (Li, Paragraph 0057], “The image capturing device 160 may be and/or include any suitable device that is capable of capturing image data of the subject. For example, the image capturing device 160 may include a camera (e.g., a digital camera, an analog camera, etc.), a red-green-blue (RGB) sensor, an RGB-depth (RGB-D) sensor, or another device that can capture color image data of the subject”)
The rationale applied to the rejection of claim 6 has been incorporated herein.
As per Claim 9, Chon in view of Li and Akdemir teaches the method of claim 8, wherein the operating system comprises an AppleTM operating system or an Android operating system. (Li, Paragraph [0069], “FIG. 3, the mobile device 300 may include a communication platform 310, a display 320, a graphics processing unit (GPU) 330, a central processing unit (CPU) 340, an I/O 350, a memory 360, and storage 390. In some embodiments, any other suitable component, including but not limited to a system bus or a controller (not shown), may also be included in the mobile device 300. In some embodiments, a mobile operating system 370 (e.g., iOS™, Android™, Windows Phone™)”)
The rationale applied to the rejection of claim 8 has been incorporated herein.
As per Claim 11, Chon in view of Li teaches the method of claim 1,
Chon in view of Li does not explicitly teach wherein the device comprises a front-facing camera located on the same side of the device as the screen of the device, and/or a back-facing camera located on the backside of the device relative to the screen of the device.
Akdemir teaches wherein the device comprises a front-facing camera located on the same side of the device as the screen of the device, and/or a back-facing camera located on the backside of the device relative to the screen of the device. (Akdemir, Paragraph [0089], [0096], “The mobile device 130 may collect depth images from a camera 142a, in which case a front camera and/or a back camera, for example, may be employed”)
Thus it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to implement the teachings of Akdemir into Chon in view of Li because by utilizing a mobile device with two cameras, one front and one back will allow images to be taken by both cameras.
Therefore it would have been obvious to one of ordinary skill to combine the three references to obtain the invention in Claim 11.
Claim 26 is rejected under 35 U.S.C. 103 as being unpatentable over Chon et al. US2017/0055878 hereinafter referred to as Chon in view of Li et al. US2021/0290166 hereinafter referred to as Li as applied to Claim 25 and further in view of El Kaffas et al. US2020/0323516 hereinafter referred to as El Kaffas.
As per Claim 26, Chon in view of Li teaches the method of claim 25,
Chon in view of Li does not explicitly teach wherein the period of time that is determined clinically acceptable is at least 20 seconds.
El Kaffas teaches wherein the period of time that is determined clinically acceptable is at least 20 seconds. (El Kaffas, Paragraph [0028], “all patients were asked to either hold their breath (for up to 30 seconds) or to breathe shallow (in patients unable to hold their breaths) to minimize motion artifacts”)
Thus it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to implement the teachings of El Kaffas into Chon in view of Li because by holding breath while images are being taken will reduce motion artifacts in the images being taken.
Therefore it would have been obvious to one of ordinary skill to combine the three references to obtain the invention in Claim 26.
Claim 30 is rejected under 35 U.S.C. 103 as being unpatentable over Chon et al. US2017/0055878 hereinafter referred to as Chon in view of Li et al. US2021/0290166 hereinafter referred to as Li as applied to Claim 1 and further in view of Ohara US2019/0357866.
As per Claim 30, Chon in view of Li teaches the method of claim 1, wherein the method comprises relaying exhalation after end of a breath hold by the subject to a radiation device and/or an imaging device; (Li, Figure 1, Paragraph [0044])
Chon in view of Li does not explicitly teach instructing a radiation device to pause radiation being delivered to the ROI when the subject exhales after the end of breath hold; or instructing an imaging device to pause imaging of the region of interest when the subject exhales after the end of breath hold.
Ohara teaches instructing a radiation device to pause radiation being delivered to the ROI when the subject exhales after the end of breath hold; or instructing an imaging device to pause imaging of the region of interest when the subject exhales after the end of breath hold. (Ohara, Figure 7, Paragraph [0095], “a voice instructing a breath holding motion, a voice instructing an exhaling motion, a voice instructing a breath holding motion, a voice instructing an inhaling motion, and a voice notifying the end of imaging are outputted at predetermined timings from the motion instruction outputter 44. The subject takes a deep breath in accordance with the auto voice (voice guide). When the radiographer brings the hand switch 321 into a state further depressed (two-step depression) five seconds after the hand switch 321 is depressed one step (in the state where the subject has breathed deeply), irradiation of radiation is started by the radiation source 52. Further, at a timing at which a voice saying “OK, imaging is over.” is outputted by the motion instruction outputter 44, the radiographer brings the hand switch 321 into an OFF state, and thereby irradiation of radiation by the radiation source 52 is stopped”)
Thus it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to implement the teachings of Ohara into Chon in view of Li because by taking images while the patient holds their breath will reduce motion artifacts in the images being taken.
Therefore it would have been obvious to one of ordinary skill to combine the three references to obtain the invention in Claim 30.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MING HON whose telephone number is (571)270-5245. The examiner can normally be reached M-F 9am - 5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emily Terrell can be reached on 571-270-3717. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MING Y HON/Primary Examiner, Art Unit 2666
Prosecution Insights