DETAILED ACTION
Claims 1-18 are pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on September 5, 2024 has been considered by the examiner.
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d).
Claim Objections
Claims 1 and 14 are objected to because of the following informalities:
Claim 1 line 6 states, “wherein at least one the image data and the additional data includes information regarding one or more vital signs of a patient being the target of that image source” (emphasis added). This appears to be a typographical error and should be “at least one of the” (emphasis added).
Claim 14 line 3 states, “wherein the control signal is configured for controlling the display device to display data from number of image sources, wherein the number of image sources” (emphasis added). This appears to be a typographical error and should be “from a number of image sources” (emphasis added).
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 16 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 16 recites the limitation "the priority" in line 4. There is insufficient antecedent basis for this limitation in the claim. There is no recitation of “a priority” in claim 1. It is unclear what “the priority” is referring to. Therefore, applicant has failed to particularly point out and distinctly claim the subject matter which the inventor or joint inventor regards as the invention.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 6, 11-12, 15, and 17-18 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Receveur et al. (U.S. Publication No. US 20210219873 A1) ("Receveur").
Regarding claim 1, Receveur discloses a medical image processing device (Fig. 2; [0003-0006]) for controlling display of an image from a plurality of image sources ([0061-0062]), the medical image processing device comprising processing circuitry (Fig. 5; [0082-0083]) configured to:
receive a plurality of image signals from a respective plurality of image sources (Fig. 4, step 302; [0043-0045], [0053], and [0061], wherein the video feeds (i.e. image signals) from one or more cameras (i.e. respective plurality of image sources) are received), the plurality of image signals each including image data for display ([0041], [0043], and [0061], wherein each video feed has image data for display at the centralized monitoring station) and additional data ([0045-0046], wherein patient monitoring computing device concurrently receives data (i.e. additional data) from patient monitoring devices including heart rate monitors and pulse oximeters), wherein at least one the image data and the additional data includes information regarding one or more vital signs of a patient being the target of that image source ([0045-0046], wherein the heart rate monitors and pulse oximeters measure vital signs of the patient at the bed being monitored by the camera, and these vital signs are processed by the patient monitoring computing device alongside the video feed from the corresponding camera);
generate an assessment for each of the plurality of image signals in accordance with processing performed on at least one of the image data and the additional data of each image signal (Fig. 4; [0054], [0057], and [0075-0077], wherein the video images (i.e. image data) are processed and assessed to determine whether a condition has occurred with a patient); and
generate a control signal for controlling a display device to display an image comprising image data of at least one of the image signals from the plurality of image sources in accordance with the assessment which has been generated for each of the plurality of image signals ([0060-0062], wherein the alert system uses assessed conditions (i.e. assessment) of the image data to control the display at the centralized monitoring station (i.e. display device) by modifying and highlighting the video feed in color based on the assessed conditions to indicate which patients require caregiver attention).
Claims 17 and 18 have limitations that are substantially similar to claim 1. Therefore, the rejection applied to claim 1, please see above, also applies equally to claims 17 and 18. Furthermore, Receveur teaches a medical image processing method of controlling display of an image from a plurality of image sources (Fig. 4, [0003-0006] and [0073]) and a non-transitory computer readable medium ([0084]) storing a computer program product comprising instructions ([0084]) which, when the instructions are implemented by a computer (Fig. 1; [0051]), cause the computer to perform a medical image processing method of controlling display of an image from a plurality of image sources ([0061-0062]).
Regarding claim 6, Receveur discloses the medical image processing device according to claim 1, wherein the processing performed on the at least one of the image data and the additional data of each image signal is processing to determine the priority of the image signal from each image source ([0057], [0060], and [0079], wherein the condition assessor and alert system selects (i.e. prioritizes) video feeds based on assessed conditions where certain conditions, such as a patient fall or anticipated bed exit, will trigger an alert while other conditions, such as sleeping, do not trigger an alert, establishing a priority of image signals).
Regarding claim 11, Receveur discloses the medical image processing device according to claim 1, wherein the control signal is configured for controlling a display device to display an image comprising image data from a plurality of image sources in accordance with the assessment which has been generated for each of the plurality of image sources ([0060-0062], wherein the alert system uses assessed conditions (i.e. assessment) of the image data to control the display at the centralized monitoring station (i.e. display device) by modifying and highlighting the video feed in color based on the assessed conditions to indicate which patients require caregiver attention).
Regarding claim 12, Receveur discloses the medical image processing device according to claim 11, wherein the control signal is configured for controlling the display device to display the plurality of image sources in separate image panels, with a coloured outline on each panel determined in in accordance with the assessment which has been generated for each of the plurality of image signals ([0060-0062], wherein the alert system uses assessed conditions (i.e. assessment) of the image data to control the display at the centralized monitoring station (i.e. display device) by modifying and highlighting the entire video feed in color (i.e. coloured outline) based on the assessed conditions to indicate which patients require caregiver attention).
Regarding claim 15, Receveur discloses the medical image processing device according to claim 1, wherein the processing performed on the at least one of the image data and the additional data of each image signal to generate an assessment for each of the plurality of image signals includes use of a machine learning or neural network model ([0054], [0056], and [0071], wherein the analysis of the images (i.e. image data) to extract features for the assessment utilize convolutional neural networks.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 2 and 4 are rejected under 35 U.S.C. 103 as being unpatentable over Receveur et al. (U.S. Publication No. US 20210219873 A1) ("Receveur") in view of Johnson et al. (U.S. Publication No. US 20150199892 A1) ("Johnson").
Regarding claim 2, Receveur discloses the medical image processing device according to claim 1.
However, Receveur fails to teach wherein the processing performed on the at least one of the image data and the additional data of each image signal is processing to determine whether the image signal from each image source is active or inactive. Johnson, on the other hand, teaches movement activity within the video frame determines whether the event is active or inactive. More specifically and as it relates to the applicant’s claims, Johnson discloses wherein the processing performed on the at least one of the image data and the additional data of each image signal is processing to determine whether the image signal from each image source is active or inactive ([0070-0072], wherein movement activity within the video frame (i.e. image data) is considered active and no change detected between a pair of video frames is considered inactive).
Johnson is combinable with Receveur because they are from the same art of image processing.
The suggestion/motivation for doing so would have been to enable further discrimination of mere changes between video frames from motion, and more importantly, to further distinguish the types of patient movement from mere motion detected within the patient's room (Johnson, [0077]).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate wherein the processing performed on the at least one of the image data and the additional data of each image signal is processing to determine whether the image signal from each image source is active or inactive, as taught by Johnson, into the medical image processing device, as taught by Receveur, to obtain the invention as specified in claim 2.
Regarding claim 4, Receveur and Johnson disclose the medical image processing device according to claim 2.
However, Receveur fails to teach wherein the processing to determine whether the image signal from each image source is active or inactive is based on at least one of a level of motion detected in the image data, a type of an action detected in the image data, a type of person detected in the image data, information regarding a noise level determined from the additional data, schedule information determined from the additional data and the vital signs of a patient being the target of that image source. Johnson, on the other hand, teaches determining activity or inactivity based on movement. More specifically, and as it relates to the applicant’s claims, Johnson discloses wherein the processing to determine whether the image signal from each image source is active or inactive is based on at least one of a level of motion detected in the image data ([0070-0072], wherein movement activity within the video frame (i.e. image data) is considered active and no change detected between a pair of video frames is considered inactive), a type of an action detected in the image data, a type of person detected in the image data, information regarding a noise level determined from the additional data, schedule information determined from the additional data and the vital signs of a patient being the target of that image source.
Johnson is combinable with Receveur because they are from the same art of image processing.
The suggestion/motivation for doing so would have been to enable further discrimination of mere changes between video frames from motion, and more importantly, to further distinguish the types of patient movement from mere motion detected within the patient's room (Johnson, [0077]).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate wherein the processing to determine whether the image signal from each image source is active or inactive is based on at least one of a level of motion detected in the image data, a type of an action detected in the image data, a type of person detected in the image data, information regarding a noise level determined from the additional data, schedule information determined from the additional data and the vital signs of a patient being the target of that image source. Johnson, on the other hand, teaches determining activity or inactivity based on movement., as taught by Johnson, into the medical image processing device, as taught by Receveur and Johnson, to obtain the invention as specified in claim 4.
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Receveur et al. (U.S. Publication No. US 20210219873 A1) ("Receveur") in view of Johnson et al. (U.S. Publication No. US 20150199892 A1) ("Johnson") and further in view of Nam (U.S. Publication No. US 20210335071 A1) (“Nam”).
Regarding claim 3, Receveur and Johnson disclose the medical image processing device according to claim 2.
However, Receveur and Johnson fail to teach wherein the processing circuitry is configured to determine that an image signal from an image source is inactive if the image data of the image signal does not contain any image information. Nam, on the other hand, teaches a camera being in standby mode when not collecting image data. More specifically and as it relates to the applicant’s claims, Nam discloses wherein the processing circuitry is configured to determine that an image signal from an image source is inactive if the image data of the image signal does not contain any image information ([0087], wherein a camera (i.e. image source) is in standby mode when not enabled and is not collecting image data (i.e. inactive when image data does not contain any image information)).
Nam is combinable with Receveur and Johnson because they are from the same art of image processing.
The suggestion/motivation for doing so would have been to the reduce the amount of CPU hours, RAM, and memory spent on having to collect, store, and/or process image (Nam, [0092]).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate wherein the processing circuitry is configured to determine that an image signal from an image source is inactive if the image data of the image signal does not contain any image information, as taught by Nam, into the medical image processing device according to claim 2, as taught by Receuver and Johnson, to obtain the invention as specified in claim 3.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Receveur et al. (U.S. Publication No. US 20210219873 A1) ("Receveur") in view of Johnson et al. (U.S. Publication No. US 20150199892 A1) ("Johnson") and further in view of Wallace (U.S. Publication No. US 20200367762 A1) (“Wallace”).
Regarding claim 5, Receveur and Johnson disclose the medical image processing device according to claim 2.
However, Receveur and Johnson fail to teach wherein the control signal is configured for controlling the display device to display an image comprising image data from a given image source when the image signal from that image source is active. Wallace, on the other hand, teaches triggering and displaying video observation when alerted. More specifically and as it relates to the applicant’s claims, Wallace discloses wherein the control signal is configured for controlling the display device to display an image comprising image data from a given image source when the image signal from that image source is active ([0074], wherein a camera identifies movement (i.e. active image source) and triggers and displays video observation (i.e. displaying an image comprising image data) of the patient when alerted).
Wallace is combinable with Receveur and Johnson because they are from the same art of image processing.
The suggestion/motivation for doing so would have been to reduce patient falls and notify personnel (Wallace, [0074]).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate wherein the control signal is configured for controlling the display device to display an image comprising image data from a given image source when the image signal from that image source is active, as taught by Wallace, into the medical image processing device, as taught by Receveur and Johnson, to obtain the invention as specified in claim 5.
Claims 7-9 are rejected under 35 U.S.C. 103 as being unpatentable over Receveur et al. (U.S. Publication No. US 20210219873 A1) ("Receveur") in view of Foos (U.S. Publication No. US 20230021568 A1) (“Foos”).
Regarding claim 7, Receveur disclose the medical image processing device according to claim 6.
However, Receveur fail to teach wherein the priority of the image signal from each image source is determined based on the information regarding one or more vital signs of the patient being the target of that image source. Foos, on the other hand, teaches analyzing vital signs to assess whether or not there is urgency related to any particular change in value. More specifically and as it relates to the applicant’s claims, Foos discloses wherein the priority of the image signal from each image source is determined based on the information regarding one or more vital signs of the patient being the target of that image source ([0035], wherein vital signs are assessed for urgency related to any change in value).
Foos is combinable with Receveur and Johnson because they are from the same art of image processing.
The suggestion/motivation for doing so would have been to urgently alert staff of the requirement for remedial activity (Wallace, [0035]).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate wherein the priority of the image signal from each image source is determined based on the information regarding one or more vital signs of the patient being the target of that image source, as taught by Foos, into the medical image processing device, as taught by Receveur and Johnson, to obtain the invention as specified in claim 7.
As for claim 8, Foos discloses that vital signs of the patient include at least one of the body temperature of the patient, the pulse rate of the patient, the respiration rate of the patient and the blood pressure of the patient. See paragraph 0003 where a heart rate is listed a monitored vital sign.
Regarding claim 9, Receveur discloses the medical imaging apparatus according to claim 6.
However, Receveur fails to teach wherein the priority of the image signal from each image source located in an operation room is determined in accordance with at least one of a level of skill of medical staff performing a medical operation, a level of experience of medical staff performing a
medical operation, a type of surgery being performed and a duration of a surgery being performed. Wolf, on the other hand, teaches selecting video footage within an operating room for display based on surgeon skill level. More specifically and as it relates to the applicant’s claims, Wolf discloses wherein the priority of the image signal from each image source located in an operation room (column 88 lines 54-55, wherein an operating room is a room within a hospital) is determined in accordance with at least one of a level of skill of medical staff performing a medical operation (column 41 lines 20-27; column 42 lines 33 and 53-63, wherein video footage (i.e. image signal) is associated with surgeon skill level and a video footage is selected (i.e. prioritized) for display based on this association), a level of experience of medical staff performing a medical operation, a type of surgery being performed and a duration of a surgery being performed.
Wolf is combinable with Receveur because they are from the same art of image processing.
The suggestion/motivation for doing so would have been to enable analysis of current and/or historical surgical footage to identify features of surgery, patient conditions, and other features to predict and improve surgical outcomes (Wolf, column 154 lines 5-8).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate wherein the priority of the image signal from each image source located in an operation room is determined in accordance with at least one of a level of skill of medical staff performing a medical operation, a level of experience of medical staff performing a medical operation, a type of surgery being performed and a duration of a surgery being performed, as taught by Wolf, into the medical image processing device, as taught by Receveur, to obtain the invention as specified in claim 9.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Receveur et al. (U.S. Publication No. US 20210219873 A1) ("Receveur") in view of Wallace (U.S. Publication No. US 20200367762 A1) (“Wallace”).
Regarding claim 10, Receveur discloses the medical imaging apparatus according to claim 6.
Although Receveur discloses wherein the priority of the image signal from each image source is determined in accordance with at least one of whether a bed is used or not, a level of skill of medical staff caring for the patient, a level of experience of medical staff caring for the patient, a characteristic of the patient ([0077] and [0079-0080], wherein selected (i.e. prioritized) video feeds (i.e. image signal) with assessed conditions, such as a patient fall or bed exit which are determined by movement characteristics of the patient (i.e. characteristic of the patient), will trigger an alert), a type of disease suffered by the patient, and a type of injury suffered by the patient, Receveur fails to specifically teach image sources being located in an intensive care unit. Wallace, on the other hand, teaches an intensive care unit as an example of a hospital room ([0020]).
Wallace is combinable with Receveur because they are from the same art of image processing.
The suggestion/motivation for doing so would have been to include a versatile "monitoring device 20 [which] may be positioned in a variety of locations (e.g., room 10), including, but not limited to, hospital rooms including intensive care unit, step down units, and standard hospital beds, in nursing homes, assisted living, or in private homes" (Wallace, [0020]). Receveur could benefit from being utilized in a variety of locations as shown by Wallace.
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate image sources located in an intensive care unit, as taught by Wallace, into the medical image processing device, as taught by Receveur, to obtain the invention as specified in claim 10.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Receveur et al. (U.S. Publication No. US 20210219873 A1) ("Receveur") in view of Fu et al. (U.S. Patent No. US 11678011 B1) (“Fu”).
Regarding claim 13, Receveur discloses the medical imaging apparatus according to claim 12.
However, Receveur fails to teach wherein the control signal is configured for controlling the display device to display the image sources in the separate image panels wherein at least one of the size of each image panel and the layout of the image panels is determined in accordance with the assessment which has been generated for each of the plurality of image signals. Fu, on the other hand, teaches a change in size and layout of the video feeds based on priority of detected events. More specifically and as it relates to the applicant’s claims, Fu discloses wherein the control signal is configured for controlling the display device to display the image sources in the separate image panels wherein at least one of the size of each image panel and the layout of the image panels is determined in accordance with the assessment which has been generated for each of the plurality of image signals (column 3 lines 23-26, wherein events are detected based on motion (i.e. generated assessment); Fig. 5 and 6; wherein video streams (i.e. image sources) displayed as video feeds (i.e. video panels) are enlarged or reduced in size, resulting in a changed layout, in response to the detected events).
Fu is combinable with Receveur because they are from the same art of image processing.
The suggestion/motivation for doing so would have been to enable an operator to see more detail in the enlarged video stream, with a detected event, compared to the amount of detail in the other video streams (column 27 lines 10-14).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate wherein the control signal is configured for controlling the display device to display the image sources in the separate image panels wherein at least one of the size of each image panel and the layout of the image panels is determined in accordance with the assessment which has been generated for each of the plurality of image signals, as taught by Fu, into the medical image processing device, as taught by Receveur, to obtain the invention as specified in claim 13.
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Receveur et al. (U.S. Publication No. US 20210219873 A1) ("Receveur") in view of Drive et al. (U.S. Patent No. US 8049748 B2) (“Drive”).
Regarding claim 14, Receveur discloses the medical image processing device according to claim 1.
However, Receveur fails to teach wherein the control signal is configured for controlling the display device to display data from number of image sources, wherein the number of image sources which form the plurality of image sources is determined in accordance with the assessment which has been generated for each of the image signals. Drive, on the other hand, teaches only displaying video feeds with detected motion events. More specifically and as it relates to the applicant’s claims, Drive discloses wherein the control signal is configured for controlling the display device to display data from number of image sources, wherein the number of image sources which form the plurality of image sources is determined in accordance with the assessment which has been generated for each of the image signals (column 3 lines 59-67 to column 4 lines 1-7, wherein video feeds (i.e. image sources) with events detected by motion (i.e. generated assessment) are added to the display while video feeds with no events are dropped from the display).
Drive is combinable with Receveur because they are from the same art of image processing.
The suggestion/motivation for doing so would have been to increase the amount of information provided to the operator so that the operator has more situational awareness (Drive, column 1 lines 51-53).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate wherein the control signal is configured for controlling the display device to display data from number of image sources, wherein the number of image sources which form the plurality of image sources is determined in accordance with the assessment which has been generated for each of the image signals, as taught by Drive, into the medical image processing device, as taught by Receveur, to obtain the invention as specified in claim 14.
Allowable Subject Matter
Claim 16 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 16, the primary reason for indication of allowable subject matter is that the prior art fails to teach or reasonably suggest wherein the assessment for each the plurality of images includes a score determined based on at least one of whether the image signal from each image source is active or inactive and the priority of the image signal from each image source, in combination with the other elements of the claim.
The closest prior arts, Receveur et al. (U.S. Publication No. US 20210219873 A1) and Derenne et al. (U.S. Publication No. US 20150109442 A1) teach the following:
Receveur teaches processing and assessing video images to determine whether a condition has occurred with a patient at the bed, but does not teach that assessment including a score based on at least one of whether the image signal from each image source is active or inactive and the priority of the image signal from each image source.
Derenne teaches generating a score based on the gait of the patient and alerting the patient’s caregiver if that score exceeds a threshold indicating a fall risk, which teaches the score determining the priority of the image signal from each image source and not the score determined based on the priority of the image signal from each image source.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RACHEL Y DANG whose telephone number is (571)438-9519. The examiner can normally be reached Monday - Thursday: 7am - 4:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John Villecco can be reached at (571) 272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RACHEL Y DANG/Examiner, Art Unit 2661
/JOHN VILLECCO/Supervisory Patent Examiner, Art Unit 2661
Prosecution Insights