Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 41-48, 53 and 55-60 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter as follows. Regarding claims 41, 59, and 60, the claims are directed to an abstract idea, namely mental processes. The claims are not integrated into a practical application and the claims lack an inventive concept. Furthermore, claims 42-48, 53, and 55-58 are also directed to an abstract idea, specifically, evaluating features in an environment and informing a person associated with a medical procedure. Thus, all the listed claims are considered non-statutory subject matter.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 41-48, 53, 55-60 are rejected under 35 U.S.C. 102(a)(1) and 35 U.S.C. 102(a)(2) as being anticipated by Talmor et al. (US 20200335208 A1), hereinafter Talmor.
Regarding claim 41, Talmor teaches A non-transitory computer readable medium including instructions that, when executed by at least one processor, cause the at least one processor to (Para. 57 see "a computer program product accessible from a computer-usable or computer-readable medium providing program code for use by or in connection with a computer or any instruction execution system." Para. 59 see "a data processing system suitable for storing and/or executing program code may include at least one processor coupled directly or indirectly to memory elements." Para. 61 see "The solutions, techniques, systems, devices, components and methods in the specification and accompanying drawings are described in the context of a medical procedure."). perform operations for addressing time sensitive situations in surgical procedures, the operations comprising: (Abstract see "Disclosed is a system for medical procedure tracking, evaluation and assistance, wherein one or more video cameras, one or more acoustic sensors or one or more medical device interfaces acquire video, audio or medical device feeds from a medical treatment setting. A scene evaluation module detects scene related features in the video, audio or medical device feeds. A procedure compliance assessment module compares one or more scene related features detected and reported by the scene evaluation module to a list of expected actions or equipment usages associated with the procedure being performed in the treatment setting. A procedure assistance module provides compliance based procedure related action recommendations or instructions from within the list of expected actions or equipment usages." Para. 241 see "system provided resuscitation relevant recommendations/indications/notifications, given to medical staff member(s), include: ‘You have initiated the resuscitation procedure 2 minutes ago, and an electric shock has NOT yet been provided—please give a shock ASAP’; ‘Chest compressions being performed are too slow, please speed up their frequency’; ‘It is time for drug therapy, please give patient 1 milligram of epinephrine’; ‘The staff member command given was referred to a wrong dosage, please give 1 milligram of epinephrine’; and ‘You have provided the drug therapy, but have not flushed’.). receiving first surgical footage captured using at least one image sensor from an ongoing surgical procedure; (Para. 66 see "a procedure setting data acquisition module, having one or more, or any number of, video cameras."). analyzing the first surgical footage to identify a time sensitive situation; (Para. 67 see "Video feeds from the camera(s), audio feeds from the audio sensors and/or medical device outputs, may be provided to a scene/setting evaluation module which may include signal processing circuits and algorithms—optionally including and utilizing artificial intelligence, machine learning and deep learning models and methodologies—to detect scene related features, including performed actions and medical equipment usage... from the specific treatment setting covered by the cameras and audio sensors, the medical devices monitoring the treated patient and/or retrieved medical records of the patient." Para. 241 discloses monitoring time sensitive situations.). selecting a time period for initiating an action to address the time sensitive situation; (Para. 142 see "System notifications, alerts, messages and/or assistive tools, may be provided substantially in real time and/or may be provided at time points set by the system based on the flow and logic of the medical procedure being carried-out and/or sub steps thereof." Para. 241 discloses monitoring time sensitive situations.). receiving second surgical footage captured using the at least one image sensor from the ongoing surgical procedure after the identification of the time sensitive situation; (Para. 75 see "system for tracking, evaluating and facilitating medical procedures, in accordance with some embodiments; the shown system includes: A microphone/sound-recording unit, a camera/imaging unit and a medical device/s and/or medical sensor/s interface unit—adapted for monitoring, optionally in parallel/concert, the medical procedure being carried out and its environment and for collecting information about it." Para. 236 see "Data is continuously/intermittently received or retrieved from interfaced medical devices, the patient's medical records and additional inputs from the environment and from functionally associated devices and systems."). analyzing the second surgical footage to determine that no action to address the time sensitive situation was initiated within the selected time period; (Para. 118 see "Mismatchings, deltas, or deviations/divergences, between medical setting/scene features and respective reference records of the identified procedure/steps, and/or differences/divergences there between, may be registered. Mismatchings or deltas may for example include: differences in the positions and/or orientations of medical staff members and medical equipment or devices: procedure step actions not performed or performed differently from their respective description in the procedure records; commands or requests by medical staff members and medical equipment or devices, not executed; medical staff members and medical equipment or devices not present at the procedure setting; auxiliary medical support and decision support tools and resources not present, not being used, or being used not in accordance with the procedure records; medicines/drugs given to the patient at times and/or dosages mismatching the procedure records; measured physiological parameters deviating/diverging from those that are typical for the procedure; and/or other." Para. 241 discloses time sensitive situations where a notification is provided when an action has not occurred.). and in response to the determination that no action to address the time sensitive situation was initiated within the selected time period, providing information indicative of a need to address the time sensitive situation. (Para. 69 see "A procedure assistance module, based on the results of the comparison of the setting features to the expected actions and usages associated with the procedure, may generate and provide suggestions and recommendations to medical staff members participating in the procedure and/or commands to medical devices utilized as part of the procedure." Para. 139 see "A Notification and Recommendation Module, or output unit, may issue notifications/advises/notices/alerts to medical staff members, based on the features detected/recognized by the Procedure Setting Features Recognition Module and/or based on their deviation/divergence from, or correlation to, existing procedure records." Para. 142 see "System notifications, alerts, messages and/or assistive tools, may be provided substantially in real time and/or may be provided at time points set by the system based on the flow and logic of the medical procedure being carried-out and/or sub steps thereof." Para. 241 see "You have initiated the resuscitation procedure 2 minutes ago, and an electric shock has NOT yet been provided—please give a shock ASAP.").
Regarding claim 42, Talmor teaches The non-transitory computer readable medium of claim 41. wherein the time sensitive situation includes at least one of a bleeding, leakage, blockage of blood flow, or compression. (Para. 241 see "Chest compressions being performed are too slow, please speed up their frequency.").
Regarding claim 43, Talmor teaches The non-transitory computer readable medium of claim 41. wherein the operations further comprises analyzing the first surgical footage to select the time period for initiating the action to address the time sensitive situation. (Para. 142 see "System notifications, alerts, messages and/or assistive tools, may be provided substantially in real time and/or may be provided at time points set by the system based on the flow and logic of the medical procedure being carried-out and/or sub steps thereof.").
Regarding claim 44, Talmor teaches The non-transitory computer readable medium of claim 41. wherein the selection of the time period for initiating the action to address the time sensitive situation is based on a patient associated with the ongoing surgical procedure. (Para. 63 see "The analysis results may be documented/logged at least partially in respect of: the identified procedure, the identified procedure step and/or information about the specific patient on which the procedure is being carried out." Para. 237 "Potentially relevant data inputs, retrieved by the system from the patient's medical records, include: Allergies (e.g. an allergy to latex gloves that may indicate the medical team not to use gloves); Age (e.g. infant/child/adult)—may affect the resuscitation algorithm/protocol type; Weight—may affect resuscitation related medicine/drug dosages; and Other important medical history records that may raise concerns and/or alter/modify treatments.").
Regarding claim 45, Talmor teaches The non-transitory computer readable medium of claim 41. wherein the operations further comprises providing second information in response to the identification of the time sensitive situation, the second information differs from the information provided in response to the determination that no action to address the time sensitive situation was initiated within the selected time period. (Para. 241 see "system provided resuscitation relevant recommendations/indications/notifications, given to medical staff member(s), include... ‘It is time for drug therapy, please give patient 1 milligram of epinephrine’... ‘You have provided the drug therapy, but have not flushed’." (Examiner note: the system instructs giving 1mg of epinephrine during the time sensitive situation period. The system later identifies the dosage was given but must flush.)).
Regarding claim 46, Talmor teaches The non-transitory computer readable medium of claim 41. wherein the operations further comprising: analyzing the second surgical footage to detect a surgical instrument; (Para. 108 see "A Procedure Setting Features Recognition Module, or unit, may detect, extract and/or recognize features from the medical procedure setting, wherein features may include any combination of: medical procedure staff members; the subject/patient being treated as part of the medical procedure; and/or equipment, devices, tools and/or medications/drugs present in the medical environment." Para. 110 see "the procedure setting features recognition module may include video signature detection functionality for correlating detected images—for example, utilizing optical face/movement recognition and/or optical item identification technologies/methodologies implementing logic(s)/circuits—with specific actions, specific medical equipment usage, specific medical tools usage and/or specific chemicals/medicines/drugs usage."). analyzing the second surgical footage to determine a type of the surgical instrument; (Para. 108 see "Extracted features may be detected, may include, may be characterized and/or their type/role/action/activity may be recognized."). determining a relationship between the type of the surgical instrument and the time sensitive situation; (Para. 117 see "A Procedure Compliance Assessment Module, or unit, may compare one or more of the medical setting related features detected/recognized by the Procedure Setting Features Recognition Module to a list of characteristics—for example, expected actions or equipment usages within the reference records of multiple medical procedures—associated with the specific procedure, and/or specific procedure step(s), identified by the Procedure Identification Module as being performed/carried-out in the procedure setting."). and basing the determination that no action to address the time sensitive situation was initiated within the selected time period on the relationship between the type of the surgical instrument and the time sensitive situation. (Para. 118 see "Mismatchings, deltas, or deviations/divergences, between medical setting/scene features and respective reference records of the identified procedure/steps, and/or differences/divergences there between, may be registered. Mismatchings or deltas may for example include... commands or requests by medical staff members and medical equipment or devices, not executed; medical staff members and medical equipment or devices not present at the procedure setting; auxiliary medical support and decision support tools and resources not present, not being used, or being used not in accordance with the procedure records; medicines/drugs given to the patient at times and/or dosages mismatching the procedure records; measured physiological parameters deviating/diverging from those that are typical for the procedure; and/or other." Para. 241 discloses time sensitive situations where a notification is provided when an action has not occurred.).
Regarding claim 47, Talmor teaches The non-transitory computer readable medium of claim 41. wherein the operations further comprising: analyzing the second surgical footage to detect an interaction between a surgical instrument and an anatomical structure; (Para. 108 see "A Procedure Setting Features Recognition Module, or unit, may detect, extract and/or recognize features from the medical procedure setting, wherein features may include any combination of... the subject/patient being treated as part of the medical procedure; and/or equipment, devices, tools... Extracted features may be detected... based on a combination of... shapes, positions and/or orientations, in the acquired images/video." Para. 110 see "the procedure setting features recognition module may include video signature detection functionality for correlating detected images... with specific actions, specific medical equipment usage, specific medical tools usage." (Examiner: Shapes may be extracted features of a patient who is an anatomical structure or body parts of the patient. The system monitors medical tool usage and actions performed.)). and basing the determination that no action to address the time sensitive situation was initiated within the selected time period on the interaction. (Para. 118 see "Mismatchings, deltas, or deviations/divergences, between medical setting/scene features and respective reference records... may be registered. Mismatchings or deltas may for example include... procedure step actions not performed or performed differently from their respective description in the procedure records." Para. 241 discloses time sensitive situations where a notification is provided when an action has not occurred.).
Regarding claim 48, Talmor teaches The non-transitory computer readable medium of claim 41. wherein the operations further comprising: analyzing the second surgical footage to detect one or more surgical actions; (Para. 110 see "the procedure setting features recognition module may include video signature detection functionality for correlating detected images—for example, utilizing optical face/movement recognition and/or optical item identification technologies/methodologies implementing logic(s)/circuits—with specific actions, specific medical equipment usage, specific medical tools usage and/or specific chemicals/medicines/drugs usage."). and basing the determination that no action to address the time sensitive situation was initiated within the selected time period on the detected one or more surgical actions. (Para. 118 see "Mismatchings, deltas, or deviations/divergences, between medical setting/scene features and respective reference records of the identified procedure/steps, and/or differences/divergences there between, may be registered. Mismatchings or deltas may for example include: differences in the positions and/or orientations of medical staff members and medical equipment or devices: procedure step actions not performed or performed differently from their respective description in the procedure records.").
Regarding claim 53, Talmor teaches The non-transitory computer readable medium of claim 41. wherein the operations further comprising: analyzing the second surgical footage to identify a second time sensitive situation; (Para. 241 see "system provided resuscitation relevant recommendations/indications/notifications, given to medical staff member(s), include... ‘It is time for drug therapy, please give patient 1 milligram of epinephrine’... ‘You have provided the drug therapy, but have not flushed’." (Examiner note: the system instructs giving 1mg of epinephrine during the first time sensitive situation period. The system later identifies the dosage was given but must flush as the second time sensitive situation.)). and in response to the identification of the second time sensitive situation, updating the selection of the time period. (Para. 142 see "System notifications, alerts, messages and/or assistive tools, may be provided substantially in real time and/or may be provided at time points set by the system based on the flow and logic of the medical procedure being carried-out and/or sub steps thereof." Para. 241 discloses monitoring time sensitive situations. (Examiner note: the system would provide a time point based on the flow and logic of the second time sensitive situation.)).
Regarding claim 55, Talmor teaches The non-transitory computer readable medium of claim 41. wherein the operations further comprising: receiving third surgical footage captured using the at least one image sensor from the ongoing surgical procedure after the information indicative of the need to address the time sensitive situation is provided; (Para. 75 see "system for tracking, evaluating and facilitating medical procedures, in accordance with some embodiments; the shown system includes: A microphone/sound-recording unit, a camera/imaging unit and a medical device/s and/or medical sensor/s interface unit—adapted for monitoring, optionally in parallel/concert, the medical procedure being carried out and its environment and for collecting information about it." Para. 236 see "Data is continuously/intermittently received or retrieved from interfaced medical devices, the patient's medical records and additional inputs from the environment and from functionally associated devices and systems."). analyzing the third surgical footage to determine that no action to address the time sensitive situation was initiated within a second time period; (Para. 118 see "Mismatchings, deltas, or deviations/divergences, between medical setting/scene features and respective reference records of the identified procedure/steps, and/or differences/divergences there between, may be registered. Mismatchings or deltas may for example include... commands or requests by medical staff members and medical equipment or devices, not executed; medical staff members and medical equipment or devices not present at the procedure setting; auxiliary medical support and decision support tools and resources not present, not being used, or being used not in accordance with the procedure records; medicines/drugs given to the patient at times and/or dosages mismatching the procedure records; measured physiological parameters deviating/diverging from those that are typical for the procedure; and/or other." Para. 142 see "System notifications, alerts, messages and/or assistive tools, may be provided substantially in real time and/or may be provided at time points set by the system based on the flow and logic of the medical procedure being carried-out and/or sub steps thereof." Para. 241 discloses time sensitive situations where a notification is provided when an action has not occurred.). and in response to the determination that no action to address the time sensitive situation was initiated within the second time period, providing second information indicative of the need to address the time sensitive situation, (Para. 241 see "system provided resuscitation relevant recommendations/indications/notifications, given to medical staff member(s), include... ‘It is time for drug therapy, please give patient 1 milligram of epinephrine’... ‘You have provided the drug therapy, but have not flushed’." (Examiner note: the system instructs giving 1mg of epinephrine during a time sensitive situation period. The system later identifies the dosage was given but must flush. Para. 75 and 236 disclose a continuous process. The system would repeatedly identify that no action has occurred and would continue to provide new information indicating a need to address the time sensitive situation.)). the second information is of greater intensity than the first information. (Para. 241 see "You have initiated the resuscitation procedure 2 minutes ago, and an electric shock has NOT yet been provided—please give a shock ASAP." Para. 212 see "the Indication Components may take the form of: a light source… a sound source… a vibrating unit... An item indication, in accordance with some embodiments, may take the form of a constant signal (e.g. continuous light), a breaking signal (e.g. a blinking light), a strengthening or weakening signal (e.g. a light getting brighter and brighter), a changing signal (e.g. a light gradually changing the color of its emitted light) and/or any combination of the above." (Examiner note: A light getting brighter and brighter indicates greater intensity. All capitals like "ASAP" indicate greater intensity.)).
Regarding claim 56, Talmor teaches The non-transitory computer readable medium of claim 41. wherein the operations further comprising: receiving third surgical footage captured using the at least one image sensor from the ongoing surgical procedure after the information indicative of the need to address the time sensitive situation is provided; (Para. 75 see "system for tracking, evaluating and facilitating medical procedures, in accordance with some embodiments; the shown system includes: A microphone/sound-recording unit, a camera/imaging unit and a medical device/s and/or medical sensor/s interface unit—adapted for monitoring, optionally in parallel/concert, the medical procedure being carried out and its environment and for collecting information about it." Para. 236 see "Data is continuously/intermittently received or retrieved from interfaced medical devices, the patient's medical records and additional inputs from the environment and from functionally associated devices and systems."). analyzing the third surgical footage to determine that an action to address the time sensitive situation was initiated within a second time period; determining that the initiated action is insufficient to successfully address the time sensitive situation; and in response to the determination that the initiated action is insufficient to successfully address the time sensitive situation, providing second information. (Para. 241 see "system provided resuscitation relevant recommendations/indications/notifications, given to medical staff member(s), include... ‘It is time for drug therapy, please give patient 1 milligram of epinephrine’; ‘The staff member command given was referred to a wrong dosage, please give 1 milligram of epinephrine’." (Examiner note: the system instructs giving 1mg of epinephrine during the first time sensitive situation period. The system later identifies the wrong dosage was giving during the second time sensitive situation period and provides this as second information.)).
Regarding claim 57, Talmor teaches The non-transitory computer readable medium of claim 41. wherein the information is provided to a surgeon performing the ongoing surgical procedure. (Para. 78 see "The exemplary processing and analysis of the data further includes the generation/issuance of decision and knowledge support suggestions/notifications, to the medical staff carrying out the current medical procedure." Para. 214 see "multiple Indication Components of the storage device may simultaneously indicate multiple stored items... items for a surgeon, items for an anesthesiologist, items for a nurse.").
Regarding claim 58, Talmor teaches The non-transitory computer readable medium of claim 41. wherein the information is provided to a person outside an operating room, the person is associated with the ongoing surgical procedure. (Para. 139 see "The Notification and Recommendation Module may trigger the issuance of informative messages to the medical/paramedical staff participating in the medical procedure and/or to other persons (e.g. non-present experts).").
Claim 59 is rejected under the same analysis as claim 41 above.
Claim 60 is rejected under the same analysis as claim 41 above.
Allowable Subject Matter
Claim(s) 49-52, 54 is/are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Karkanias et al. (US 20080140444 A1) discloses a system that can enable clinical trial compliance determination by viewing sequences of images captured during an event.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALEXANDER J VAUGHN whose telephone number is (571) 272-5253. The examiner can normally be reached M-F 8:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ANDREW MOYER can be reached on (571) 272-9523. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ALEXANDER JOSEPH VAUGHN/Examiner, Art Unit 2675
/EDWARD PARK/Primary Examiner, Art Unit 2675