Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
In the amendment filed 03/05/2026, the following has occurred: claims 1, 9, 11, 13-14, 18, and 20 have been amended and claim 8 has been canceled. Now, claims 1-7 and 9-20 are pending.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-8 and 9-20 rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 2A Prong One
Claims 1 and 20 (claim 1 representative) recite receiving one or more images depicting a medical environment and a medical workflow performed in the medical environment, the medical workflow comprising a plurality of predefined stages; determining contextual information associated with the medical workflow based on the one or more images; identifying which of the plurality of predefined stages of the medical workflow is the current stage of the medical workflow based on the one or more images; determining a subsequent stage of the medical workflow based on the identified current stage, the subsequent stage being a predefined stage of the plurality of predefined stages that is to occur after the identified current stage; and presenting, to medical staff located within the medical environment during the identified current stage, first medical content associated with the contextual information and second medical content associated with the subsequent stage, the second medical content associated with an activity to be performed during the subsequent stage, wherein at least some of the first medical content, at least some of the second medical content, or at least some of both is presented to a first subset of the medical staff and at least some of the first medical content, at least some of the second medical content, or at least some of both is presented to a second subset of the medical staff such that different content is presented to the first subset of the medical staff than the second subset of medical staff depending on different roles and/or activities of the medical staff.
These limitations, as drafted, given the broadest reasonable interpretation, encompass managing interactions between people, which is a subgrouping of Certain Methods of Organizing Human Activity. For example, the claims encompass a user looking at a picture of a medical environment, such as a surgical room where users are performing operations, determining context in the environment, such as an operation being performed, identifying a stage in the environment, such as the beginning of an operation, determining a next stage such as the middle of the operation, and providing the users in the environment information about the beginning and middle of the operation. Additionally, different information could be provided to different types of individuals, such as providing different information to doctors and nurses. As explained, the claim involves an “image,” but this is part of the abstract idea because it encompasses a user looking at a picture and generating information based on this observation. Such manual steps encompass Certain Methods of Organizing Human Activity.
Claims 2-7 and 9-19 incorporate the abstract idea identified above and recite additional limitations that expand on the abstract idea. For example, claims 2, 4, 7, and 15 further expand on the identification of stages, which is part of the abstract idea as identified above. Claims 5 and 19 further expand on the contextual information, which is part of the abstract idea as identified above. Claims 6 and 11-13 further recite generating a report. This would further expand on Certain Methods of Organizing Human Activity, identified above. Claims 8-10, 14, and 16 further expand on the providing medical content, which is part of the abstract idea as identified above. Claims 17-18 further expand on identifying the tasks, which is part of the abstract idea as identified above.
Step 2A Prong Two
This judicial exception is not integrated into a practical application because the remaining elements amount to no more than general purpose computer components programmed to perform the abstract ideas along with adding elements similar to adding the words “apply it” to the abstract idea, and generally linking the abstract idea to a particular technological environment, along with insignificant, extra-solution data gathering activity.
Claims 1-7 and 9-20, directly or indirectly, recite the following additional elements at a high level of generality and merely utilized as tools to implement the abstract idea:
Claims 1, 6, 8, 9, 14, and 20:
presenting, selecting, and sending with one or more display devices.
Claim 20:
one or more processors programmed to perform a method.
The written description discloses that the recited computer components encompass generic components including “a general-purpose computer selectively activated or reconfigured by a computer program stored in the computer” (see paragraph 0062). As set forth in the MPEP 2106.04(d) “merely including instructions to implement an abstract idea on a computer” is an example of when an abstract idea has not been integrated into a practical application.
Claim 3 recites the following additional elements at a high level of generality, involving no more than extra-solution data gathering activity:
inputting the one or more images into one or more machine learning models to obtain one or more image representations.
These additional elements are recited at a high degree of generality and are merely involved in insignificant extra solution data gathering of inputting and obtaining. As set forth in MPEP 2106.05(g) insignificant, extra-solution activity, such as insignificant acquisition and data transmission, is an example of when an abstract idea has not been integrated into a practical application.
Claims 10-11 and 14 recite the following additional elements at a high level of generality, similar to adding the words “apply it” to the abstract idea:
detecting using one or more machine learning models.
The one or more machine learning models element is broadly recited and there is no indication that the combination of the model(s) and the other recited limitations provides any type of technical improvement or an improvement to another technical field.
Step 2B
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because as discussed above with respect to integration into a practical application, the additional elements are recited at a high level of generality, and the written description indicates that these elements are generic computer components. Using generic computer components to perform abstract ideas does not provide a necessary inventive concept. See Alice, 573 U.S. at 223 (“mere recitation of a generic computer cannot transform a patent-ineligible abstract idea into a patent-eligible invention.”).
Insignificant, extra solution, data gathering activity (e.g. inputting and obtaining data) has been found to not amount to significantly more than an abstract idea (see MPEP 2106.05(g) and Electric Power Group, LLC v. Alstom S.A., 830 F.3d 1350, 1354-55, 119 USPQ2d 1739, 1742 (Fed. Cir. 2016)).
Additionally, the aforementioned additional elements, considered in combination, do not provide an improvement to a technical field or provide a technical improvement to a technical problem. Therefore, whether considered alone or in combination, the additional elements do not amount to significantly more than the abstract idea.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-2, 6-7, 9, and 15-20 is/are rejected under 35 U.S.C. 102(a)(1) and (a)(2) as being anticipated by White, US Patent Application Publication No. 2018/0082480.
As per claim 1, White teaches a method for evaluating a medical workflow, comprising: receiving one or more images depicting a medical environment and a medical workflow performed in the medical environment (see paragraphs 0022 and 0025; cameras capture images of the environment where a procedure will take place or is taking place), the medical workflow comprising a plurality of predefined stages (see paragraph 0055; surgical procedure includes a plurality of surgical steps (stages)); determining contextual information associated with the medical workflow based on the one or more images (see paragraphs 0072 and 0074; captured images, sent to the server, are used to present information (contextual information) for steps of the surgical procedure); identifying which of the plurality of predefined stages of the medical workflow is the current stage of the medical workflow based on the one or more images (see paragraph 0070; the database is accessed for ordered steps for the surgical procedure for creating representations of the actions associated with each step; paragraph 0074; based on images from the camera, the system determines the current surgical step completion and the subsequent surgical step for presentation); determining a subsequent stage of the medical workflow based on the identified current stage, the subsequent stage being a predefined stage of the plurality of predefined stages that is to occur after the identified current stage (see paragraph 0074; based on images from the camera, the system determines the current surgical stage completion and the subsequent surgical stage for presentation); and presenting, to medical staff located within the medical environment during the identified current stage, first medical content associated with the contextual information and second medical content associated with the subsequent stage, the second medical content associated with an activity to be performed during the subsequent stage (see paragraph 0074; based on images from the camera, the system determines the current surgical stage completion and the subsequent surgical stage for presentation to the members involved in the surgery), wherein at least some of the first medical content, at least some of the second medical content, or at least some of both is presented via a first display device associated with a first subset of the medical staff and at least some of the first medical content, at least some of the second medical content, or at least some of both is presented via a second display device associated with a second subset of the medical staff such that different content is presented to the first subset of medical staff than the second subset of medical staff depending on different roles and/or activities of the medical staff (see paragraph 0020; the AR devices may use different roles (e.g. surgeon, surgical assistant, nurse, etc.) for displaying different information that is relevant to the different roles; paragraph 0027 describes an example of displaying a virtual object relevant to performing nurse type duties and displaying a virtual object relevant to performing surgical duties).
As per claim 2, White teaches the method of claim 1 as described above. White further teaches the medical workflow comprises a first stage occurring prior to a beginning of a medical procedure, a second stage occurring during the medical procedure, and a third stage occurring after the medical procedure has been completed (see paragraph 0044; before the procedure begins, a first stage may occur to identify components used during the procedure; paragraph 0074; examples of stages occurring during the medical procedure; paragraph 0049; a stage occurring after the procedure is complete may be displayed until the items used in the procedure are removed), wherein identifying the stage of the medical workflow comprises: determining whether the one or more images were captured during the first stage, the second stage, or the third stage (see paragraph 0074; based on images from the camera, the system determines the current surgical stage completion and the subsequent surgical stage for presentation).
As per claim 6, White teaches the method of claim 1 as described above. White further teaches generating a report comprising at least some of the first medical content, the second medical content, or the first medical content and the second medical content (see paragraph 0074; the information displayed to the medical staff being a report, representing information on stages of the procedure); and presenting, via one or more display devices, the report to the one or more medical staff (see paragraph 0074; the information displayed to the medical staff being a report, representing information on stages of the procedure).
As per claim 7, White teaches the method of claim 1 as described above. White further teaches generating an audio message based on subsequent stage information, the audio message describing the subsequent stage of the medical workflow (see paragraph 0057; provides an audio indication that a surgical instrument is needed in a later surgical step).
As per claim 9, White teaches the method of claim 1 as described above. White further teaches selecting the first display device, wherein selecting the first display device comprises: identifying, based on the one or more images, one or more relevant medical staff of the first subset of medical staff associated with the subsequent stage (see paragraph 0020; the system controls AR devices associated with the different operators to display the relevant information for each stage to each operator); and identifying, based on the one or more images, the first display device based on proximity to the one or more relevant medical staff (see paragraph 0037; AR device of the operator is identified based its location within the environment (changes or updates to a location of the AR device) of which the operator is a part).
As per claim 15, White teaches the method of claim 1 as described above. White further teaches determining, based on the contextual information, that the medical workflow has progressed from a first stage to a second stage (see paragraph 0055; determines that the surgical step has been completed and proceeds to a second surgical step); and updating the first medical content based on the medical workflow progressing from the first stage to the second stage (see paragraph 0055; presenting on the display a representation for a second surgical step).
As per claim 16, White teaches the method of claim 1 as described above. White further teaches presenting the first medical content and the second medical content comprises: identifying at least one of (i) one or more checklists associated with the medical workflow or (ii) one or more videos associated with the medical workflow (see paragraph 0087; presenting video of the surgical step); and generating at least one of the first medical content or the second medical content based on the at least one of (i) the one or more checklists or (ii) the one or more videos (see paragraph 0087; presenting video of the surgical step).
As per claim 17, White teaches the method of claim 1 as described above. White further teaches identifying, based on the one or more images, a current task of a plurality of tasks associated with the identified stage (see paragraph 0074; the information displayed to the medical staff being a report, representing information on stages of the procedure); determining, based on the one or more images, that the current task has been completed; and modifying a checklist associated with the identified stage to indicate that the current task has been completed (see paragraph 0055; determines that the surgical step has been completed and proceeds to a second surgical step).
As per claim 18, White teaches the method of claim 1 as described above. White further teaches identifying, from a plurality of tasks associated with the identified current stage, a current task and a subsequent task (see paragraph 0074; the information displayed to the medical staff being a report, representing information on stages of the procedure); determining, based on the one or more images, that the subsequent task has been started by the medical staff prior to completion of the current task (see paragraph 0090; determines that a different surgical step is being performed before the surgical step has been completed); and sending a notification to the medical staff indicating that the current task has not been completed (see paragraph 0090;sends an alert indicating the different surgical step is being performed).
As per claim 19, White teaches the method of claim 1 as described above. White further teaches the contextual information comprises information associated with cleaning of the medical environment (see paragraph 0039; cleaning items).
Claim 20 recites substantially similar system limitations to method claim 1 and, as such, is rejected for similar reasons as given above.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 3-5, 10-12, and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over White, US Patent Application Publication No. 2018/0082480 in view of Thornton, US Patent Application Publication No. 2020/0350063.
As per claim 3, White and Thornton teaches the method of claim 1 as described above. White does not explicitly teach inputting the one or more images into one or more machine learning models to obtain one or more image representations. Thornton teaches inputting the one or more images into one or more machine learning models to obtain one or more image representations (see paragraph 0058; camera feed is continuously processed by a trained machine learning model). It would have been obvious to one of ordinary skill in the art at the time of the effective filing date to implement the machine learning of Thornton for processing captured images of White with the motivation of quickly and efficiently providing context for the medical treatment environment of White (see paragraph 0015 of Thornton).
As per claim 4, White and Thornton teaches the method of claim 3 as described above. White does not explicitly teach identifying the stage of the medical workflow comprises: identifying activities of the one or more medical staff based on the one or more image representations, the stage of the medical workflow being determined based on the identified activities. Thornton further teaches identifying the stage of the medical workflow comprises: identifying activities of the one or more medical staff based on the one or more image representations, the stage of the medical workflow being determined based on the identified activities (see paragraph 0015; activities of medical staff can be indicated by presence of doctor at treatment location, presence of cleaning staff at treatment location, among many others). It would have been obvious to one of ordinary skill in the art at the time of the effective filing date to implement the machine learning of Thornton for the reasons given above with respect to claim 3.
As per claim 5, White and Thornton teaches the method of claim 4 as described above. White further teaches determining the contextual information comprises: detecting, based on the one or more image representations, one or more objects present within the one or more images (see paragraph 0072; camera captures surgical instrument used in a next step of the operation); and generating the contextual information based on the one or more detected objects (see paragraph 0072; captured instrument used for providing information on the next step).
As per claim 10, White the method of claim 9 as described above. White does not explicitly teach identifying the one or more relevant medical staff comprises: detecting, using one or more machine learning models, activities of the one or more relevant medical staff based on the one or more images, the one or more relevant medical staff being identified from the one or more medical staff based on the detected activities. Thornton further teaches identifying the one or more relevant medical staff comprises: detecting, using one or more machine learning models, activities of the one or more relevant medical staff based on the one or more images, the one or more relevant medical staff being identified from the one or more medical staff based on the detected activities (see paragraphs 0015 and 0058; after patient arrives, subsequent stages are detected and associated with a timer throughout treatment, which may include doctor presences, cleaning staff presence, etc.). It would have been obvious to one of ordinary skill in the art at the time of the effective filing date to implement the machine learning of Thornton for the reasons given above with respect to claim 3.
As per claim 11, White teaches the method of claim 1 as described above. White does not explicitly teach detecting, using one or more machine learning models, activities of the one or more medical staff during the medical workflow based on the one or more images; and generating a report for the medical workflow based on the detected activities. Thornton further teaches detecting, using one or more machine learning models, activities of the one or more medical staff during the stage of the medical workflow based on the one or more images (see paragraphs 0015 and 0058; after patient arrives, subsequent stages are detected and associated with a timer throughout treatment, which may include doctor presences, cleaning staff presence, etc.); and generating a report for the medical workflow based on the detected activities (see paragraph 0019; context assessment for display to user can include any combination of the medical content). It would have been obvious to one of ordinary skill in the art at the time of the effective filing date to implement the machine learning of Thornton for the reasons given above with respect to claim 3.
As per claim 12, White and Thornton teaches the method of claim 11 as described above. White does not explicitly teach generating the report comprises: generating an efficiency score indicating an efficiency of the medical workflow, the report comprising the generated efficiency score. Thornton further teaches generating the report comprises: generating an efficiency score indicating an efficiency of the medical workflow, the report comprising the generated efficiency score (see paragraph 0020; output of model can include a computed efficiency of medical treatment location and efficiency of medical staff). It would have been obvious to one of ordinary skill in the art at the time of the effective filing date to calculate efficiency within the surgical environment or White with the motivation of improving efficiency of an operating room through measuring data metrics during the operation (see paragraph 0013 of Thornton).
As per claim 14, White teaches the method of claim 1 as described above. White does not explicitly teach detecting, using one or more machine learning models, activities of the one or more medical staff during the stage of the medical workflow based on the one or more images; updating at least one of the first medical content or the second medical content based on the activities; and presenting, using one or more display devices, at least one of the updated first medical content or the updated second medical content to the one or more medical staff. Thornton further teaches detecting, using one or more machine learning models, activities of the one or more medical staff during the stage of the medical workflow based on the one or more images (see paragraph 0015; activities of medical staff can be indicated by presence of doctor at treatment location, presence of cleaning staff at treatment location, among many others); updating at least one of the first medical content or the second medical content based on the activities (see paragraphs 0063-0064; context assessment is generated and updated); and presenting, using one or more display devices, at least one of the updated first medical content or the updated second medical content to the one or more medical staff (see paragraph 0064; updated context assessment is displayed in a treatment region corresponding to where the images are taken). It would have been obvious to one of ordinary skill in the art at the time of the effective filing date to implement the machine learning of Thornton for the reasons given above with respect to claim 3.
Claim(s) 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over White, US Patent Application Publication No. 2018/0082480 in view of Thornton, US Patent Application Publication No. 2020/0350063 and further in view of Amanatullah, US Patent Application Publication No. 2020/0253683.
As per claim 13 White and Thornton teaches the method of claim 12 as described above. White does not explicitly teach generating the efficiency score comprises: determining a number of medical staff associated with the identified current stage of the medical workflow based on the detected activities of the one or more medical staff; and comparing the number of medical staff to a predefined number of medical staff to be used for the identified current stage of the medical workflow, the efficiency score being based at least in part on a result of the comparison. Thornton further teaches generating the efficiency score comprises: determining a number of medical staff associated with the stage of the medical workflow based on the detected activities of the one or more medical staff (see paragraph 0020; determines whether doctor/nurse/cleaning staff has arrived/departed). It would have been obvious to one of ordinary skill in the art at the time of the effective filing date to calculate efficiency within the surgical environment or White for the reasons given above with respect to claim 12. White and Thornton does not explicitly teach comparing the number of medical staff to a predefined number of medical staff to be used for the stage of the medical workflow, the efficiency score being based at least in part on a result of the comparison. Amanatullah teaches comparing a number of medical staff to a predefined number of medical staff to be used for a medical workflow, an efficiency score being based at least in part on a result of the comparison (see paragraph 0121; efficiency of a medical staff can be calculated for a procedure based on to historic averages for similar procedures and medical staff). It would have been obvious to one of ordinary skill in the art at the time of the effective filing date to add this efficiency score to the efficiency scoring of White and Thornton with the motivation of improving benchmark metrics for cases, surgeons, hospitals, and hospital systems in the use of Thornton (see paragraph 0120 of Amanatullah).
Response to Arguments
In the remarks filed 03/05/2026, Applicant argues (1) the claims integrate the abstract idea into a practical application by providing technical improvements to computer-assisted surgery through data being presented to medical staff at medical devices located near them; (2) the claims integrate the abstract idea into a practical application by providing technical improvements to the operation of the computer by providing information tailored to the roll of the respective medical staff; (3) the claims are similar to published example 37; (4) the claims as amended distinguish over the teachings of Thornton.
In response to argument (1), to show that the involvement of a computer assists in improving the technology, the claims must recite the details regarding how a computer aids the method, the extent to which the computer aids the method (MPEP 2106305(a) II). Claim 1 recites first and/or second medical content “is presented via a first display device associated with a first subset of the medical staff” and first and/or second medical content “is presented via a second display device associated with a second subset of the medical staff” such that different content is presented to first staff and second staff based on different rolls and/or activities. Therefore, the different content is provided to different medical staff, but the particular device used by each medical staff is unrelated to selecting or tailoring the presented content. As explained in the updated rejections, providing different medical content to different types of medical staff (such as certain information being provided to a doctor and other information being provided to a nurse) encompasses the management of interactions between people in a medical environment. Because claim 1 does not recite the argued technical features related to content being tailored to different types of devices and based relative location to certain medical staff, this argument is not persuasive.
In response to argument (2), as explained above, providing different information that is relevant to respective medical staff members is part of the identified abstract idea. Similar to above, providing different relevant information to a nurse and a doctor encompasses the management of interactions between people in a medical environment. Therefore, this argument is not persuasive.
In response to argument (3), the examiner respectfully disagrees that providing different information to different individuals is analogous to a particular manner of arranging icons on a display based specifically on how they are used on the computer. As explained above, the content provided to the users, as recited in the independent claims, is unrelated to any technical aspects of any of the computer elements. Rather, it is based on their respective roles in a medical environment. Because of these differences, this argument is not persuasive.
Applicant’s argument (4) has been fully considered but is moot in view of the new grounds of rejection set forth above.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to C. Luke Gilligan whose telephone number is (571)272-6770. The examiner can normally be reached Monday through Friday 9:00 - 5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Robert Morgan can be reached at 571-272-6773. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHRISTOPHER L GILLIGAN/ Primary Examiner, Art Unit 3683
Prosecution Insights