DETAILED ACTION
Status
This Final Office Action is in response to the communication filed on 20 January 2026. No claims have been cancelled, claim1-4 and 6-19 have been amended, and no claims have been added. Therefore, claims 1-20 are pending and presented for examination.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
A summary of the Examiner’s Response to Applicant’s amendment:
Applicant’s amendment does not overcome the rejection(s) under 35 USC § 101; therefore, the Examiner maintains the rejection(s) while updating phrasing in keeping with current examination guidelines.
Applicant’s amendment overcomes the rejection(s) under 35 USC §§ 102 and/or 103; therefore, the Examiner indicates allowability over the prior art.
Applicant’s arguments are found to be not persuasive; please see the Response to Arguments below.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Please see the following Subject Matter Eligibility (“SME”) analysis:
For analysis under SME Step 1, the claims herein are directed to a system (claims 1-10), method (claims 11-16), and non-transitory computer-readable media (claims 17-20), which would be classified under one of the listed statutory classifications (SME Step 1=Yes).
For analysis under revised SME Step 2A, Prong 1, independent claim 1 recites a system, comprising: a processor; a non-transitory, computer-readable media having instructions stored thereon that, when executed by the processor, cause the processor to perform acts comprising: receiving video data from a camera operably connected to the processor, the video data being associated with a patient disposed in a room; determining, using a first trained machine learning model and based at least in part on the video data, a first occurrence of a first event within the room and associated with the patient; generating, using the first trained machine learning model, a first annotation indicative of the first event, the first annotation comprising a description of the first event and associated patient data; determining, based at least in part on the video data and the first annotation and using the first trained machine learning model, a confidence score associated with the first event; updating, by the first trained machine learning model and based at least in part on the first event and the first annotation, an electronic medical record of the patient; generating, using the first trained machine learning model and based at least in part on the updating, an updated electronic medical record of the patient, wherein the updated electronic medical record includes the first occurrence of the first event and the first annotation; receiving, from a caregiver or an input system: second occurrence of a second event based at least in part on the video data, a second annotation based at least in part on the second occurrence of the second event, an accuracy of the first annotation based at least in part on the first annotation and the second annotation, and a manually updated electronic medical record, wherein the manually updated electronic medical record includes the second annotation; generating, based at least in part on the accuracy of the first annotation, training data; and generating, based at least in part on the training data, a second trained machine learning model, wherein the second trained machine learning model: determines a third occurrence of a third event similar to the second occurrence of the second event based on the video data, and generates a third annotation similar to the second annotation.
Independent claims 11 and 17 are analyzed similar to claim 1 above since claim 11 is directed to a method comprising the same or similar activities, except the method is merely performed “at a computing device”, and claim 17 is directed to “[o]ne or more non-transitory computer-readable media having instructions stored thereon that, when executed by one or more processors, cause the one or more processors to perform operations comprising: receiving, at a computing device … video data from a camera”.
The dependent claims (claims 2-10, 12-16, and 18-20) appear to be encompassed by the abstract idea of the independent claims since they merely indicate what data is used to train the model (claim 2), the event as identifying a person (claims 3 and 14) based on a unique identifier (claim 15), tracking the identified person and annotating based on their interaction with the patient or equipment (claim 16), the annotation being based on temperature, blood pressure or oxygenation, heart rate, or movement (claim 4), determining a display is presenting patient data and determining the patient data (claims 5 and 20), the update being based on determining a prescribed procedure and a compliance score thereof (claims 6 and 18), determining a second event, annotation, and update based on the use of a second camera in a second room (claim 7), a caregiver station displaying the first and second video data, event, and annotation(s) (claims 8 and 12), receiving input and updating based on the input (claim 9), determining an event compliance score and when below a threshold, presenting a request for input (claims 10 and 13), and/or receiving patient vital data and annotating based on that data (claim 19).
The underlined portions of the claims are an indication of elements additional to the abstract idea (to be considered below).
The claim elements may be summarized as the idea of updating a patient medical record based on patient room event occurrence(s) and manually updating a medical record with annotation(s); however, the Examiner notes that although this summary of the claims is provided, the analysis regarding subject matter eligibility considers the entirety of the claim elements, both individually and as a whole (or ordered combination). This idea is within the Certain methods of organizing human activity (e.g. … commercial or legal interactions such as … legal obligations, … or business relations; and/or managing personal behavior or relationships between people such as social activities, teaching, and following rules or instructions) grouping(s) of subject matter.
To the extent that generating training data and training a machine learning model based on the manually entered received data is recited, this also appears to be encompassed by the abstract idea since the training data is apparently – it is recited as – the manually (i.e., human) entered data used to improve, retrain, or generate another model. Humans have long interacted with each other by refining the training of other persons so as to improve accuracy.
Applicant’s specification ¶¶ 0003-0005 indicate that “In patient care environments … caregivers employ a variety of medical devices … that interact with patient monitoring devices which display a significant amount of patient health information. Such information is typically displayed on handheld monitoring devices or stationary monitoring devices with limited visual ‘real estate’”. Multiple patients are monitored, information constantly fluctuating, and it is therefore difficult to locate, evaluate, and respond to and difficult to chart the patient data since there are only a small number of caregivers to enter patient data into electronic medical records (EMRs)…. It is also indicated that it can be difficult to determine if the patient's care plan or EMR needs to be updated. (at 0003) So, “[t]he example embodiments of the present disclosure are directed toward overcoming the deficiencies described above. (at 0004) Therefore, “the systems and techniques described herein provide a system including a camera …, one or more processors, and one or more non-transitory, computer-readable media having instructions stored thereon that, when executed by the one or more processors, cause the one or more processors to performs one or more acts. The acts may include receiving video data from the camera of the room including a representation of a patient and determining, using a machine learning model trained using training data including annotated video data of patient care facilities with annotations of event data, an event occurring within the room. The acts may also include generating, using the machine learning model, an annotation associated with the event, and updating, in response to the event and based on the annotation, an electronic medical record of the patient.” (at 0005). This specifically indicates that the claimed activities are what humans, such as caregivers, could do and/or are doing or expected to do, except for the use of the indicated cameras, computers, and modeling. Further, the claims specifically indicate receiving manual inputs for the second occurrence event and annotation as well as an accuracy indication.
The Examiner notes that although mental processes (e.g., concepts performed in the human mind such as observation, evaluation, judgment, and/or opinion) are implicated as an additional grouping based on the monitoring and observation of events, where the updating of a medical record can be done via the use of pen or pencil and paper.
Additionally, mathematical concepts (e.g., relationships, formulas, equations, and/or calculations) are also implicated as an additional grouping based on the use of machine learning to model event occurrence and determining a confidence score.
Therefore, the claims are found to be directed to an abstract idea.
For analysis under revised SME Step 2A, Prong 2, the above judicial exception is not integrated into a practical application because the additional elements do not impose a meaningful limit on the judicial exception when evaluated individually and as a combination. The additional elements are using a system, comprising: a processor; a non-transitory, computer-readable media having instructions stored thereon that, when executed by the processor, cause the processor to perform acts, a/the camera being operably connected to the processor, using an electronic patient medical record (at claim 1), using a computing device (at claim 11), and one or more non-transitory computer-readable media having instructions stored thereon that, when executed by one or more processors, cause the one or more processors to perform operations comprising at least receiving at a computing device video data from a camera (at claim 17).
These additional elements do not reflect an improvement in the functioning of a computer or an improvement to other technology or technical field, effect a particular treatment or prophylaxis for a disease or medical condition (there is no medical disease or condition, much less a treatment or prophylaxis for one), implement the judicial exception with, or by using in conjunction with, a particular machine or manufacture that is integral to the claim, effect a transformation or reduction of a particular article to a different state or thing (there is no transformation/reduction of a physical article), and/or apply or use the judicial exception in some other meaningful way beyond generically linking use of the judicial exception to a particular technological environment.
The claims appear to merely apply the judicial exception, include instructions to implement an abstract idea on a computer, or merely use a computer as a tool to perform the abstract idea. The additional elements appear to merely add insignificant extra-solution activity to the judicial exception and/or generally link the use of the judicial exception to a particular technological environment or field of use.
For analysis under SME Step 2B, the claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements, as indicated above, are merely “[a]dding the words ‘apply it’ (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, e.g., a limitation indicating that a particular function such as creating and maintaining electronic records is performed by a computer, as discussed in Alice Corp.” that MPEP § 2106.05(I)(A) indicates to be insignificant activity.
There is no indication the Examiner can find in the record regarding any specialized computer hardware or other “inventive” components, but rather, the claims merely indicate computer components which appear to be generic components and therefore do not satisfy an inventive concept that would constitute “significantly more” with respect to eligibility. Applicant ¶ 00110 (as submitted, 0110 as published) indicates that the computer(s) used to implement the invention are envisioned to include generic or general-purpose computers such as “a laptop computer, desktop computer, tablet, server, or other electronic device.”
The individual elements therefore do not appear to offer any significance beyond the application of the abstract idea itself, and there does not appear to be any additional benefit or significance indicated by the ordered combination, i.e., there does not appear to be any synergy or special import to the claim as a whole other than the application of the idea itself.
The dependent claims, as indicated above, appear encompassed by the abstract idea since they merely limit the idea itself; therefore the dependent claims do not add significantly more than the idea.
Therefore, SME Step 2B=No, any additional elements, whether taken individually or as an ordered whole in combination, do not amount to significantly more than the abstract idea, including analysis of the dependent claims.
Please see the Subject Matter Eligibility (SME) guidance and instruction materials at https://www.uspto.gov/patent/laws-and-regulations/examination-policy/subject-matter-eligibility, which includes the latest guidance, memoranda, and update(s) for further information.
Allowable Subject Matter
Claims 1-20 are indicated as allowable over the prior art of record.
The following is a statement of reasons for the indication of allowable subject matter:
Amarasingham et al. (U.S. Patent Application Publication No. 2015/0213202) appears to be the closest prior art, indicating the processor and media as receiving patient room video and analyzing it through artificial intelligence (AI) and machine learning (ML), but does not appear explicit regarding entering annotations in an electronic medical record (EMR) using the AI/ML, but does offer an interface for manual entry of notes, and also that the model(s) used are retrained periodically (Amarasingham at 0037, 0045, 0064, 0067).
Leonard (U.S. Patent Application Publication No. 2016/0321415) teaches teaches a similar method, system, and non-transitory, tangible machine-readable storage medium using artificial intelligence for detecting interactions, extracting and analyzing streamed data so as to provide summary notes to an EMR.
Giataganas et al. (U.S. Patent Application Publication No. 2019/0279765) indicates using artificial intelligence and/or machine learning for operating room video (and audio) analysis to enter EMR notes (Giataganas at 0021, 0037, 0045, 0066).
Evans et al. (U.S. Patent Application Publication No. 20210398676) indicates reporting a confidence level of a machine learning algorithm with respect to image analysis (Evans at 0040, 0043, 0046).
Therefore, although the concepts of the claims appear discussed by the prior art of record, it does not appear reasonable to combine the number and various levels of art so as to arrive at the specific claimed invention.
Response to Arguments
Applicant's arguments filed 20 January 2026 have been fully considered but they are not persuasive.
Applicant first argues the 101 eligibility rejection (Remarks at 14-20), first alleging that “the claims cannot be fairly said to fall into one or more of the groupings enumerated in the 2019 Guidelines” (Id. at 15); however, although the 2019 Guidelines have been superseded by the MPEP, the current groupings are the same and the claims are using a machine learning (ML) model to provide annotations to an electronic medical record (EMR), then using manual entries (i.e., human activity) to provide “a manually updated electronic medical record” and annotations as well as an indication of accuracy of the ML entry, then using the manual entry to train another ML model for similar or more events. Where a human would or could otherwise perform the activities of the ML model, and the manual entries are explicitly human activities, the claims are directed to one of certain methods of organizing human activity.
The Examiner notes that Applicant further argues that “the claims also now recite the determination of a real-time condition status that changes based upon real-time changes to conditions, events, and occurrences, and the generation of alert data in a form that causes a processor of a receiving device to initiate an application and display a time-sensitive alert associated with the recited unsatisfied condition” (Id. at 16). However, there is no apparent indication of any “real-time” activity required at the claims, nor is there an indication of a “condition status”, nor is there any “alert” as far as the Examiner can determine, and there does not appear to be any “initiat[ing of] an application”, other then (apparently) the mere initiating of software to perform the computer activities as inherent to any computer-implemented activity. As such, Applicant’s argument appears irrelevant to the instant claims.
Applicant then argues that “the claims are integrated into a practical application” (Id. at 16-17) by “the use of machine learning” to perform activities and “generating training data to improve the trained machine learning model”, where “At least these features show that the claims are integrated into a practical application specific to medical treatment” (Id. at 16). However, as indicated above, these are considered part of the abstract idea – people have long modeled behavior and analysis so as to improve the results when learning or training for activities (such as job training, athletic training, school, etc.). Law school and CLE classes being readily understood examples of updating and refining the accuracy of learning and training as related to current precedent.
Applicant then argues Bascom and Berkheimer as related to well-understood, routine, conventional (“WURC”) activity (Remarks at 17-20), alleging that “the Office's rejection is perplexing because it purportedly cites to Applicant's specification to assert that multiple features of the independent claims are well-understood, routine, and conventional” (Id. at 19). However, the specification citations are specifically indicated as pointing to the apparent admission that the claims are directed to an abstract idea (“This specifically indicates that the claimed activities are what humans, such as caregivers, could do and/or are doing or expected to do, except for the use of the indicated cameras, computers, and modeling”). This DOES NOT allege WURC activity – it indicates the abstract idea. The rejection does not indicate particular activities as WURC, but rather the rejection indicates that elements additional to the abstract idea are considered insignificant as “apply it’ or similar.
Therefore, Applicant’s arguments are not considered to be persuasive.
Applicant then argues the prior art rejections (Remarks at 20-26); however, as indicated above, the amendments appear to overcome the prior art of record. Therefore, Applicant’s arguments are considered moot and not persuasive.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Kusuma et al. (U.S. Patent Application Publication No. 2020/0160985, hereinafter Kusuma) indicates “The patient-centric eco-system operates in healthcare facility's (HC-FAC) treatment/recovery/waiting rooms using audio/visual (presence) sensors and image sensors (serial static images or video). Sensors capture patient presence sensory data in rooms. Computing devices process data and obtain room-to-room patient unique transition data, patient unique treatment/recovery timing data, and treatment/recovery image data. System digitally tags and segments the treatment/recovery images (serial static images or video) to generate time-stamped patient unique treatment/recovery image data which is serial static images or video clip(s). Patient/provider replay command displays processed images or video clip. The display may be on a multimedia medical documentation presentation platform. To monitor and track patient flow through HC-FAC, a basic process flow user interface (UI) is a 3-column display (arrivals, treatment/recovery and check-out) tracking all patients at the HC-FAC with patient-display tiles manual/auto moved column-to-column as each patient transitions through HC-FAC.” (at Abstract).
Molinaro et al., Contactless Vital Signs Monitoring From Videos Recorded With Digital Cameras: An Overview, Front. Physiol., 17 February 2022, Sec. Physio-logging, Volume 13 – 2022, https://doi.org/10.3389/fphys.2022.801709, downloaded 15 September 2025 from https://www.frontiersin.org/journals/physiology/articles/10.3389/fphys.2022.801709/full, indicating capturing vital signs via video camera monitoring.
Lepakshi VA. Machine Learning and Deep Learning based AI Tools for Development of Diagnostic Tools. Computational Approaches for Novel Therapeutic and Diagnostic Designing to Mitigate SARS-CoV-2 Infection. 2022:399–420. doi: 10.1016/B978-0-323-91172-6.00011-X. Epub 2022 Jul 15. PMCID: PMC9300557. Downloaded 12 May 2026 from https://pmc.ncbi.nlm.nih.gov/articles/PMC9300557/, discussing artificial intelligence for medical diagnosis (at § 18.2, pp. 400-401), that recurrent neural networks are used for video analysis (at § 18.3.2, p. 406), methods to evaluate performance of classification models (at § 18.3.3, pp. 407-408).
Krenzer. et al., Fast machine learning annotation in the medical domain: a semi-automated video annotation tool for gastroenterologists. BioMed Eng OnLine 21, 33 (2022). https://doi.org/10.1186/s12938-022-01001-x, downloaded 12 May 2026 from https://link.springer.com/article/10.1186/s12938-022-01001-x#citeas, discussing that “Machine learning, especially deep learning, is becoming more and more relevant in research and development in the medical domain. For all the supervised deep learning applications, data is the most critical factor in securing successful implementation and sustaining the progress of the machine learning model. Especially gastroenterological data, which often involves endoscopic videos, are cumbersome to annotate. Domain experts are needed to interpret and annotate the videos. To support those domain experts, we generated a framework. With this framework, instead of annotating every frame in the video sequence, experts are just performing key annotations at the beginning and the end of sequences with pathologies, e.g., visible polyps. Subsequently, non-expert annotators supported by machine learning add the missing annotations for the frames in-between” (at Abstract).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SCOTT D GARTLAND whose telephone number is (571)270-5501. The examiner can normally be reached M-F 8:30 AM - 5 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kambiz Abdi can be reached at 571-272-6702. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SCOTT D GARTLAND/
Primary Examiner, Art Unit 3685
Prosecution Insights