Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Detailed Action
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Notice to Applicant
The following is a Final Office action to Application Serial Number 17/782,716, filed on June 6, 2022. In response to Examiner’s Office Action of April 16, 2025, Applicant, on July 7, 2025, amended claims 1, 5-8, 10, 13-16 and 20. Claims 1, 5-10, 13-16 and 18-20 are pending in this application and have been rejected below.
Response to Amendment
Applicant’s amendments are acknowledged.
Regarding the 35. U.S.C. § 101 rejection, Applicant’s arguments have been
considered but are insufficient to overcome the rejection. Please refer to the 35 U.S.C.§ 101 rejection for further explanation and rationale.
The 35 U.S.C. § 103 rejections are hereby amended pursuant to applicants’ amendments. Updated 35 U.S.C. § 103 rejections have been applied to amended claims. Please refer to the § 103 rejection for further explanation and rationale.
On Pg. 8-9 of the Remarks, regarding 35 U.S.C. § 103 rejections. Applicant states prior art fails to disclose amended claim language which recites “the classifier is trained to identify contextual factors that impact respective KPIs, wherein the contextual factors are not derived from the performances of monitored individuals and the KPI benchmarker generates a determined KPI for a subject based on the identified contextual factors." In response, new ground(s) of rejection is made necessitated by amendment see MPEP 706.07a where Li is now applied for Claims 1, 10 and 16. Regarding the 35 U.S.C. § 103 rejection, Applicant’s arguments with respect to claims has been considered but are moot in view of the new grounds of rejection.
Response to Arguments
Applicant’s arguments filed July 7, 2025 have been fully considered but they are not persuasive and/or are moot in view of the revised rejections. Applicant’s arguments will be addressed herein below in the order in which they appear in the response filed July 7, 2025.
On Pg. 7-8 of the Remarks, regarding 35 U.S.C. § 101 rejections, Applicant states the alleged abstract idea does not fall under the USPTO category of Methods of Organizing Human Activities and therefore, characterizing the claims as falling under a subcategory that does not fit the claims is improper. Applicant further states the claims also provide the "how" for any abstract idea consistent with that indicated by the Examiner If the claims were properly found to recite an abstract idea. In response, the claims are directed to the abstract idea grouping of Methods of Organizing Human Activity-managing interactions. The claims primarily recite the additional element of using computer components to perform each step. The “digital information repository”, “benchmark performance engine”, “processor”, “computing apparatus”, “memory”, “computer:” and “computer-readable storage medium” is recited at a high-level of generality, such that it amounts no more than mere instructions to apply the exception using a computer component. See MPEP 2106.05(f). Please see the additional analysis below.
On Pg. 10-11 of the Remarks, regarding 35 U.S.C. § 101 rejections, Applicant states even if the claims were properly found to include an abstract idea, the claims spell out how a technology solution to a technology problem is provided by leveraging a structured data format that is used to extract a clinical context based on a clinical context extraction algorithm and based on a workflow context extraction algorithm that is separate from the clinical context extraction algorithm. In response, the claims fail to recite any improvements to another technology or technical field, improvements to the functioning of the computer itself, use of a particular machine, effecting a transformation or reduction of a particular article to a different state or thing, and/or an additional element applies or uses the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception. See 84 Fed. Reg. 55. In particular, there is a lack of improvement to a computer or technical field in performing the analysis.
On Pg. 11 of the Remarks, regarding 35 U.S.C. § 103 rejections. Applicant states SOLILOV and LI., whether taken individually or in combination, do not teach extraction from the data in the structured format in the digital data repository a clinical context based on a clinical context extraction algorithm and a workflow context based on a workflow context extraction algorithm that is separate from the clinical context extraction algorithm. In response Examiner respectfully disagrees. Solilov discloses in Par. 105 multiple algorithms to extract and analyze data for workflow context and clinical context (i.e. patient data). See Par. 105 [workflow context algorithm]5) autonomous and continuous (or substantially continuous) evaluation of contextual KPIs with intelligent model selection to isolate events of interest using success-driven statistical algorithm; …;[clinical context algorithm} 7) cross-checking or validation of the information across redundant networked information sources to achieve statistically significant event identification. Solilov discloses structured formatting In Par. 136-137-Generally, computer-executable instructions include routines, programs, objects, components, data structures, etc., that perform particular tasks or implement particular abstract data types. Please see 103 analysis below for further detail.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 5-10, 13-16 and 18-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Claims 1, 5-10, 13-16, and 18-20 are directed to context-based performance benchmarking.
Claim 1 recites a system for context-based performance benchmarking, Claim 10 recites a method for context-based performance benchmarking and Claim 16 recites an article of manufacture for context-based performance benchmarking, which include access the data in the structured format; wherein the classifier is trained to identify contextual factors that impact respective KPIs, wherein the contextual factors are not derived from the performances of monitored individuals and the KPI benchmarker generates a determined KPI for a subject based on the identified contextual factors; determine a KPI of interest for the individual of interest based at least in part on the data in the structured format in the digital data repository about the performances of the individual of interest and the identified contextual factors; extract from the data in the structured format in the digital data repository a clinical context based on a clinical context extraction algorithm and a workflow context based on a workflow context extraction algorithm that is separate from the clinical context extraction algorithm(Claim 1). Retrieving data in a structured format indicative of performances of monitored individuals, including performances of an individual of interest and a patient-specific clinical and workflow context; identifying, by a trained classifier, contextual factors that impact respective key performance indicators (KPIs) that are not derived from the performances of the monitored individuals; using a natural language processing algorithm or a database query to extract from the data in the structured format in the digital data repository a clinical context based on a clinical context extraction algorithm and a workflow context based on a workflow context extraction algorithm that is separate from the clinical context extraction algorithm; and generating, by a performance benchmarking engine, a KPI for the individual of interest based on data in the structured format in the digital data repository about the performances of the individual of interest and the identified contextual factors (Claim 10). accessing data in a structured format indicative of performances of monitored individuals, including performances of the individual of interest and a patient-specific clinical and workflow context, from a digital data repository; identifying, by a trained classifier, contextual factors that impact respective key performance indicators (KPIs) that are not derived from the performances of the monitored individuals; using a natural language processing algorithm or a database query to extract from the data in the structured format in the digital data repository a clinical context from the digital data repository based on a clinical context extraction algorithm and a workflow context from the digital data repository based on a workflow context extraction algorithm that is separate from the clinical context extraction algorithm; and generating a KPI for the individual of interest based on data in the structured format in the digital data repository indicative of the performances of the individual of interest and the identified contextual factors; (Claim 16). As drafted, this is, under its broadest reasonable interpretation, within the Abstract idea grouping of “Methods of Organizing Human Activities” – managing interactions. The recitation of “system”, “computer”, “digital data repository”, “benchmark performance engine”, “benchmarker”, “classifier”, “processor”, “database”; “computing apparatus”, “memory”, and “computer-readable storage medium”, provide nothing in the claim elements to preclude the step from being “Methods of Organizing Human Activity”- managing interactions. Accordingly, the claim recites an abstract idea.
This judicial exception is not integrated into a practical application. The claims primarily recite the additional element of using computer components to perform each step. The “system”, “computer”, “digital data repository”, “benchmark performance engine”, “benchmarker”, “classifier”, “processor”, “database”; “computing apparatus”, “memory”, and “computer-readable storage medium” is recited at a high-level of generality, such that it amounts no more than mere instructions to apply the exception using a computer component. See MPEP 2106.05(f). Furthermore, the claim 1, claim 10 and claim 16 recite using a trained classifier and “natural language processing”. The specification discloses the machine learning at a high-level of generality, providing examples of different techniques that may be applied. The general use of a machine learning analysis does not provide a meaningful limitation to transform the abstract idea into a practical application. Therefore, currently, the trained classifier /natural language processing is solely used a tool to perform the instructions of the abstract idea.
Accordingly, the additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claims also fail to recite any improvements to another technology or technical field, improvements to the functioning of the computer itself, use of a particular machine, effecting a transformation or reduction of a particular article to a different state or thing, and/or an additional element applies or uses the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception. See 84 Fed. Reg. 55. In particular, there is a lack of improvement to a computer or technical field in performance benchmarking analysis.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements when considered both individually and as an ordered combination do not amount to significantly more than the abstract idea. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of “system”, “computer”, “digital data repository”, “benchmark performance engine”, “benchmarker”, “classifier”, “processor”, “database”; “computing apparatus”, “memory”, and “computer-readable storage medium” is insufficient to amount to significantly more. (See MPEP 2106.05(f) – Mere Instructions to Apply an Exception – “Thus, for example, claims that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not render an abstract idea eligible.” Alice Corp., 134 S. Ct. at 235). Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept.
The claim fails to recite any improvements to another technology or technical field, improvements to the functioning of the computer itself, use of a particular machine, effecting a transformation or reduction of a particular article to a different state or thing, adding unconventional steps that confine the claim to a particular useful application, and/or meaningful limitations beyond generally linking the use of an abstract idea to a particular environment. See 84 Fed. Reg. 55. Viewed individually or as a whole, these additional claim element(s) do not provide meaningful limitation(s) to transform the abstract idea into a patent eligible application of the abstract idea such that the claim(s) amounts to significantly more than the abstract idea itself. With regards to receiving data and step 2B, it is M2106.05(d)- Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information) and Storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015). Regarding 2B and the trained classifier- it is MPEP 2106.05(f).
Examiner concludes that the additional elements in combination fail to amount to significantly more than the abstract idea based on findings that each element merely performs the same function(s) in combination as each element performs separately. The claim is not patent eligible. Thus, taken alone, the additional elements do not amount to significantly more than the above-identified judicial exception (the abstract idea). Looking at the limitations as an ordered combination adds nothing that is not already present when looking at the elements taken individually.
Dependent Claims 5-9, 13-15 and 18-20 recite information includes wherein the performance benchmarking engine includes one or more of a patient-specific clinical and a workflow factor identifying module configured to determine the factors from the clinical context and the workflow; wherein one or more of the patient- specific clinical and the workflow factor identifying module is configured to determine the factors based on at least one of a supervised prediction or a classification; wherein the performance benchmarking engine further includes a benchmark performance module configured to determine the key performance indicator of interest for the individual of interest to remove a performance bias introduced by the factors; wherein the performance benchmarking engine further includes a benchmark performance module configured to determine the key performance indicator of interest for the individual of interest by excluding factors that introduce the performance bias; an output device configured to display the determined key performance indicator of interest; and further narrowing the abstract idea. These recited limitations in the dependent claims do not amount to significantly more than the above-identified judicial exceptions in Claims 1, 10 and 16. Regarding Claims, 5-9, and 18-20, and the additional elements of “performance benchmarking engine”; “module”; “digital information repository”; “digital data repository” and “output device” it is M2106.05(d)- Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 5-6, 9-10, 13-14, 16 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Solilov et al., US Publication No. 20130132108A1 [hereinafter Solilov], in view of Li et al., US Patent No. 11250948B2, [hereinafter Li].
Regarding Claim 1,
Solilov teaches
A system, for automatically compensating for contextual factors during computer generation of a key performance indicator (KPI) of an individual of interest, comprising: a digital data repository configured to electronically store data in a structured format indicative of performances of monitored individuals, including performances of the individual of interest and a patient-specific clinical and workflow context (Solilov Abstract; Par. 41- KPIs can be highlighted and associated with actions in response to various conditions, such as, but not limited to, long patient wait times, a modality that is underutilized, a report for stroke, a performance metric that is not meeting hospital guidelines, or a referring physician that is continuously requesting films when exams are available electronically through a hospital portal. Performance indicators addressing specific areas of performance can be acted upon in real time (or substantially real time accounting for processing, storage/retrieval, and/or transmission delay), for example. Par. 45; “Par. 90-“Certain examples provide an extensible workflow definition wherein a generic event can be defined which represents any state. An example engine dynamically adapts to needs of a customer without planning in advance for each possible workflow of the user. For example, if a user's workflow is defined today to include A, B, C, and D, the definition can be dynamically expanded to include E, F, and G and be tracked, measured, and accommodated for performance without creating rows and columns in a workflow state database for each workflow eventuality in advance.”; Par. 6; Fig. 4; Par. 108; Par. 112; Par. 136-137- data structures; Fig 4 [401 data is patient specific i.e. RIS data]; Par. 13-“ FIG. 3 depicts a flow diagram for an example method for computation and output of operational metrics for patient and exam workflow.; Par. 7-8- The example method includes extracting context information from the identified patterns and data mined information. The example method includes dynamically creating contextual performance indicators based on the context and pattern information. ;Par. 22-24; Par. 23-Certain examples help streamline a patient scanning process in radiology by providing transparency to workflow occurring in disparate systems. Current patient scanning workflow in radiology is managed using paper requisitions printed from a radiology information system (RIS) or manually tracked on dry erase whiteboards. Given the disparate systems used to track patient prep, lab results, oral contrast, it is difficult for Technologists to be efficient, as they need to poll the different systems to check status of patient.);
and a computing apparatus configured to electronically access the data in the structured format stored in the digital data repository, comprising: a memory configured to store computer instructions for a performance benchmarking engine comprising.(Solilov Par. 108; Par. 114; Fig. 4; Par. 136-137-“ Certain embodiments include computer-readable media for carrying or having computer-executable instructions or data structures stored thereon. Such computer-readable media may be any available media that may be accessed by a general purpose or special purpose computer or other machine with a processor. By way of example, such computer-readable media may comprise RAM, ROM, PROM, EPROM, EEPROM, Flash, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to carry or store desired program code in the form of computer-executable instructions or data structures and which can be accessed by a general purpose or special purpose computer or other machine with a processor. Combinations of the above are also included within the scope of computer-readable media. Computer-executable instructions comprise, for example, instructions and data which cause a general purpose computer, special purpose computer, or special purpose processing machines to perform a certain function or group of functions. Generally, computer-executable instructions include routines, programs, objects, components, data structures, etc., that perform particular tasks or implement particular abstract data types. Computer-executable instructions, associated data structures, and program modules represent examples of program code for executing steps of certain methods and systems disclosed herein. The particular sequence of such executable instructions or associated data structures represent examples of corresponding acts for implementing the functions described in such steps.”)
and a processor configured to execute the stored computer instructions for the performance benchmarking engine to determine a KPI of interest for the individual of interest based at least in part on the data in the structured format in the digital data repository about the performances of the individual of interest and the identified contextual factors... (Solilov Par. 81-84-“At block 320, one or more patient(s) and/or equipment of interest are selected for evaluation and review. For example, one or more patients in one or more hospital departments and one or more pieces of imaging equipment (e.g., CT scanners) are selected for review and KPI generation. At block 330, scheduled procedures are displayed for review. At block 340, a user can specify one or more conditions to affect interpretation of the data in the data set. For example, the user can specify whether any or all states relevant to a workflow of interest have or have not been reached. For example, the user also has an ability to pass relevant filter(s) that are specific to a hospital workflow. A resulting data set is built dynamically based on the user conditions. At block 350, a completion time for an event of interest is determined. At block 360, a delay associated with the event of interest is evaluated. At block 370, one or more reasons for delay can be provided. For example, equipment setup time, patient preparation time, conflicted usage time, etc., can be provided as one or more reasons for a delay.; Par. 125; Fig. 5; Fig. 4; Par. 112; Par. 136-137)
wherein the performance benchmarking engine includes a patient-specific clinical and workflow profiling module configured to use a natural language processing algorithm or a database query to extract from the data in the structured format in the digital data repository a clinical based on a clinical context extraction algorithm and a workflow context based on a workflow context extraction algorithm that is separate from the clinical context extraction algorithm, (Solilov Par. 8-10; Par. 43; Par. 105-“ FIG. 4 illustrates an example alerting and decision-making system 400. The example system 400 provides an “intelligent” alerting and decision-making artificial intelligence engine based on dynamic contextual KPIs of continuous (or substantially continuous) future-progress data distribution statistical pattern matching. The engine includes: 1) a plug and play collection of existing healthcare departmental workflows; 2) pattern recognition of a healthcare departmental workflow based on historical and current data; 3) context extraction from data-mined information that forms a basis for contextual KPIs with healthcare-specific departmental filtering applied to provide intelligent metrics; 4) dynamic creation of contextual KPIs by joining one or more healthcare specific departmental workflow contexts; [workflow context algorithm]5) autonomous and continuous (or substantially continuous) evaluation of contextual KPIs with intelligent model selection to isolate events of interest using success-driven statistical algorithm; 6) continuously (or substantially continuously) evolving monitoring based on user-evaluated success rate of identified events alerting or auto-evaluation algorithm if system is run in an autonomous mode with add-ons (e.g., expansions or additions);[clinical context algorithm} 7) cross-checking or validation of the information across redundant networked information sources to achieve statistically significant event identification ; 8) hospital enterprise network aware feedback incorporation and tandem cross-cooperative operation; etc. The engine may operate fully autonomous or semi-autonomously, for example. While user input is not needed for functional operation, user input yields faster conversion to an improved or optimal operational point, for example.”; Par. 24-30- Current dashboard solutions are typically based on data in a RIS or picture archiving and communication system (PACS). Certain examples provide an ability to aggregate data from a plurality of sources including RIS, PACS, modality, virtual radiography (VR), scheduling, lab, pharmacy systems, etc. A flexible workflow definition enables example systems and methods to be customized to customer workflow configuration with relative ease.; Par. 69-“ The data aggregation engine 210 has pre-built exam and patient events, and supports an ability to add custom events to map to site workflow. The engine 210 provides a user interface in the form of an inquiry view, for example, to query for audit event(s). The inquiry view supports queries using the following criteria within a specified time range: patient, exam, staff, event type(s), etc. The inquiry view can be used to look up audit information on an exam and visit events within a certain time range (e.g., six weeks). The inquiry view can be used to check a current workflow status of an exam. The inquiry view can be used to verify staff patient interaction audit compliance information by cross-referencing patient and staff information.”; Par. 108; Par. 112-113; Par. 136-137)
Solilov teaches contextual analysis and the feature is expounded upon by Li:
…a classifier and a key performance indicator (KPI) benchmarker, wherein the classifier is trained to identify contextual factors that impact respective KPIs, wherein the contextual factors are not derived from the performances of monitored individuals and the KPI benchmarker generates a determined KPI for a subject based on the identified contextual factors (Li Col 3 – sparse data; Col 6- disparate data; Col 4; Col 9- FIG. 10 is a block diagram of the change characterization module 720, which may receive KPI time series data and change detection thresholds by run type with confidence, for each detected driver. Change characterization module 720 may contain a change type classifier 1010, which classifies results from various analysis types into a change profile. The change characterization module turns KPI time series into interpretable features, e.g., persistent growth (slow rise over time), emerging decline (sharp drop after slow decline). To characterize KPI data, supervised machine learning classifiers (e.g., rules-based, neural net, etc.) or unsupervised classifiers (e.g., clustering, etc.) may be used. Change characterization module 720 generates change patterns with confidence 1020.FIG. 11 is a flowchart of change impact learning module 620. This module assesses the impact of change in a total performance score.; Col 23);
Solilov and Li are directed to contextual analysis. It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have improve upon data analysis of Solilov, as taught by Li, by utilizing machine learning analysis with a reasonable expectation of success of arriving at the claimed invention. One of ordinary skill in the art would have been motivated to make the modification to the teachings of Solilov with the motivation of identifying and optimizing performance drivers (Li Abstract).
Regarding Claim 2, - Cancelled
Regarding Claim 3, - Cancelled
Regarding Claim 4 and Claim 12 - Cancelled,
Regarding Claim 5, Claim 13 and Claim 18,
wherein the performance benchmarking engine includes a patient-specific clinical and workflow factor identifying module configured to determine a workflow factor from the clinical context and a workflow (Solilov Fig 4 [401 data is patient specific i.e. RIS data]; Par. 13-“ FIG. 3 depicts a flow diagram for an example method for computation and output of operational metrics for patient and exam workflow.; Par. 7-8- The example method includes extracting context information from the identified patterns and data mined information. The example method includes dynamically creating contextual performance indicators based on the context and pattern information; Par. 25-26; Par. 37-“ Certain examples help provide an understanding of the real-time operational effectiveness of an enterprise and help enable an operator to address deficiencies. Certain examples thus provide an ability to collect, analyze and review operational data from a healthcare enterprise in real time or substantially in real time given inherent processing, storage, and/or transmission delay. The data is provided in a digestible manner adjusted for factors that may artificially affect the value of the operational data (e.g., patient wait time) so that an appropriate responsive action may be taken.”; Par.42; Par. 47; Par. 103)
Regarding Claim 6 and Claim 14,
wherein the patient-specific clinical and workflow factor identifying module is configured to determine a workflow factor based on at least one of a supervised prediction or a classification. (Solilov Fig 4 [401 data is patient specific i.e. RIS data]; Par. 13-“ FIG. 3 depicts a flow diagram for an example method for computation and output of operational metrics for patient and exam workflow.; Par. 7-8- The example method includes extracting context information from the identified patterns and data mined information. The example method includes dynamically creating contextual performance indicators based on the context and pattern information; Par. 25-26; Par. 72-77; Par. 94; Par. 109; Par. 112-113-“ The contextual ordering of events 405 is provided to a historical context repository 406. The historical context repository 406, at 3 in the example of FIG. 4, provides data-mined information for context extraction by a predictive modeler 407 to provide a basis for one or more contextual KPIs. In certain examples, healthcare-specific departmental filtering is applied to provide only “intelligent” metrics applicable to a particular workflow, situation, constraints, and/or environment at hand. The predictive modeler 407 processes the historical context information to provide input to one or more optimization and enhancement engines 408. The optimization and enhancement engines 408 shown in the example of FIG. 4 include a workflow decision engine 409 and a result effectiveness analysis engine 410. The predictive modeler 407 can also provide feedback 411 to the contextual analysis engine 403.”; Par. 108; Par. 112-113; Par. 136-137)
Regarding Claim 9,
The system of claim 1, further comprising: an output device configured to display the determined key performance indicator of interest. (Solilov Par. 24-26; Par. 71- The real-time dashboard 220 supports a variety of capabilities (e.g., in a web-based format). The dashboard 220 can organize KPI by facility and allow a user to drill-down from an enterprise to an individual facility (e.g., a hospital). The dashboard 220 can display multiple KPI simultaneously (or substantially simultaneously), for example. The dashboard 220 provides an automated “slide show” to display a sequence of open KPI. The dashboard 220 can be used to save open KPI, generate report(s), export data to a spreadsheet, etc.”)
Regarding Claim 10,
Solilov teaches
A computer-implemented method for automatically compensating for contextual factors during computer generation of a key performance indicator (KPI) of an individual of interest, comprising: retrieving data in a structure format indicative of performances of monitored individuals, including performances of an individual of interest and a patient-specific clinical and workflow context, from a digital data repository; (Solilov Abstract; Par. 41- KPIs can be highlighted and associated with actions in response to various conditions, such as, but not limited to, long patient wait times, a modality that is underutilized, a report for stroke, a performance metric that is not meeting hospital guidelines, or a referring physician that is continuously requesting films when exams are available electronically through a hospital portal. Performance indicators addressing specific areas of performance can be acted upon in real time (or substantially real time accounting for processing, storage/retrieval, and/or transmission delay), for example.; Par. 45; “Par. 90-“Certain examples provide an extensible workflow definition wherein a generic event can be defined which represents any state. An example engine dynamically adapts to needs of a customer without planning in advance for each possible workflow of the user. For example, if a user's workflow is defined today to include A, B, C, and D, the definition can be dynamically expanded to include E, F, and G and be tracked, measured, and accommodated for performance without creating rows and columns in a workflow state database for each workflow eventuality in advance.”; Par. 6; Par. 136-137; Fig 4 [401 data is patient specific i.e. RIS data]; Par. 13-“ FIG. 3 depicts a flow diagram for an example method for computation and output of operational metrics for patient and exam workflow.; Par. 7-8- The example method includes extracting context information from the identified patterns and data mined information. The example method includes dynamically creating contextual performance indicators based on the context and pattern information. ;Par. 22-24; Par. 23-Certain examples help streamline a patient scanning process in radiology by providing transparency to workflow occurring in disparate systems. Current patient scanning workflow in radiology is managed using paper requisitions printed from a radiology information system (RIS) or manually tracked on dry erase whiteboards. Given the disparate systems used to track patient prep, lab results, oral contrast, it is difficult for Technologists to be efficient, as they need to poll the different systems to check status of patient.); Par. 108);
using a natural language processing algorithm or a database query to extract from the data in the structured format in the digital data repository a clinical context based on a clinical context extraction algorithm and a workflow context based on a workflow context extraction algorithm that is separate from the clinical context extraction algorithm, (Solilov Par. 8-10; Par. 43; Par. 105-“ FIG. 4 illustrates an example alerting and decision-making system 400. The example system 400 provides an “intelligent” alerting and decision-making artificial intelligence engine based on dynamic contextual KPIs of continuous (or substantially continuous) future-progress data distribution statistical pattern matching. The engine includes: 1) a plug and play collection of existing healthcare departmental workflows; 2) pattern recognition of a healthcare departmental workflow based on historical and current data; 3) context extraction from data-mined information that forms a basis for contextual KPIs with healthcare-specific departmental filtering applied to provide intelligent metrics; 4) dynamic creation of contextual KPIs by joining one or more healthcare specific departmental workflow contexts; [workflow context algorithm]5) autonomous and continuous (or substantially continuous) evaluation of contextual KPIs with intelligent model selection to isolate events of interest using success-driven statistical algorithm; 6) continuously (or substantially continuously) evolving monitoring based on user-evaluated success rate of identified events alerting or auto-evaluation algorithm if system is run in an autonomous mode with add-ons (e.g., expansions or additions);[clinical context algorithm} 7) cross-checking or validation of the information across redundant networked information sources to achieve statistically significant event identification ; 8) hospital enterprise network aware feedback incorporation and tandem cross-cooperative operation; etc. The engine may operate fully autonomous or semi-autonomously, for example. While user input is not needed for functional operation, user input yields faster conversion to an improved or optimal operational point, for example.”; Par. 24-30- Current dashboard solutions are typically based on data in a RIS or picture archiving and communication system (PACS). Certain examples provide an ability to aggregate data from a plurality of sources including RIS, PACS, modality, virtual radiography (VR), scheduling, lab, pharmacy systems, etc. A flexible workflow definition enables example systems and methods to be customized to customer workflow configuration with relative ease.; Par. 69-“ The data aggregation engine 210 has pre-built exam and patient events, and supports an ability to add custom events to map to site workflow. The engine 210 provides a user interface in the form of an inquiry view, for example, to query for audit event(s). The inquiry view supports queries using the following criteria within a specified time range: patient, exam, staff, event type(s), etc. The inquiry view can be used to look up audit information on an exam and visit events within a certain time range (e.g., six weeks). The inquiry view can be used to check a current workflow status of an exam. The inquiry view can be used to verify staff patient interaction audit compliance information by cross-referencing patient and staff information.”; Par. 108; Par. 112-113; Par. 136-137)
and generating, by a performance benchmarking engine, a KPI for the individual of interest based on data in the structured format in the digital data repository about the performances of the individual of interest and the identified contextual factors. (Solilov Par. 81-84-“At block 320, one or more patient(s) and/or equipment of interest are selected for evaluation and review. For example, one or more patients in one or more hospital departments and one or more pieces of imaging equipment (e.g., CT scanners) are selected for review and KPI generation. At block 330, scheduled procedures are displayed for review. At block 340, a user can specify one or more conditions to affect interpretation of the data in the data set. For example, the user can specify whether any or all states relevant to a workflow of interest have or have not been reached. For example, the user also has an ability to pass relevant filter(s) that are specific to a hospital workflow. A resulting data set is built dynamically based on the user conditions. At block 350, a completion time for an event of interest is determined. At block 360, a delay associated with the event of interest is evaluated. At block 370, one or more reasons for delay can be provided. For example, equipment setup time, patient preparation time, conflicted usage time, etc., can be provided as one or more reasons for a delay.; Par. 125; Fig. 5; Par. 108; Par. 136-137)
Solilov teaches contextual analysis and the feature is expounded upon by Li:
identifying, by a trained classifier, contextual factors that impact respective key performance indicators (KPIs) that are not derived from the performances of the monitored individuals; (Li Col 3 – sparse data; Col 6- disparate data; Col 4; Col 9- FIG. 10 is a block diagram of the change characterization module 720, which may receive KPI time series data and change detection thresholds by run type with confidence, for each detected driver. Change characterization module 720 may contain a change type classifier 1010, which classifies results from various analysis types into a change profile. The change characterization module turns KPI time series into interpretable features, e.g., persistent growth (slow rise over time), emerging decline (sharp drop after slow decline). To characterize KPI data, supervised machine learning classifiers (e.g., rules-based, neural net, etc.) or unsupervised classifiers (e.g., clustering, etc.) may be used. Change characterization module 720 generates change patterns with confidence 1020.FIG. 11 is a flowchart of change impact learning module 620. This module assesses the impact of change in a total performance score.; Col 23);
Solilov and Li are directed to contextual analysis. It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have improve upon data analysis of Solilov, as taught by Li, by utilizing machine learning analysis with a reasonable expectation of success of arriving at the claimed invention. One of ordinary skill in the art would have been motivated to make the modification to the teachings of Solilov with the motivation of identifying and optimizing performance drivers (Li Abstract).
Claim 11 and Claim 17 - Cancelled
Regarding Claim 16,
Solilov teaches
A non-transitory computer-readable storage medium storing computer executable instructions which when executed by a processor of a computer cause the processor to execute a method for automatically compensating for contextual factors during computer generation of a key performance indicator (KPI) of an individual of interest, the method including: accessing data in a structured format indicative of performances of monitored individuals, including performances of the individual of interest and a patient-specific clinical and workflow context , from a digital data repository; (Solilov Abstract; Par. 9; Solilov Abstract; Par. 41- KPIs can be highlighted and associated with actions in response to various conditions, such as, but not limited to, long patient wait times, a modality that is underutilized, a report for stroke, a performance metric that is not meeting hospital guidelines, or a referring physician that is continuously requesting films when exams are available electronically through a hospital portal. Performance indicators addressing specific areas of performance can be acted upon in real time (or substantially real time accounting for processing, storage/retrieval, and/or transmission delay), for example.; Par. 45; “Par. 90-“Certain examples provide an extensible workflow definition wherein a generic event can be defined which represents any state. An example engine dynamically adapts to needs of a customer without planning in advance for each possible workflow of the user. For example, if a user's workflow is defined today to include A, B, C, and D, the definition can be dynamically expanded to include E, F, and G and be tracked, measured, and accommodated for performance without creating rows and columns in a workflow state database for each workflow eventuality in advance.”; Par. 6; Par. 136-137; Fig 4 [401 data is patient specific i.e. RIS data]; Par. 13-“ FIG. 3 depicts a flow diagram for an example method for computation and output of operational metrics for patient and exam workflow.; Par. 7-8- The example method includes extracting context information from the identified patterns and data mined information. The example method includes dynamically creating contextual performance indicators based on the context and pattern information. ;Par. 22-24; Par. 23-Certain examples help streamline a patient scanning process in radiology by providing transparency to workflow occurring in disparate systems. Current patient scanning workflow in radiology is managed using paper requisitions printed from a radiology information system (RIS) or manually tracked on dry erase whiteboards. Given the disparate systems used to track patient prep, lab results, oral contrast, it is difficult for Technologists to be efficient, as they need to poll the different systems to check status of patient.); Par. 108); Par. 6;);
using a natural language processing algorithm or a database query to extract from the data in the structured format in the digital data repository a clinical context from the digital data repository based on a clinical context extraction algorithm and a workflow context from the digital data repository based on a workflow context extraction algorithm that is separate from the clinical context extraction algorithm, (Solilov Par. 8-10; Par. 43; Par. 105-“ FIG. 4 illustrates an example alerting and decision-making system 400. The example system 400 provides an “intelligent” alerting and decision-making artificial intelligence engine based on dynamic contextual KPIs of continuous (or substantially continuous) future-progress data distribution statistical pattern matching. The engine includes: 1) a plug and play collection of existing healthcare departmental workflows; 2) pattern recognition of a healthcare departmental workflow based on historical and current data; 3) context extraction from data-mined information that forms a basis for contextual KPIs with healthcare-specific departmental filtering applied to provide intelligent metrics; 4) dynamic creation of contextual KPIs by joining one or more healthcare specific departmental workflow contexts; [workflow context algorithm]5) autonomous and continuous (or substantially continuous) evaluation of contextual KPIs with intelligent model selection to isolate events of interest using success-driven statistical algorithm; 6) continuously (or substantially continuously) evolving monitoring based on user-evaluated success rate of identified events alerting or auto-evaluation algorithm if system is run in an autonomous mode with add-ons (e.g., expansions or additions);[clinical context algorithm} 7) cross-checking or validation of the information across redundant networked information sources to achieve statistically significant event identification ; 8) hospital enterprise network aware feedback incorporation and tandem cross-cooperative operation; etc. The engine may operate fully autonomous or semi-autonomously, for example. While user input is not needed for functional operation, user input yields faster conversion to an improved or optimal operational point, for example.”; Par. 24-30- Current dashboard solutions are typically based on data in a RIS or picture archiving and communication system (PACS). Certain examples provide an ability to aggregate data from a plurality of sources including RIS, PACS, modality, virtual radiography (VR), scheduling, lab, pharmacy systems, etc. A flexible workflow definition enables example systems and methods to be customized to customer workflow configuration with relative ease.; Par. 69-“ The data aggregation engine 210 has pre-built exam and patient events, and supports an ability to add custom events to map to site workflow. The engine 210 provides a user interface in the form of an inquiry view, for example, to query for audit event(s). The inquiry view supports queries using the following criteria within a specified time range: patient, exam, staff, event type(s), etc. The inquiry view can be used to look up audit information on an exam and visit events within a certain time range (e.g., six weeks). The inquiry view can be used to check a current workflow status of an exam. The inquiry view can be used to verify staff patient interaction audit compliance information by cross-referencing patient and staff information.”; Par. 108; Par. 112-113; Par. 136-137)
and generating, by a performance benchmarking engine, a KPI for the individual of interest based on data in a structured format in the digital data repository indicative of the performances of the individual of interest and the identified contextual factors (Solilov Par. 81-84-“At block 320, one or more patient(s) and/or equipment of interest are selected for evaluation and review. For example, one or more patients in one or more hospital departments and one or more pieces of imaging equipment (e.g., CT scann