DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
In the amendment filed 02/24/2026 the following occurred: Claims 1-6, 9-14, and 17-19 were amended; and Claims 7-8 and 20 were cancelled. Claims 1-6 and 9-19 are presented for examination.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-6 and 9-19 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
Claims 1-6 and 9-19 are drawn to a system, a method and a device, which is/are statutory categories of invention (Step 1: YES).
Independent claim 1 recites perform an analysis of patient data using a clinical data product (CDP); summarize a clinical context of the analysis performed by the CDP on the patient data, wherein the clinical context is summarized into a structured text file including labeled data in a standardized format; perform a health equity assessment of the CDP based on the summarized clinical context and the clinical feedback, using one or more disparity assessment models to assess an algorithmic bias of the CDP towards a specific patient population, the labeled data of the structured text file used as input data for the one or more disparity assessment models; calculate a confidence score for the CDP based on the health equity assessment, the confidence score indicating a degree of confidence that a subsequent analysis of a subsequent patient using the CDP does not suffer from algorithmic bias.
Independent claim 13 recites summarizing a clinical context of the analysis performed by the CDP on the patient data, wherein the clinical context is summarized into a structured text file including labeled data in a standardized format; performing a health equity assessment of the CDP, the health equity assessment evaluating the summarized clinical context and the rating using one or more disparity assessment models to assess an algorithmic bias of the CDP, the labeled data of the structured text file used as input data for the one or more disparity assessment models; calculating a confidence score for the CDP, based on the assessed algorithmic bias.
Independent claim 19 recites to deploy a clinical data product (CDP) on patient data; receive an output of the CDP… assess the algorithmic bias of the CDP.
Respective dependent claims 2-6, 9-12, and 14-18, but for the inclusion of the additional elements specifically addressed below, provide recitations further limiting the invention of the independent claims.
Said recited limitations, as drafted, under their broadest reasonable interpretation, cover certain methods of organizing human activity, as reflected in the specification, which states that having “identifying bias in AI algorithms used in [clinician] tools” (see: specification paragraph 1). If a claim limitation, under its broadest reasonable interpretation, covers managing personal behavior or relationships or interactions between people, then it falls within the “Certain Methods of Organizing Human Activity” grouping of abstract ideas. The present claims cover certain methods of organizing human activity because they address a problem where “patient may not be representative of the training patient population”, and “[a]s a result of this algorithmic bias, an inappropriate treatment plan for the patient may be proposed, which may lead to a poorer patient outcome” (see: specification paragraph 21). The recited limitations provide, “[t]o address this problem, a confidence and health equity assessment system” so that “biases in various CDPs may be identified and rectified, improving the quality and equity of clinical data products and enabling care providers and patients to make more informed decisions” (see: specification paragraph 28). As addressed in the specification, the present claims further address that “the health equity assessment includes evaluating the summarized clinical context data using one or more disparity assessment models” (see: specification paragraph 88), and “each confidence score indicates a degree of confidence (for example, on a scale of one to 10) that the output of the CDP does not suffer from algorithmic bias with respect to a corresponding disparity factor” (see: specification paragraph 109). If a claim limitation, under its broadest reasonable interpretation, covers mathematical relationships, or mathematical formulas or equations, or mathematical calculations, then it falls within the “Mathematical Concepts” grouping of abstract ideas. The present claims cover mathematical concepts because “disparity assessment models may assess the output of the CDP in accordance with various metrics and/or techniques…may assess an accuracy, precision, or sensitivity of the CDP…may use metrics such as calculating an F1 score; calculating a mean squared error (MAE) or root mean squared error (RMSE); calculating an R-squared value; calculating an area under a receiver operating characteristic (ROC) curve; or a different metric” (see: specification paragraph 88), and further, the “confidence score…may be an average…may be calculated…may be a lowest confidence score…” (see: specification paragraph 124) and “may provide a numeric confirmation” (see: specification paragraph 125). Accordingly, the claims recite an abstract idea(s) (Step 2A Prong One: YES).
This judicial exception is not integrated into a practical application. The claims are abstract but for the inclusion of the additional elements including a “system including a health equity assessment system and a clinician computing device, the system comprising: one or more processors and non-transitory memory storing instructions that when executed, cause the one or more processors to:…via the health equity assessment system…via the health equity system…trained…via the health equity assessment system…” (claim 1), “by a large language model (LLM) that generates a natural language text summary of the patient data” (claim 4), “LLM is prompted to…” (claim 5), “wherein further instructions are stored in the non-transitory memory that when executed, cause the one or more processors to perform” (claim 6), “A machine-implemented method for a health equity assessment system of a healthcare system, the method comprising:…” (claim 13), and “computing device comprising a processor and memory storing instructions that, when executed by the processor, cause the processor to:…a health equity assessment system configured to…” (claim 19), which are additional elements that are recited at a high level of generality (e.g., the “one or more processors and non-transitory memory” are configured to perform functions through no more than a statement than that “instructions” stored on said memory “cause” said one or more processors to perform said functions; the “trained” models are employed though no more than a statement than that an assessment is performed “using” said trained models; the “large language model (LLM)” is employed though no more than a statement than that text summary is performed “by” said LLM or that it is “prompted”; the “machine” is configured to perform a method though no more than a statement than that said machine is “implemented”; the “health equity assessment system” is configured to perform functions though no more than a statement than that said functions are performed “via” said health equity assessment system) such that they amount to no more than mere instruction to apply the exception using generic computer elements. See: MPEP 2106.05(f).
The claims recite the additional elements of “collect clinical feedback from a clinician of an output of the CDP generated during the analysis, the clinical feedback collected using a CDP rating tool micro-application of the clinician computing device; send the clinical feedback to the health equity assessment system…wherein the CDP rating tool micro-application displays: a first control element that when selected, sends a positive rating of the output of the CDP to the health equity assessment system; a second control element that when selected, sends a negative rating of the output of the CDP to the health equity assessment system; a third control element that when selected, sends an indication that sufficient information is not available to rate the output of the CDP to the health equity assessment system; and a fourth control element that allows a user of the CDP rating tool micro-application to send text data regarding the output of the CDP to the health equity assessment system” (claim 1), “via the fourth control element and store the written feedback in a retraining backlog with accumulated feedback from the clinician and/or other clinicians…” (claim 6), “receiving a rating of a clinical data product (CDP) used by a clinician in an analysis of patient data, the rating collected with a CDP rating tool micro-application of a clinician computing device operably coupled to the health equity assessment system, wherein the CDP rating tool micro-application displays: a first control element that when selected, sends a positive rating of the output of the CDP to the health equity assessment system; a second control element that when selected, sends a negative rating of the output of the CDP to the health equity assessment system; a third control element that when selected, sends an indication that sufficient information is not available to rate the output of the CDP to the health equity assessment system; and a fourth control element that allows a user of the CDP rating tool micro-application to send text data regarding the output of the CDP to the health equity assessment system…” (claim 13), and “deploy a CDP rating tool micro-application comprising a rating tool including control elements displayed with the output of the CDP, the rating tool configured to: receive clinical feedback from the clinician regarding an algorithmic bias of the CDP; and send the clinical feedback to…wherein the control elements of the rating tool include: a first control element that when selected, causes the rating tool to send a positive rating of the output of the CDP to the health equity assessment system; a second control element that when selected, causes the rating tool to send a negative rating of the output of the CDP to the health equity assessment system; a third control element that when selected, causes the rating tool to send an indication that sufficient information is not available to rate the output of the CDP to the health equity assessment system; and a fourth control element that allows a user of the rating tool to send text data regarding the output of the CDP to the health equity assessment system” (claim 19), which are nominal or tangential addition to the abstract idea(s) and amount to extra-solution activity concerning mere data gathering. The addition of an insignificant extra-solution activity limitation does not impose meaningful limits on the claim such that is it not nominally or tangentially related to the invention. In the claimed context, these claimed additional elements are incidental to the performance of the recited abstract idea(s) as outlined in the recitations above. Similarly, the claims recite the additional elements of “and store the confidence score in a database for later retrieval and display within a CDP catalog” (claims 1 and 13), which are considered an insignificant post-solution activity concerning an insignificant application, and similarly, the addition of insignificant extra-solution activity does not impose meaningful limits on the claim such that is it not nominally or tangentially related to the invention. In the claimed context, these claimed additional elements are incidental to the performance of the recited abstract idea(s) as outlined in the recitations above. See: MPEP 2106.05(g).
The combination of these additional elements is no more than mere instructions to apply the exception using generic computer elements and limitations directed toward extra-solution activity. Accordingly, even in combination, these additional elements do not integrate the abstract idea(s) into a practical application because they do not impose any meaningful limits on practicing the abstract idea(s). Accordingly, the claims are directed to an abstract idea(s) (Step 2A Prong Two: NO).
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea(s) into a practical application, using the additional elements to perform the abstract idea(s) amounts to no more than mere instructions to apply the exception using generic components. Mere instructions to apply an exception using generic components cannot provide an inventive concept. See MPEP 2106.05(f).
Further, the claimed additional elements directed toward extra-solution activity, identified above, are not sufficient to amount to significantly more than the judicial exception because they are generic components that are configured to perform well-understood, routine, and conventional activities previously known to the industry. See: MPEP 2106.05(d). Said additional elements are recited at a high level of generality and provide conventional functions that do not add meaningful limits to practicing the abstract idea(s). The originally filed specification supports this conclusion:
Paragraph 46, where “…Clinical user feedback 209 may be stored at health equity assessment system 214 (e.g., in clinical user feedback database 118 of FIG. 1)…The overall health equity assessment of CDP 206 may then be stored at health equity assessment system 214 (e.g., in confidence score database 120 of FIG. 1)…”
Paragraph 51, where “…For example, the dashboard data may be organized in a spatial layout in columns and/or rows, and may include control elements for performing actions or accessing additional data (e.g., drilling down)...”
Paragraph 64, where “…the health equity assessment may be retrieved from a database (e.g., confidence score database 120) of the health equity assessment system…”
Paragraph 77, where “…a set of control elements (e.g., selectable icons) may be displayed that allows the clinician to provide clinical feedback to the health equity assessment system…”
Paragraph 79, where “It should be noted that in other embodiments, the set of control elements 1000 of the CDP rating tool may appear differently, and may include different icons, images, or symbols, and may include a greater or fewer number of options.”
Paragraph 83, where “…The rating (and written feedback, if included) may be stored in a database of the health equity assessment system (e.g., clinical user feedback database 118 of FIG. 1)…”
Paragraph 112, where “…method 600 includes storing the multi-dimensional confidence score for the CDP in a database of the health equity assessment system (e.g., confidence score database 120 of FIG. 1). The database may be accessible to users of the CDP, for example, within a clinical workflow application, via a hospital network or the Internet. The users may include clinicians, researchers, product owners, data scientists, or other types of users, as described above in reference to FIG. 4. For example, the multi-dimensional confidence score and/or other health equity assessment data may be retrieved from the database…”
Paragraph 121, where “…Each corresponding row of a CDP may include various information about the CDP and/or control elements that may be selectable to display additional information or may be displayed in a different manner.”
The claims recite the additional elements directed to pre-solution and post-solution activity, as recited and indicated above, each of which amount to extra-solution activity. The specification (e.g., as excerpted above) does not indicate that the additional element(s) provide anything other than well‐understood, routine, and conventional functions when claimed in a merely generic manner (as they are presently). The concepts of performing clinical tests on individuals to obtain input for an equation has been identified by the courts as insignificant extra-solution activity. See: MPEP 2106.05(g). Further, the concepts of receiving or transmitting data over a network, such as using the Internet to gather data, and storing and retrieving information in memory have been identified by the courts as well-understood, routine, and conventional activities. See: MPEP 2106.05(d)(II).
Viewing the limitations as an ordered combination, the claims simply instruct the additional elements to implement the concept described above in the identification of abstract idea(s) with routine, conventional activity specified at a high level of generality in a particular technological environment. Hence, the claims as a whole, considering the additional elements individually and as an ordered combination, do not amount to significantly more than the abstract idea(s) (Step 2B: NO).
Dependent claim(s) 2-6, 9-12, and 14-18, when analyzed as a whole, considering the additional elements individually and/or as an ordered combination, are held to be patent ineligible under 35 U.S.C. 101 because the additional recited limitation(s) fail(s) to establish that the claim(s) is/are not directed to an abstract idea(s) without significantly more. These claims fail to remedy the deficiencies of their parent claims above, and are therefore rejected for at least the same rationale as applied to their parent claims above, and incorporated herein.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 13-16 is/are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Application Publication 2022/0398411 to Cowen in view of U.S. Patent Application Publication 2008/0033790 to Nickerson in view of U.S. Patent Application Publication 2023/0104655 to Amarasingham in view of U.S. Patent Application Publication to 2020/0137002 to Chavda further in view of U.S. Patent Application Publication 2019/0265870 to Sheth.
As per claim 13, Cowen teaches a machine-implemented method for a health equity assessment system of a healthcare system (see: Cowen, Fig. 1, Fig. 18, and paragraph 55, 65, and 137-139, is met by one or more electronic devices implementing a software platform, including a server, client devices, and memory), the method comprising:
receiving a rating (see: Cowen, paragraph 8-9, 16-17, 24, 68-69, 77, and 96, is met by survey responses and displaying a prompt and inputting data corresponding to a plurality of demographic groups into the trained machine-learning model to generate a plurality of outputs, where the model may be used to create a diagnosis) of a clinical data product (CDP) (see: Cowen, paragraph 59, 63, 70, 80, 94, and 96, is met by a machine-learning model can be used to annotate the recorded responses and prompt the participant for particular responses) used by a clinician in an analysis of patient data (see: Cowen, paragraph 8-9, 16-17, 24, 69, 77, and 96, is met by displaying a prompt and inputting data corresponding to a plurality of demographic groups into the trained machine-learning model, where such data inputted may include one or more characteristics such as a gender, an age, one or more races or ethnicities, a country of origin, a first language, a second language, personality, well-being, mental health, or a subjective socioeconomic status);
summarizing a clinical context of the analysis performed by the CDP on the patient data (see: Cowen, paragraph 8, 16-17, 24, and 96, is met by inputting data corresponding to a plurality of demographic groups into the trained machine-learning model to generate a plurality of outputs, where the model may be used to create a diagnosis, where such data inputted may include one or more characteristics such as a gender, an age, one or more races or ethnicities, a country of origin, a first language, a second language, personality, well-being, mental health, or a subjective socioeconomic status);
performing a health equity assessment of the CDP, the health equity assessment evaluating the summarized clinical context and the rating to assess an algorithmic bias of the CDP (see: Cowen, paragraph 8, 16-17, 24, 64, 68, 87, 96, and 106, is met by transferring the recorded data and survey data to a server, and identifying a bias of the trained machine-learning model based on the plurality of outputs, where to test for bias in a trained machine-learning model, the model may be evaluated on data from participants of differing demographic groups to determine the differential effects within each group on the predictions of the model);
calculating a confidence score for the CDP, based on the assessed algorithmic bias (see: Cowen, paragraph 8, 16, and 106, is met by the differential effects on the behavior of the model may then be examined within each group, for example gender groups returning one likeliness outcome for females and another differing likeliness outcome for males so as to indicate the bias/differences between gender groups); and
Cowen fails to specifically teach the following limitations met by Nickerson as cited:
the rating collected with a CDP rating tool micro-application of a clinician computing device operably coupled to the health equity assessment system (see: Nickerson, paragraph 7 and 23, is met by a first element that, upon initial display of a web page including a particular content, is viewable on the web page on or near the particular content included on the web page and solicits specific user reactions concerning the particular content included on the web page), wherein the CDP rating tool micro-application displays: a first control element that when selected, sends a positive rating of the output of the CDP to the health equity assessment system; a second control element that when selected, sends a negative rating of the output of the CDP to the health equity assessment system; a third control element that when selected, sends an indication that sufficient information is not available to rate the output of the CDP to the health equity assessment system; and a fourth control element that allows a user of the CDP rating tool micro-application to send text data regarding the output of the CDP to the health equity assessment system (see: Nickerson, Fig. 3A, and paragraph 26 and 51, is met by feedback form windows where for instance positive is met by “Yes” and “Love it”, negative is met by “No” and “Hate it”, sufficient information is not available to rate is met by “Not sure”, “feel neutral about it”, “Don’t remember/Not sure”, and sending text data is met by the open-ended comment box)
It would have been obvious to one of ordinary skill at the time the invention was filed to modify the system for testing bias in machine learning models as taught by Cowen to include feedback form windows including “Yes” and “Love it”, “No” and “Hate it”, “Not sure”, “feel neutral about it”, “Don’t remember/Not sure”, and an open-ended comment box as taught by Nickerson with the motivation of measuring and reporting user reactions (see: Nickerson, paragraph 2).
Cowen and Nickerson fail to specifically teach wherein the clinical context is summarized into a structured text file including labeled data in a standardized format such that the labeled data of the structured text file used as input data for the one or more disparity assessment models; however, Amarasingham teaches an AI engine including a natural language generation (NLG) model such as a BERT model, GPT model, GPT-2 model or GPT-3 model, which may generate a summary that summarizes patients' conditions, diagnoses, and treatment as a single summary (see: Amarasingham, paragraph 68).
It would have been obvious to one of ordinary skill at the time the invention was filed to modify the generation of a plurality of outputs as taught by Cowen and Nickerson to include an AI engine including a natural language generation (NLG) model such as a BERT model, GPT model, GPT-2 model or GPT-3 model, which may generate a summary that summarizes patients' conditions, diagnoses, and treatment as a single summary as taught by Amarasingham with the motivation of providing, in an automated way that is not time consuming, a thoughtfully written summary at the time of a shift change, change in level of care or setting of care, or other care transition can reduce the likelihood of communication errors, improve the expediency of care, and reduce stress (see: Amarasingham, paragraph 6).
Cowen, Nickerson, and Amarasingham fail to specifically teach using one or more disparity assessment models; however, Chavda teaches specialized artificial intelligence (A.I.) performance scoring configured to interact with another customer-facing A.I., where the specialized A.I. performance scoring generates an evaluation score reflecting bias related to any topic across a plurality of demographic profiles, such as gender, age, race, nationality, and language nativism (see: Chavda, paragraph 53 and 67-69).
It would have been obvious to one of ordinary skill at the time the invention was filed to modify the identifying a bias as taught by Cowen, Nickerson, and Amarasingham to include specialized artificial intelligence (A.I.) performance scoring configured to interact with another customer-facing A.I., where the specialized A.I. performance scoring generates an evaluation score reflecting bias related to any topic across a plurality of demographic profiles as taught by Chavda with the motivation of identifying and mitigating bias to provide better responses and/or training data, which then feedbacks into increased user experiences to create a positive feedback loop (see: Chavda, paragraph 53).
Cowen, Nickerson, Amarasingham, and Chavda fail to specifically teach storing the confidence score in a database for later retrieval and display within a CDP catalog; however, Sheth teaches data storage may take the form of one or more databases that may maintain model datasets including a memory that stores data such as modeled values, models, and/or any information associated with factors, modeled values, and/or models, such as factor risk, model risk, and/or model drift, where modeled values are any values generated by a model based on the factors (see: Sheth, paragraph 20, 23, 30-31, 41, 50, and 60).
It would have been obvious to one of ordinary skill at the time the invention was filed to modify the memory as taught by Cowen, Nickerson, Amarasingham, and Chavda to include one or more databases that may maintain model datasets including a memory that stores data such as modeled values as taught by Sheth with the motivation of enabling graphical and/or textual illustration in a graphical user interface (see: Sheth, paragraph 28) and/or with the motivation of use in a dynamic monitoring process (see: Sheth, paragraph 50).
As per claim 14, Cowen, Nickerson, Amarasingham, Chavda, and Sheth teach the invention as claimed, see discussion of claim 13, and further teach:
wherein the clinical context includes demographic data with respect to a plurality of disparity factors of a patient population for which the algorithmic bias may be detected, the disparity factors including at least race, gender, age, disability, social determinants of health (SDOH), and socio-economic status (SES) (see: Cowen, paragraph 8, 16-17, 24, 69, 77, and 96, is met by data inputted may include one or more characteristics such as a gender, an age, one or more races or ethnicities, a country of origin, a first language, a second language, personality, well-being, mental health, or a subjective socioeconomic status).
As per claim 15, Cowen, Nickerson, Amarasingham, Chavda, and Sheth teach the invention as claimed, see discussion of claim 14, and further teach:
wherein the confidence score is a multi-dimensional confidence score comprising a plurality of confidence scores for a respective plurality of disparity factors (see: Chavda, Fig. 2A-2B, and paragraph 41, 56, 59, 61, and 64, is met by generated component scores can be utilized to generate an overall score).
It would have been obvious to one of ordinary skill at the time the invention was filed to modify the system for testing bias in machine learning models as taught by Cowen, Nickerson, Amarasingham, Chavda, and Sheth to include generated component scores can be utilized to generate an overall score as taught by Chavda with the motivation of identifying and mitigating bias to provide better responses and/or training data, which then feedbacks into increased user experiences to create a positive feedback loop (see: Chavda, paragraph 53).
As per claim 16, Cowen, Nickerson, Amarasingham, Chavda, and Sheth teach the invention as claimed, see discussion of claim 14, but fail to specifically teach the following limitations met by Amarasingham as cited:
wherein summarizing the clinical context into the text summary further comprises using a large language model (LLM) to generate a natural language text summary of the patient data (see: Amarasingham, paragraph 68, is met by an AI engine including a natural language generation (NLG) model such as a BERT model, GPT model, GPT-2 model or GPT-3 model, which may generate a summary that summarizes patients' conditions, diagnoses, and treatment as a single summary).
It would have been obvious to one of ordinary skill at the time the invention was filed to modify the generation of a plurality of outputs as taught by Cowen, Nickerson, Amarasingham, Chavda, and Sheth to include an AI engine including a natural language generation (NLG) model such as a BERT model, GPT model, GPT-2 model or GPT-3 model, which may generate a summary that summarizes patients' conditions, diagnoses, and treatment as a single summary as taught by Amarasingham with the motivation of providing, in an automated way that is not time consuming, a thoughtfully written summary at the time of a shift change, change in level of care or setting of care, or other care transition can reduce the likelihood of communication errors, improve the expediency of care, and reduce stress (see: Amarasingham, paragraph 6).
Claim(s) 17-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Application Publication 2022/0398411 to Cowen in view of U.S. Patent Application Publication 2008/0033790 to Nickerson in view of U.S. Patent Application Publication 2023/0104655 to Amarasingham in view of U.S. Patent Application Publication to 2020/0137002 to Chavda in view of U.S. Patent Application Publication 2019/0265870 to Sheth further in view of U.S. Patent 12,118,513 to Lu.
As per claim 17, Cowen, Nickerson, Amarasingham, Chavda, and Sheth teach the invention as claimed, see discussion of claim 16, but fail to specifically teach the following limitations met by Lu as cited:
wherein the natural language text summary is included in a listing of the CDP in the CDP catalog displayed to clinicians (see: Lu, Fig. 5D, and column 29, lines 23-30, is met by displaying a list of prompts where there is a description of actions associated with each particular prompt that has been created by a generative AI system).
It would have been obvious to one of ordinary skill at the time the invention was filed to modify the machine learning models in the system for testing bias in machine learning models as taught by Cowen, Nickerson, Amarasingham, Chavda, and Sheth to include being displayed as a list of prompts where there is description of actions associated with each particular prompt that has been created by a generative AI system as taught by Lu with the motivation of allowing a user to assess whether to perform the action associated with the prompt (see: Lu, column 26, lines 29-42).
As per claim 18, Cowen, Nickerson, Amarasingham, Chavda, Sheth, and Lu teach the invention as claimed, see discussion of claim 17, and further teach:
at least a number of patient cases included (see: Cowen, paragraph 112, is met by a total number of participants).
Where the following limitations are met by Lu as cited:
wherein: the listing of the CDP in the CDP catalog is configured to display a preview of the natural language text summary, the preview including in the natural language text summary; and the natural language text summary is displayed in a display panel of a graphical user interface (GUI) of the CDP catalog that can be reached directly by selecting the preview while the health equity assessment system is in an unlaunched state (see: Lu, Fig. 5D, column 29, lines 23-30, is met by displaying a list of prompts where there is a description of actions associated with each particular prompt that has been created by a generative AI system, where the text describing the actions is a preview displayed in response to an input on the particular prompt).
It would have been obvious to one of ordinary skill at the time the invention was filed to modify the system for testing bias in machine learning models including a total number of participants as taught by Cowen, Nickerson, Amarasingham, Chavda, Sheth, and Lu to include being displayed as a list of prompts where there is description of actions associated with each particular prompt that has been created by a generative AI system, where the text describing the actions is a preview displayed in response to an input on the particular prompt, as taught by Lu with the motivation of allowing a user to assess whether to perform the action associated with the prompt (see: Lu, column 26, lines 29-42).
Claim(s) 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Application Publication 2022/0398411 to Cowen in view of U.S. Patent Application Publication 2008/0033790 to Nickerson.
As per claim 19, Cowen teaches a computing device comprising a processor and memory storing instructions that, when executed by the processor, cause the processor to (see: Cowen, Fig. 1, Fig. 18, and paragraph 55, 65, and 137-139, is met by one or more electronic devices implementing a software platform, including a server, client devices, and memory):
deploy a clinical data product (CDP) on patient data (see: Cowen, paragraph 59, 63, 70, 80, 94, and 96, is met by a machine-learning model can be used to annotate the recorded responses and prompt the participant for particular responses);
receive an output of the CDP (see: Cowen, paragraph 59, 63, 70, 80, and 94, is met by receiving recorded response data from a participant);
rating tool configured to: receive clinical feedback from the clinician (see: Cowen, paragraph 8-9, 16-17, 24, 68-69, 77, and 96, is met by survey responses and displaying a prompt and inputting data corresponding to a plurality of demographic groups into the trained machine-learning model to generate a plurality of outputs, where the model may be used to create a diagnosis) regarding an algorithmic bias of the CDP (see: Cowen, paragraph 8-9, 16-17, 24, 69, 77, and 96, is met by displaying a prompt and inputting data corresponding to a plurality of demographic groups into the trained machine-learning model, where such data inputted may include one or more characteristics such as a gender, an age, one or more races or ethnicities, a country of origin, a first language, a second language, personality, well-being, mental health, or a subjective socioeconomic status); and
send the clinical feedback to a health equity assessment system configured to assess the algorithmic bias of the CDP (see: Cowen, paragraph 8, 16-17, 24, 64, 68, 87, 96, and 106, is met by transferring the recorded data and survey data to a server, and identifying a bias of the trained machine-learning model based on the plurality of outputs, where to test for bias in a trained machine-learning model, the model may be evaluated on data from participants of differing demographic groups to determine the differential effects within each group on the predictions of the model).
Though Cowen teaches application software can be deployed in different configurations, such as in a client/server arrangement or through a Web browser as a Web-based application or Web service (see: Cowen, Fig. 1, Fig. 18, and paragraph 55-56, 65, and 137-142, is met by), Cowen fails to specifically teach the following limitations met by Nickerson as cited:
a clinical data product (CDP) rating tool micro-application comprising a rating tool including control elements displayed with the output of the CDP (see: Nickerson, paragraph 7 and 23, is met by a first element that, upon initial display of a web page including a particular content, is viewable on the web page on or near the particular content included on the web page and solicits specific user reactions concerning the particular content included on the web page),
wherein the control elements of the rating tool include: a first control element that when selected, causes the rating tool to send a positive rating of the output of the CDP to the health equity assessment system; a second control element that when selected, causes the rating tool to send a negative rating of the output of the CDP to the health equity assessment system; a third control element that when selected, causes the rating tool to send an indication that sufficient information is not available to rate the output of the CDP to the health equity assessment system; and a fourth control element that allows a user of the rating tool to send text data regarding the output of the CDP to the health equity assessment system (see: Nickerson, Fig. 3A, and paragraph 26 and 51, is met by feedback form windows where for instance positive is met by “Yes” and “Love it”, negative is met by “No” and “Hate it”, sufficient information is not available to rate is met by “Not sure”, “feel neutral about it”, “Don’t remember/Not sure”, and sending text data is met by the open-ended comment box)
It would have been obvious to one of ordinary skill at the time the invention was filed to modify the system for testing bias in machine learning models as taught by Cowen to include feedback form windows including “Yes” and “Love it”, “No” and “Hate it”, “Not sure”, “feel neutral about it”, “Don’t remember/Not sure”, and an open-ended comment box as taught by Nickerson with the motivation of measuring and reporting user reactions (see: Nickerson, paragraph 2).
Novelty of Claims
As per claims 1-6 and 9-12, the closest prior art of record - U.S. Patent Application Publication 2022/0398411 to Cowen, U.S. Patent Application Publication to 2020/0137002 to Chavda, and U.S. Patent Application Publication 2019/0265870 to Sheth - neither alone nor in combination teach the invention of intendent claim 1 as they do not teach, in combination with the other claimed, “collect clinical feedback from a clinician of an output of the CDP generated during the analysis, the clinical feedback collected using a CDP rating tool micro-application of the clinician computing device; send the clinical feedback to the health equity assessment system;…wherein the CDP rating tool micro-application displays: a first control element that when selected, sends a positive rating of the output of the CDP to the health equity assessment system; a second control element that when selected, sends a negative rating of the output of the CDP to the health equity assessment system; a third control element that when selected, sends an indication that sufficient information is not available to rate the output of the CDP to the health equity assessment system; and a fourth control element that allows a user of the CDP rating tool micro-application to send text data regarding the output of the CDP to the health equity assessment system” such that the performance of a health equity assessment of the CDP is “based on the summarized clinical context and the clinical feedback” so that a confidence score is calculated “indicating a degree of confidence that a subsequent analysis of a subsequent patient using the CDP does not suffer from algorithmic bias”; therefore, the closest prior art of record does not anticipate or otherwise render the claimed invention obvious. Rejection of the claimed combination of limitations would require an unreasonable combination of the available prior art.
Response to Arguments
Applicant’s arguments from the response filed on 02/24/2026 have been fully considered and will be addressed below in the order in which they appeared.
In the remarks, Applicant argues in substance that (1) the 35 U.S.C. 101 rejection should be withdrawn in view of the amendments because, “[w]hether or not amended claim 1 includes any abstract ideas, claim 1 is not "directed to" an abstract idea but rather is directed to a system for assessing, addressing, and mitigating algorithmic bias and, thus, provides a technical solution to a technical problem…The claimed configuration addresses these issues with a two-pronged approach that includes a health equity assessment system and a CDP rating tool micro-application. The health equity assessment system is configured to calculate a confidence rating of a given CDP that reflects a confidence that the CDP does not suffer from algorithmic bias. The confidence rating may be determined based on clinician feedback while using the CDP to analyze a patient, and from output of one or more disparity assessment models that utilize labeled data from a structured text file that summarizes a clinical context of the patient/CDP analysis. The CDP rating tool micro-application enables clinicians to provide simple, uniform feedback about performance of the CDP that may then be used by the health equity assessment system to calculate the confidence rating…Accordingly, amended claim 1 recites a specific improvement to the functioning of healthcare computing systems and thus incorporates any alleged abstract idea into a practical application, as set forth in MPEP 2106.04(d). In particular, amended claim 1 recites that the clinical context is summarized into a structured text file including labeled data in a standardized format, and that this labeled data is used as input data for the one or more disparity assessment models…The micro-application and its control elements are not directed to merely data gathering. Instead, the control elements define a specific, structured interface through which clinicians provide standardized feedback that is tightly integrated into the health equity assessment pipeline. As set forth in MPEP 2106.04(d), a judicial exception may be integrated into a practical application when the additional elements reflect an improvement in the functioning of a computer or an improvement to another technology or technical field…Further, the ordered combination of elements in amended claim 1 amounts to significantly more than any alleged abstract idea. See MPEP 2106.05. The specific combination of (1) performing an analysis using a CDP, (2) collecting clinical feedback through a micro-application with four specifically defined control elements, (3) sending that feedback to a separate health equity assessment system, (4) summarizing clinical context into a structured text file with labeled data in a standardized format, (5) using that labeled data as input to trained disparity assessment models together with the clinical feedback, (6) calculating a confidence score, and (7) storing the confidence score for later retrieval and display within a CDP catalog, represents a specific, unconventional arrangement of steps and components that is not well-understood, routine, or conventional. None of the prior art cited in the Office action, individually or in combination, demonstrates that this particular ordered combination of elements was previously known or conventional in the industry...The configuration of amended claim 6 thereby provides a technical effect of reducing algorithmic bias by informing on when and how the CDP should be retrained, based on the accumulated feedback from clinicians using the CDP. It is to be appreciated that algorithmic bias is not a mental process and that detecting and accounting for algorithmic bias (e.g., by retraining the CDP) cannot be performed mentally. Thus, similar to the claims at issue in DDR Holdings, LLC v. Hotels.com et al., 113 USPQ2d 1097 (Fed. Cir. 2014), amended claim 6 is patent eligible because it does not merely recite the performance of a method known from the pre-computer world along with the requirement to perform it on a computer…”
The Examiner respectfully disagrees. Applicant’s arguments are not persuasive.
The calculation of the confidence rating is representative of the abstract idea and the CDP rating tool micro-application is representative of extra-solution activity concerning mere data gathering. The rating tool merely gathers data for the purpose of calculating the confidence rating. Even if the claims went so far as to performing clinical tests on individuals to obtain input for the calculation, such as has been identified by the courts as insignificant extra-solution activity. See: MPEP 2106.05(g). Further, it is a well settled matter that the concepts of receiving or transmitting data over a network, such as using the Internet to gather data, and storing and retrieving information in memory have been identified by the courts as well-understood, routine, and conventional activities. See: MPEP 2106.05(d)(II). To be clear, the CDP rating tool micro-application concerns technology and could possibly provide a practical application if claimed sufficiently, but as currently claimed at a high level, merely represents extra-solution activity. For example, the rating tool displays four “control element”(s) that send corresponding data “when selected”, which is a well-understood, routine, and conventional function of graphical user interface elements. That the system that receives the assessment selections summarizes it into a clinical context “wherein the clinical context is summarized into a structured text file including labeled data in a standardized format”, is representative of an abstract similarly to how creating an index, and using that index to search for and retrieve data (Int. Ventures v. Erie Indemnity I: ‘434 patent) is considered abstract by the courts. The claims here are not directed to a specific improvement to computer functionality that amount to a practical application. Rather, they are directed to the use of conventional or generic technology in a well-known environment, without any claim that the invention reflects an inventive solution to a technical problem presented by combining the two. In the present case, the claims fail to recite any elements that individually or as an ordered combination transform the identified abstract idea(s) in the rejection into a patent-eligible application of that idea.
A list of limitations is argued to “represent[] a specific, unconventional arrangement of steps and components that is not well-understood, routine, or conventional. None of the prior art cited in the Office action, individually or in combination, demonstrates that this particular ordered combination of elements was previously known or conventional in the industry.” Insofar as it is agreed that claim 1 and its dependent claims represent a novel combination of features, as addressed above (i.e., not regarding independent claims 13 and 19 and any of their dependent claims, which remain rejected under 35 U.S.C. 103), it is noted that judicial exceptions need not be old or long‐prevalent, and that even newly discovered judicial exceptions are still exceptions, despite their novelty. For example, the mathematical formula in Flook, the laws of nature in Mayo, and the isolated DNA in Myriad were all novel, but nonetheless were considered by the Supreme Court to be judicial exceptions because they were “‘basic tools of scientific and technological work’ that lie beyond the domain of patent protection.” The Supreme Court’s cited rationale for considering even “just discovered” judicial exceptions as exceptions stems from the concern that “without this exception, there would be considerable danger that the grant of patents would ‘tie up’ the use of such tools and thereby ‘inhibit future innovation premised upon them.’” The Federal Circuit has also applied this principle, for example, when holding the concept of using advertising as an exchange or currency abstract in Ultramercial, despite the patentee’s arguments that the concept was “new”.
As per the argued technical effect in view of DDR Holdings, it is argued that reducing algorithmic bias if a technical effect achieved “by information on when and how the CDP should be retrained”, but this does not concern the technical aspect of the training itself, but merely a decision concern “when and how” retraining should be performed. Reducing algorithmic bias is an abstract concept dependent upon mental processes as claimed - clinical feedback is collected from a clinician - though this is inconsequential to the rejection, which categorizes the abstract as Certain Methods of Organizing Human Activity where the ability for mental performance is not required. The claim in DDR Holdings was found to amount to significantly more because it comprised additional elements including “1) stor[ing] ‘visually perceptible elements’ corresponding to numerous host Web sites in a database, with each of the host Web sites displaying at least one link associated with a product or service of a third-party merchant, 2) on activation of this link by a Web site visitor, automatically identif[ying] the host, and 3) instruct[ing] an Internet web server of an ‘outsource provider’ to construct and serve to the visitor a new, hybrid Web page that merges content associated with the products of the third-party merchant with the stored ‘visually perceptible elements’ from the identified host Web site.” The present claims are concerned with no such comparable additional elements, but DDR Holdings does provide guidance in the software arts as to the level of detail necessary in additional elements to achieve a practical application, and commiserate amendments would achieve a practical application.
In the remarks, Applicant argues in substance that (2) the 35 U.S.C. 103 rejection should be withdrawn in view of the amendments because, “the cited references, even if combined, fail to disclose at least the amended claim 13 element of "performing a health equity assessment of the CDP, the health equity assessment evaluating the summarized clinical context and the rating using one or more disparity assessment models to assess an algorithmic bias of the CDP, the labeled data of the structured text file used as input data for the one or more disparity assessment models." Thus, the rejections of claim 13 and all claims depending therefrom should be withdrawn. Regarding claim 19, the cited references fail to disclose at least the amended claim 19 element of "deploy a CDP rating tool micro-application comprising a rating tool including control elements displayed with the output of the CDP." Even if Cowen and Kaufman were modified based on Nickerson, the claimed configuration would not be reached because the references provide no teaching or suggestion that the control elements of Nickerson would be displayed with the output of the machine learning model of Cowen, nor would one having ordinary skill in the art be motivated to display the ad feedback form windows of Nickerson with the output of the machine learning model of Cowen. In particular, the Office states on page 27 of the Office action that one having ordinary skill in the art would be motivated to include the feedback form windows of Nickerson in the system of Cowen in order to measure and report user reactions, but the "clinical feedback" of Cowen cited by the Office includes characteristics of the user that is then used to select a media content for training a machine learning model (see paragraph [0017] of Cowen), and Cowen has no teaching or suggestion of how the "user reactions" of Nickerson would be used that would motivate one to add the feedback windows of Nickerson to the system of Cowen.”
The rejections are withdrawn with regard to independent claim 1 and its dependent claims.
With regard to claim independent claims 13 and 19 and their dependent claims, the Examiner respectfully disagrees. Applicant’s arguments are not persuasive.
As per Cowen teaching the performing a health equity assessment of the CDP, the claims are to the broad concept of an assessment that returns algorithmic bias based on clinical context and a rating, but there is no limitation as to manner in which the assessment itself is performed – it is left broad. Hence, Cowen teaches the broad limitation by transferring the recorded data and survey data to a server, and identifying a bias of the trained machine-learning model based on the plurality of outputs, where to test for bias in a trained machine-learning model, the model may be evaluated on data from participants of differing demographic groups to determine the differential effects within each group on the predictions of the model (see: Cowen, paragraph 8, 16-17, 24, 64, 68, 87, 96, and 106).
It is argued that “the references provide no teaching or suggestion that the control elements of Nickerson would be displayed with the output of the machine learning model of Cowen, nor would one having ordinary skill in the art be motivated to display the ad feedback form windows of Nickerson with the output of the machine learning model of Cowen”, and that “and Cowen has no teaching or suggestion of how the "user reactions" of Nickerson would be used that would motivate one to add the feedback windows of Nickerson to the system of Cowen.” Cowen teaches survey responses and displaying a prompt and inputting data, and it is clear that, as argued, that the “user reactions” of Nickerson could be included in Cowen’s collection of data, and as discussed above, there is no limitation as to manner in which the assessment itself is performed using such data – it is left broad. As per the feedback window itself, Cowen teaches application software can be deployed in different configurations, such as in a client/server arrangement or through a Web browser as a Web-based application or Web service (see: Cowen, Fig. 1, Fig. 18, and paragraph 55-56, 65, and 137-142, is met by), and Nickerson teaches a first element that, upon initial display of a web page including a particular content, is viewable on the web page on or near the particular content included on the web page and solicits specific user reactions concerning the particular content included on the web page. It is argued that no teaching of suggestion “would be used that would motivation one to add the feedback windows of Nickerson to the system of Cowen”, but one is provided directly in the rejection: It would have been obvious to one of ordinary skill at the time the invention was filed to modify the system for testing bias in machine learning models as taught by Cowen to include feedback form windows including “Yes” and “Love it”, “No” and “Hate it”, “Not sure”, “feel neutral about it”, “Don’t remember/Not sure”, and an open-ended comment box as taught by Nickerson with the motivation of measuring and reporting user reactions (see: Nickerson, paragraph 2).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ROBERT A SOREY whose telephone number is (571)270-3606. The examiner can normally be reached Monday through Friday, 8am to 5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Fonya Long can be reached at (571) 270-5096. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ROBERT A SOREY/Primary Examiner, Art Unit 3682
Prosecution Insights