DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the
first inventor to file provisions of the AIA .
Response to Amendment
The amendments filed 04/06/2026 have been entered. The status of the claims is as follows:
Claims 1-2,4-6,8-9, 11-12, 14-17 and 19-20 are pending in the application.
Claims 3, 7, 10, 13 and 18 are cancelled.
Claims 1, 11 and 16 are amended.
Response to Arguments
In reference to the Rejection of Claims under 35 U.S.C 101:
Argument for Step 2A, Prong 1: Claims Do Not Recite an Abstract Idea:
Claim 1 as a whole recites a specific technological process operating on a machine learning model using a defined computational methodology, namely QII, to achieve a concrete technical outcome, namely a retrained ML model without bias. This is analogous to the claims in SRI International and Research Corp. Technologies, cited in MPEP section 2106.04(a)(2)(III)(A), which were found not to recite mental processes because the human mind was not equipped to perform the claimed operations. Accordingly, claim 1 does not recite a mental process abstract idea, and the rejection under Step 2A Prong One is improper. Furthermore, the Ex Parte Desjardins decision and the recent USPTO Memorandum dated December 5, 2025, which revised the MPEP to reflect that claims directed to improvements in the functioning of machine learning technology itself are not directed to a judicial exception. The Memorandum expressly states that Examiners are expected to consider existing precedent like Enfish, as discussed in MPEP section 2106, when assessing eligibility under 35 U.S.C. 101, particularly when evaluating claims related to machine learning or artificial intelligence. Claim 1 is directed to a specific improvement in how a machine learning model is trained and operated to address the technological problem of bias in ML model outputs, which, consistent with Ex Parte Desjardins, reflects an improvement in machine learning technology itself and thus is not directed to a judicial exception.
Response:
Applicant’s argument is not persuasive. Although claim 1 recites an ML model and bias reduction, the claim does not recite a specific technological improvement to computer functionality or to the operation of the ML model itself. Instead, the claim broadly performs data analysis steps, including determining model performance, measuring feature influence using Quantitative Input Influence, identifying features causing bias, adjusting and retraining the model, calculating statistical metrics, and displaying the results on a UI. These steps amount to evaluating the information output of the model using mathematical or statistical analysis, rather than improving the computer technology. The recited memory, processors, and UI are used only as generic computer components to carry out the abstract analysis and display information. Therefore, the claim remains directed to a judicial exception and is not integrated into a practical application merely because it is performed in the context of machine learning.
Argument for Step 2A, Prong 2: Alleged Abstract Idea is Integrated into a Practical
Application.
The claim as a whole describes a complete and specific technological workflow: applying QII to measure feature influence within an ML model, using that measurement to identify features causing bias, determining an adjustment to those features, retraining the ML model to eliminate the bias, using the retrained model to generate inferences, and displaying information about the bias and the driving features in a user interface. This workflow is directed to a specific technological improvement to machine learning systems, namely the technical capability to identify the specific features within a machine learning model that cause differential treatment of groups and to produce an improved, bias-free ML model. The specification supports this characterization. The specification expressly discloses that the claimed system and method identify when differences in machine learning model operation occur and provide remedies for such differences. The specification further discloses QII as a specific technical tool for measuring feature influence and describes a complete workflow for bias determination and remediation, including feature adjustment and model retraining. The Office's analysis for Step 2A Prong Two evaluated only individual fragments of the claim for practical application, which is incorrect. Specifically, the Examiner evaluated limitations such as retraining the ML model and utilizing the retrained model in isolation and concluded that these were mere instructions to implement the abstract idea on a computer. The Examiner further evaluated the UI limitation in isolation as mere data outputting. This piecemeal analysis is improper. To determine if the claim recites a practical application, the Office must look at the claim as a whole, not just a section of the claim. Therefore, the section 101 rejection is improper.
Response:
Applicant’s argument is not persuasive. When claim 1 is considered as a whole, the claim does not recite a specific technical improvement to the operation or architecture of a computer or ML system. Rather, the claim broadly recites analyzing model performance between groups, determining feature influence using QII, identifying bias-causing features, adjusting or retraining the model, generating an inference, and displaying bias-related information. These steps merely use mathematical or statistical analysis and generic ML processing to improve the content or fairness of the model’s output, without reciting any particular technical mechanism for how the ML model is improved. The cited UI also only presents the results of the analysis and does not improve the functioning of the computer itself. Accordingly, the claim does not integrate the judicial exception into a practical application, but instead applies the abstract idea using generic computer components and result-oriented ML operations.
Argument for Step 2B: Claims Recite Additional Elements that Amount to
Significantly More than the Judicial Exception:
Even if the claim were found to recite a judicial exception that is not integrated into a
practical application, claim lrecites additional elements that amount to significantly more than the judicial exception. The additional elements of the claim, considered both individually and as an ordered combination, provide significantly more than any recited abstract idea or mathematical concept. The claimed combination of: a memory and one or more computer processors operating together to execute a specific sequence of operations; applying QII to measure feature influence in the ML model; identifying the specific features causing bias; determining an adjustment to those features; retraining the ML model to obtain a retrained model without the bias; utilizing the retrained model to make inferences; and providing a user interface with specific information about the bias and the features driving it, constitutes an unconventional and specific technical solution to the technological problem of bias in machine learning models. As explained in the Memorandum of December 5, 2025, consistent with Ex Parte Desjardins, Examiners should not dismiss additional elements as mere generic computer components without considering whether such elements confer a technological improvement to a technical problem, especially as to improvements to computer components or the computer system. The retraining limitation and the inference limitation are not mere instructions to apply an abstract idea on a computer; they represent the concrete technical steps by which the machine learning model itself is improved, which is the hallmark of patent-eligible subject matter under Enfish and its progeny. The ordered combination of all of the claimed elements thus amounts to significantly more than any recited judicial exception.
Response:
Applicant’s argument is not persuasive. The additional elements, including the memory, processor, and user interface, are generic computer components used to perform the abstract idea. The steps of applying QII, identifying biased features, adjusting and retraining the model, and presenting bias information are part of the abstract analysis or merely apply it on a computer. Therefore, the combination does not amount to significantly more than the judicial exception.
In reference to Rejections of Claims Under 35 U.S.C 103:
Argument 1: The cited art does not teach determining bias by the ML model based on a difference of performance between the first group and the second group, ... the determining of the bias further comprising determining that a difference in performance of the first group and the second group exceeds a predetermined threshold
Response 1:
Applicant’s argument is not persuasive. Paragraphs ¶[0008 - 0009] in Bhide teach determining bias based on group-level prediction or fairness metrics and applying constraints on group bias. Specifically, ¶[0008] explains that the post-processing method changes predicted labels to satisfy a group fairness requirement, and ¶[0009] further teaches training a bias detector to detect a sample having bias greater than a predetermined individual bias threshold value “with constraints on a group bias”. Under the broadest reasonable interpretation, the disclosed group bias constraint reasonably encompasses determining whether a difference in performance/ fairness between 2 groups exceeds an acceptable threshold. Thus, the reference teaches determining bias by the ML model based on a performance difference between groups and determining that the difference exceeds a threshold.
Argument 2: The cited art does not teach identifying one or more features of the ML model that cause the bias by the ML model between the first group and the second group
Response 2:
Applicant’s argument is not persuasive. Bhide teaches the disputed concept under the broadest reasonable interpretation. Paragraph ¶[0045] explains that the method creates a model that identifies new samples likely to have individual bias and alters those samples to satisfy group fairness metric requirements. Paragraph ¶[0047] further teaches testing samples from the unprivileged group for individual bias and determining the outcome the sample would have received if it were in the favorable class, while leaving other samples unchanged. Paragraph ¶[0048] also teaches that the individual bias check is generalized to the entire feature space, and that the model scores a new sample and predicts whether it will have individual bias before the post-processing algorithm alters the score to achieve both individual and group fairness metrics. Thus, Bhide identifies the relevant feature-space characteristics or group attribute relationship that causes biased treatment between the first and second groups, which teaches identifying features of the ML model that cause the bias under BRI.
Argument 3: The cited art does not teach retraining the ML model based on the adjustment to the one or more features causing the bias in the ML model to obtain a retrained ML model without the bias
Response 3:
Applicant’s argument is not persuasive. Lehr teaches that when one or more bias thresholds are exceeded, the process proceeds to a bias removal process. Specifically, paragraph ¶[0124] discloses that if any bias threshold is exceeded, the process proceeds to bias removal process. Paragraph ¶[0141] further explains that the intermediary model can be used to initialize and train a new predictive model that excludes the one or more top-ranked, bias-contributing variables, and that the new predictive model is analyzed for validity and bias until a model is produced that does not violate the validity or bias thresholds. Thus, under the broadest reasonable interpretation, Lehr teaches retraining the ML model based on the adjustments to the features causing the bias, because the model is retrained after removing or excluding the bias-contributing variables.
Argument 4: The cited art does not teach providing a user interface (UI) comprising a display of information about the bias between the first group and the second group, the one or more features causing the bias, and the one or more statistical metrics
Response 4:
Applicant’s argument is not persuasive. Cabrera teaches providing a user interface that displays information about bias between groups, the features associated with the bias, and relevant statistical metrics. In particular, Figure 6 and Section 5.5 in Cabrera discloses a “Detailed Comparison View” that allows a user to compare a pinned subgroup and a hovered subgroup, including their performance metrics such as accuracy, precision, and recall, as well as label balance. Cabrera further explains that the interface compares the defining features and values of the subgroups and provides insight into the cause of performance differences and potential bias. Thus, under the broadest reasonable interpretation, Cabrera teaches a UI comprising a display of information about bias between first and second groups, features causing the bias and statistical metrics.
Argument 5: The cited art does not teach that the adjustment comprises modifying how the one or more features are bucketed by the ML model
Response 5:
Applicant’s argument is not persuasive. Lehr teaches the claimed adjustment under the broadest reasonable interpretation. Specifically, paragraph ¶[0163] discloses identifying top-ranked variables of each initialized predictive model and initializing an intermediary predictive model that excludes the identified top-ranked variables. Lehr further explains that predictive outcomes are generated by executing the intermediary predictive model with “two or more groupings” and that the model may be duplicated into “one predictive model per grouping”. Thus, Lehr teaches adjusting the model based on the identified bias-related variables by changing the variables or features included in the model and how those variables are applied across different groupings, which corresponds to modifying how the features are bucketed or grouped by the model.
Argument 6: The cited art does not teach that the UI comprises a chart for group disparity metrics showing a Difference in Means (DM) as a difference between the means of ML model scores for the first group and the second group.
Response 6:
Applicant’s argument is not persuasive. Cabrera’s Figure 4 teaches a UI chart in the form of a Subgroup Overview having strip plots for selected fairness or performance metrics, where different subgroups are displayed and compared against each other. Cabrera expressly shows that, for each metrics, the average across instances is displayed, and the pinned and hovered grouped are separately shown in red and blue. Thus, the chart allows the users to compare the means or average metric values for the first and second groups, with the difference between the plotted group values representing the group disparity or difference in means. Accordingly, under the BRI, Cabrera teaches the disputed limitation.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-2,4-6,8-12,14-17 and 19-20 are rejected under U.S.C 101 for containing an abstract idea without significantly more.
Regarding claim 1:
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
Yes, the claim is a process.
Step 2A – Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
Yes, the claim recites an abstract idea.
determining a performance of a machine learning (ML) model for a first group and a second group; - This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
determining bias by the ML model based on a difference of performance between the first group and the second group - This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
the determining of the bias comprising determining feature influence in the ML model using Quantitative Input Influence (QII) that measures a degree of influence that each input feature exerts on outputs of the ML model - This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
the determining the bias further comprising determining that a difference in performance of the first group and the second group exceeds a predetermined threshold; - This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
identifying one or more features of the ML model that cause the bias by the ML model between the first group and the second group; - This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
determining an adjustment to the identified one or more features that cause the bias between the first group and the second group; This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
calculating one or more statistical metrics associated with the bias and the one or more features causing the bias This limitation is directed to mathematical calculation (see MPEP 2106.04(a)(2) l. C.)
Step 2A – Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
No, there are no additional elements that integrate the judicial exception into a practical application. The additional elements:
a memory comprising instructions This limitation is directed to a computer merely used as a tool to perform an existing process (see MPEP 2106.05(f) (2)).
one or more computer processors, wherein the instructions, when executed by the one or more computer processors, cause the system to perform operations comprising This limitation is directed to a computer merely used as a tool to perform an existing process (see MPEP 2106.05(f) (2)).
retraining the ML model based on the adjustment to the one or more features causing the bias in the ML model to obtain a retrained ML model without the bias; Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
utilizing the retrained model to make an inference without the bias between the first group and the second group; Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
providing a user interface (UI) comprising a display of information about the bias between the first group and the second group, the one or more features causing the bias, and the one or more statistical metrics. This limitation is directed to insignificant extra-solution activity – mere data outputting (see MPEP 2106.05(g)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
No, there are no additional elements that amount to significantly more than the judicial exception. The additional elements are:
a memory comprising instructions This limitation is directed to a computer merely used as a tool to perform an existing process (see MPEP 2106.05(f) (2)).
one or more computer processors, wherein the instructions, when executed by the one or more computer processors, cause the system to perform operations comprising This limitation is directed to a computer merely used as a tool to perform an existing process (see MPEP 2106.05(f) (2)).
retraining the ML model based on the adjustment to the one or more features causing the bias in the ML model to obtain a retrained ML model without the bias; Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
utilizing the retrained model to make an inference without the bias between the first group and the second group; Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
providing a user interface (UI) comprising a display of information about the bias between the first group and the second group, the one or more features causing the bias, and the one or more statistical metrics. This limitation is directed to receiving or transmitting data over a network. The courts have recognized receiving or transmitting data over a network as well understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity (see MPEP 2106.05(d) II.).
Regarding claim 2,
Claim 2 is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which includes an abstract idea (see rejection for claim 1). The additional limitations:
discarding the one or more features from the ML model; and - This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
Regarding claim 4,
Claim 4 is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which includes an abstract idea (see rejection for claim 1). The additional limitations:
wherein the adjustment comprises modifying how the one or more features are bucketed by the ML model. - This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.) because the limitation involves adjusting features by bucketizing the feature values differently, which also means grouping the feature values into bins or buckets differently.
Regarding claim 5,
Claim 5 is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which includes an abstract idea (see rejection for claim 1). The additional limitations:
determining that the bias is caused by bias in training data of the ML model; - This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
updating the training data to eliminate the bias; and Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
retraining the ML model with the updated training data. Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
Regarding claim 6,
Claim 6 is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which includes an abstract idea (see rejection for claim 1). The additional limitations:
wherein the first group comprises a protected group and the second group comprises a complement group of the first group. – This claim merely recites a further limitation on the determining a performance of a machine learning (ML) model for a first group and a second group from claim 1, which is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
Regarding claim 8,
Claim 8 is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which includes an abstract idea (see rejection for claim 1). The additional limitations:
wherein the UI comprises a chart for group disparity metrics showing a Difference in Means (DM) as a difference between the means of ML model scores for the first group and the second group. This claim merely recites a further limitation on the providing a user interface (UI) comprising a display of information about the bias, the one or more features, and the one or more statistical metrics from claim 1, which is directed to insignificant extra-solution activity – mere data outputting (see MPEP 2106.05(g)) under step 2A Prong 2, and is directed to receiving or transmitting data over a network. The courts have recognized receiving or transmitting data over a network as well understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity (see MPEP 2106.05(d) II.) under Step 2B.
Regarding claim 9,
Claim 9 is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which includes an abstract idea (see rejection for claim 1). The additional limitations:
wherein the UI comprises a table for a plurality of features causing bias and an influence of each feature on the first group and the second group. This claim merely recites a further limitation on the providing a user interface (UI) comprising a display of information about the bias, the one or more features, and the one or more statistical metrics from claim 1, which is directed to insignificant extra-solution activity – mere data outputting (see MPEP 2106.05(g)) under Step 2A Prong 2, and is directed to receiving or transmitting data over a network. The courts have recognized receiving or transmitting data over a network as well understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity (see MPEP 2106.05(d) II.) under Step 2B.
Regarding claim 11:
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
Yes, the claim is a process.
Step 2A – Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
Yes, the claim recites an abstract idea.
determining a performance of a machine learning (ML) model for a first group and a second group; - This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
determining bias by the ML model based on a difference of performance between the first group and the second group - This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
the determining of the bias comprising determining feature influence in the ML model using Quantitative Input Influence (QII) that measures a degree of influence that each input feature exerts on outputs of the ML model - This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
the determining of the bias further comprising determining that a difference in performance of the first group and the second group exceeds a predetermined threshold; - This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
identifying one or more features of the ML model that cause the bias by the ML model between the first group and the second group; - This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
determining an adjustment to the identified one or more features that cause the bias between the first group and the second group; - This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
calculating one or more statistical metrics associated with the bias and the one or more features causing the bias This limitation is directed to mathematical calculation (see MPEP 2106.04(a)(2) l. C.)
Step 2A – Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
No, there are no additional elements that integrate the judicial exception into a practical application. The additional elements:
a memory comprising instructions This limitation is directed to a computer merely used as a tool to perform an existing process (see MPEP 2106.05(f) (2)).
one or more computer processors, wherein the instructions, when executed by the one or more computer processors, cause the system to perform operations comprising This limitation is directed to a computer merely used as a tool to perform an existing process (see MPEP 2106.05(f) (2)).
retraining the ML model based on the adjustment to the one or more features causing the bias in the ML model to obtain a retrained ML model without the bias; Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
utilizing the retrained model to make an inference without the bias between the first group and the second group; Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
providing a user interface (UI) comprising a display of information about the bias between the first group and the second group, the one or more features causing the bias, and the one or more statistical metrics. This limitation is directed to insignificant extra-solution activity – mere data outputting (see MPEP 2106.05(g)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
No, there are no additional elements that amount to significantly more than the judicial exception. The additional elements are:
a memory comprising instructions This limitation is directed to a computer merely used as a tool to perform an existing process (see MPEP 2106.05(f) (2)).
one or more computer processors, wherein the instructions, when executed by the one or more computer processors, cause the system to perform operations comprising This limitation is directed to a computer merely used as a tool to perform an existing process (see MPEP 2106.05(f) (2)).
retraining the ML model based on the adjustment to the one or more features causing the bias in the ML model to obtain a retrained ML model without the bias; Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
utilizing the retrained model to make an inference without the bias between the first group and the second group; Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
providing a user interface (UI) comprising a display of information about the bias between the first group and the second group, the one or more features causing the bias, and the one or more statistical metrics This limitation is directed to receiving or transmitting data over a network. The courts have recognized receiving or transmitting data over a network as well understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity (see MPEP 2106.05(d) II.).
Regarding claim 12,
Claim 12 is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 11 which includes an abstract idea (see rejection for claim 11). The additional limitations:
discarding the one or more features from the ML model; and - This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
Regarding claim 14,
Claim 14 is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 11 which includes an abstract idea (see rejection for claim 11). The additional limitations:
wherein the adjustment comprises modifying how the one or more features are bucketed by the ML model. - This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.) because the limitation involves adjusting features by bucketizing the feature values differently, which also means grouping the feature values into bins or buckets differently.
Regarding claim 15,
Claim 15 is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 11 which includes an abstract idea (see rejection for claim 11). The additional limitations:
determining that the bias is caused by bias in training data of the ML model; - This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
updating the training data to eliminate the bias; and Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
retraining the ML model with the updated training data. Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
Regarding claim 16:
Step 1 – Is the claim to a process, machine, manufacture or composition of matter?
Yes, the claim is a process.
Step 2A – Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
Yes, the claim recites an abstract idea.
determining a performance of a machine learning (ML) model for a first group and a second group; - This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
determining bias by the ML model based on a difference of performance between the first group and the second group - This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
the determining of the bias comprising determining feature influence in the ML model using Quantitative Input Influence (QII) that measures a degree of influence that each input feature exerts on outputs of the ML model - This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
the determining of the bias further comprising determining that a difference in performance of the first group and the second group exceeds a predetermined threshold; - This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
identifying one or more features of the ML model that cause the bias by the ML model between the first group and the second group; - This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
determining an adjustment to the identified one or more features that cause the bias between the first group and the second group; - This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
calculating one or more statistical metrics associated with the bias and the one or more features causing the bias This limitation is directed to mathematical calculation (see MPEP 2106.04(a)(2) l. C.)
Step 2A – Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
No, there are no additional elements that integrate the judicial exception into a practical application. The additional elements:
A non-transitory machine-readable storage medium including instructions that,
when executed by a machine, cause the machine to perform operations comprising This limitation is directed to a computer merely used as a tool to perform an existing process (see MPEP 2106.05(f) (2)).
retraining the ML model based on the adjustment to the one or more features causing the bias in the ML model to obtain a retrained ML model without the bias; Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
utilizing the retrained model to make an inference without the bias between the first group and the second group; Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
providing a user interface (UI) comprising a display of information about the bias between the first group and the second group, the one or more features causing the bias, and the one or more statistical metrics. This limitation is directed to insignificant extra-solution activity – mere data outputting (see MPEP 2106.05(g)).
Step 2B – Does the claim recite additional elements that amount to significantly more than the judicial exception?
No, there are no additional elements that amount to significantly more than the judicial exception. The additional elements are:
A non-transitory machine-readable storage medium including instructions that,
when executed by a machine, cause the machine to perform operations comprising This limitation is directed to a computer merely used as a tool to perform an existing process (see MPEP 2106.05(f) (2)).
retraining the ML model based on the adjustment to the one or more features causing the bias in the ML model to obtain a retrained ML model without the bias; Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
utilizing the retrained model to make an inference without the bias between the first group and the second group Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
providing a user interface (UI) comprising a display of information about the bias between the first group and the second group, the one or more features causing the bias, and the one or more statistical metrics. This limitation is directed to receiving or transmitting data over a network. The courts have recognized receiving or transmitting data over a network as well understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity (see MPEP 2106.05(d) II.).
Regarding claim 17,
Claim 17 is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 16 which includes an abstract idea (see rejection for claim 16). The additional limitations:
discarding the one or more features from the ML model; and - This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
Regarding claim 19,
Claim 19 is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 16 which includes an abstract idea (see rejection for claim 16). The additional limitations:
wherein the adjustment comprises modifying how the one or more features are bucketed by the ML model. - This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.) because the limitation involves adjusting features by bucketizing the feature values differently, which also means grouping the feature values into bins or buckets differently.
Regarding claim 20,
Claim 20 is rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 16 which includes an abstract idea (see rejection for claim 16). The additional limitations:
determining that the bias is caused by bias in training data of the ML model; - This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed in the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) Ill. C.)
updating the training data to eliminate the bias; and Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
retraining the ML model with the updated training data. Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea [see MPEP 2106.05(f)] and therefore fails to integrate the exception into a practical application.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-2, 4-6, 8-9, 11- 12, 14-17 and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Bhide et al (US 2020/0184350 A1) (hereafter referred to as “Bhide”) in view of Lehr et al (US 2020/0160180 A1) (hereafter referred to as “Lehr”), Cabrera et al. (“FAIRVIS: Visual Analytics for Discovering Intersectional Bias in Machine Learning”) (hereafter referred to as “Cabrera”) and further in view of Datta et al. (“Algorithmic Transparency via Quantitative Input Influence: Theory and Experiments with Learning Systems”) (hereafter referred to as “Datta”)
Regarding claim 1, Bhide explicitly discloses:
A system comprising: a memory comprising instructions; and (Bhide, ¶[0103]: “The computer program product may include a computer readable storage medium ( or media) having computer readable program instructions thereon for causing a processor to carry, out aspects of the present invention.”)
one or more computer processors, wherein the instructions, when executed by the one or more computer processors, cause the system to perform operations comprising: (Bhide, ¶[0103]: “The computer program product may include a computer readable storage medium ( or media) having computer readable program instructions thereon for causing a processor to carry, out aspects of the present invention.”)
determining bias by the ML model based on a difference of performance between the first group and the second group, wherein determining the bias further comprises determining that a difference in performance of the first group and the second group exceeds a predetermined threshold; (Bhide, ¶[0008]: “The starting point for the inventive approach that solves problems in the art is an individual bias detector, which finds samples whose model prediction changes when the protected attributes change, leaving all other features constant.”, ¶[0009]: “In an exemplary embodiment, the present invention can provide a post-processing computer-implemented method for post-hoc improvement of instance-level and group-level prediction metrics, the post-processing method including training a bias detector that learns to detect a sample that has an individual bias greater than a predetermined individual bias threshold value with constraints on a group bias, applying the bias detector on a run-time sample to select a biased sample in the run-time sample having a bias greater than the predetermined individual bias threshold bias value, and suggesting a de-biased prediction for the biased sample.”) [Examiner’s note: ¶[0008] describes a system that detects individual bias in samples by observing how predictions change when protected attributes (such as race or gender) are altered while keeping other features constant. This process identifies instances where bias may arise, contributing to understanding disparities in performance between groups. ¶[0009] discloses a bias detector that identifies samples with bias exceeding a predefined threshold. This threshold provides a concrete measure for determining when the difference in performance between groups qualifies as bias.]
identifying one or more features of the ML model that cause the bias by the ML model between the first group and the second group; (Bhide, ¶[0045]: “This generalizes from a computationally expensive individual bias checker to create a model that identifies new samples that likely have individual bias and to alter these samples first to achieve group fairness metric requirements.”, ¶[0047]: “Each sample from the unprivileged group (di=0) is tested for individual bias and if it is likely to have individual bias (i.e., bi=1), then this sample is assigned the outcome it would have received if it were in the favorable class, (i.e., yi=y(xk, 1)”, ¶[0048]: “At periodic intervals, an individual bias check is conducted on test samples and generalized to the entire feature space. When a new unlabeled sample comes in, the model scores it and the generalized individual bias checker predicts whether it will have individual bias.”) [Examiner’s note: The highlights discloses how an individual bias check is conducted on test samples and generalized to an entire feature space, which illustrates that specific features contributing to bias are identified during this process since the bias detector predicts whether new samples will have individual bias]
determining an adjustment to the identified one or more features that cause the bias between the first group and the second group; (Bhide, ¶[0050]: “The de-biased predictions 350 is performed in post-processing by de-biasing procedure for each sample point by perturbing the protected attribute(s) in a training set (e.g., in 303 of FIG. 3), run the perturbed examples through customer model 320, and picking the most likely prediction for the perturbed data as suggested values to modify.”, ¶[0051]: “Thereby, samples predicted to have highest individual biases (among the 'unprivileged group') by the detector are prioritized for correction, suggested correction involves running perturbed examples through the customer model and picking the most likely prediction, and an arbiter can decide whether to choose the original or the suggested de-biased prediction.”) [Examiner’s note: The de-biasing process involves systematically altering (or “perturbing”) the features associated with protected attributes and running them through the model. By running perturbed examples through the model and selecting the “most likely prediction” for the perturbed data, the process effectively determines the adjustments needed to address the bias in the model’s outputs]
calculating one or more statistical metrics associated with the bias and the one or more features causing the bias (Bhide, ¶[0057]: “During the training stage of the bias detector, the invention implements an individual bias Checker that perturbs the protected attribute in the payload data for the unprivileged group samples, and computes the individual bias scores for them by finding the difference between the probability of favorable outcomes for the perturbed and the original data.”) [Examiner’s note: The highlight illustrates the process of calculating a statistical metric (the probability difference) that quantifies the bias associated with the protected feature.]
Bhide fails to disclose:
determining a performance of a machine learning (ML) model for a first group and a second group;
retraining the ML model based on the adjustment to the one or more features causing the bias in the ML model to obtain a retrained ML model without the bias;
utilizing the retrained model to make an inference without the bias between the first group and the second group;
the determining of the bias comprising determining feature influence in the ML model using Quantitative Input Influence (QII) that measures a degree of influence that each input feature exerts on outputs of the ML model,
providing a user interface (UI) comprising a display of information about the bias between the first group and the second group, the one or more features causing the bias, and the one or more statistical metrics
However, Cabrera explicitly discloses:
determining a performance of a machine learning (ML) model for a first group and a second group; (Cabrera, Page 50, Figure 4: “In the Subgroup Overview users can see how different subgroups compare to one another according to various performance metrics. As more metrics are selected at the top, additional strip plots are added to the interface. Here, a user has pinned the Female subgroup and hovers over the Male subgroup.
PNG
media_image1.png
316
851
media_image1.png
Greyscale
”, Page 50, Col. 1, ¶2]: “When a user clicks the “Generate Subgroups” button (Fig. 3), FAIRVIS splits the data into the specified subgroups and calculates various performance metrics for them. These groups are then represented in the multiple strip plots as lines corresponding to their performance for the respective metric.”, Page 50, Col. 1, ¶[5]: “In total, users can select from the following metrics: Accuracy, Recall, Specificity, Precision, Negative Predictive Value, False Negative Rate, False Positive Rate, False Discovery Rate, False Omission Rate, and F1 score. These metrics were selected as they are typically the most common metrics used for evaluating the equity and performance of classification models.”) [Examiner’s note: a first group i.e., female group, a second group i.e., male group, a performance of machine learning model for these groups is determined by the following metrics: accuracy, recall, precision, etc. and shown in figure 4]
providing a user interface (UI) comprising a display of information about the bias between the first group and the second group, the one or more features causing the bias, and the one or more statistical metrics (Cabrera, Page 52, Figure 6:
PNG
media_image2.png
538
442
media_image2.png
Greyscale
, Page 52, Col. 1, Section 5.5, ¶[2-3]: “A user is able to see the details for two groups in the Detailed Comparison View, the pinned and hovered group. A group can be pinned when a user clicks on it in the Subgroup Overview or Suggested and Similar Subgroup View, and is designated by a light red across the UI. The hovered group is designated by a light blue across the UI. These two distinct colors allow users to see a selected group’s information across various different views. There are three primary components in the Detailed Comparison View, as seen in Fig. 6. The topmost component is a bar chart displaying how a group performs for selected performance metrics.”, Col. 1, Section 5.5, ¶[4]: “The second component in the Detailed Comparison View is a bar chart for the ground truth label balance of both selected subgroups. The label imbalance is important because it can often explain extreme values for metrics like recall and precision and can suggest reasons for bias (C6)”) [Examiner’s note: a user interface i.e., the Detailed Comparison View UI, information about the bias i.e., the label imbalance between the statistical metrics (i.e., accuracy, precision, recall), the features i.e., sex, race, marital status, relationship]
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Bhide and Cabrera. Bhide teaches a bias detector used to prioritize data samples in a bias mitigation algorithm aiming to improve the group fairness measure of disparate impact. Cabrera teaches a mixed-initiative visual analytics system that integrates a novel subgroup discovery technique for users to audit the fairness of machine learning models. One of ordinary skill would have motivation to combine Bhide and Cabrera to help data scientists and the general public understand and create more equitable algorithmic systems by the interactive visualization and enable users to audit the fairness of machine learning models (Cabrera, Abstract)
However, Datta explicitly discloses:
the determining of the bias comprising determining feature influence in the ML model using Quantitative Input Influence (QII) that measures a degree of influence that each input feature exerts on outputs of the ML model, (Datta, Pg. 599, Col. 1, ¶[3]: “We formalize transparency reports by introducing a family of Quantitative Input Influence (QII) measures that capture the degree of influence of inputs on outputs of the system.”)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Bhide and Datta. Bhide teaches a bias detector used to prioritize data samples in a bias mitigation algorithm aiming to improve the group fairness measure of disparate impact. Datta teaches a family of Quantitative Input Influence (QII) measures that capture the degree of influence of inputs on outputs of systems. One of ordinary skill would have motivation to combine Bhide and Datta because MPEP 2143 sets forth the Supreme Court rationales for obviousness including: (D) Applying a known technique to a known device (method, or product) ready for improvement to yield predictable results; (E): “Obvious to try” choosing from a finite number of identified, predictable solutions, with a reasonable expectation of success; (F) Known work in one field of endeavor may prompt variations of it for use in either the same field or a different one based on design incentives or other market forces if the variations are predictable to one of the ordinary skill in the art.
However, Lehr explicitly discloses:
retraining the ML model based on the adjustment to the one or more features causing the bias in the ML model to obtain a retrained ML model without the bias; (Lehr, ¶[0108]: “With reference to FIG. 13, shown is a model generation and training process 1300.”, ¶[0124]: “In one or more embodiments, if any of the one or more bias thresholds is exceeded, the process 1300 proceeds to bias removal process 1400”, ¶[0141]: “In various embodiments, when the process 1400 proceeds to process 1300, the intermediary model (e.g., metadata 145 thereof) can be used to initialize and train a new predictive model that excludes the one or more top-ranked, bias-contributing variables. Thus, in at least one embodiment, the predictive model generated based on each variable compartment can be a new, novel treatment of the dataset that we are comparing to the treatment of the primary dataset (e.g., in previous iterations of the predictive model). The new predictive model can be analyzed for validity and bias, and the processes 1300 and 1400 can be repeated as required to produce an iteration of the predictive model that does not violate validity or bias thresholds.”) [Examiner’s note: The highlight explains a process where a predictive model is retrained by modifying its features to address bias. If the intermediary model fails to meet validity or bias thresholds, certain high-ranking features identified as contributing to bias are excluded. Using this adjusted dataset (without the bias contributing features), a new predictive model is trained. The retraining process is repeated iteratively to refine the model until it meets the require thresholds for validity and bias]
utilizing the retrained model to make an inference without the bias between the first group and the second group; (Lehr, ¶[0121]: “At step 1309, the process 1300 includes generating predicted outcomes 136 using the trained, validated, and error threshold-satisfying iteration of the predictive model… The predictive model can generate a set of predicted outcomes 136 based on the input data and the weighted variables”) [Examiner’s note: the final predictive model used for generating predicted outcomes (i.e., making an inference) using the validated and error threshold-satisfying model (i.e., the model excluding bias feature)]
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Bhide and Lehr. Bhide teaches a bias detector used to prioritize data samples in a bias mitigation algorithm aiming to improve the group fairness measure of disparate impact. Lehr teaches systems and processes for bias removal in a predictive performance model. One of ordinary skill would have motivation to combine Bhide and Lehr to promote fairness and equity in training prediction models. Removing bias ensures that the mode’s predictions are fair and do not disproportionately favor or disadvantage any specific group. A bias-free model provides more reliable and accurate predictions, enabling better and more informed decisions based on actual performance rather than systemic or historical disparities. (Lehr, ¶[0115])
Regarding claim 2, the combination of Bhide, Lehr and Cabrera discloses all the limitations of claim 1 (as shown in the rejections above).
Bhide in view of Lehr and Cabrera further discloses:
wherein the adjustment comprises discarding the one or more features from the ML model (Lehr, ¶[0141]: “In various embodiments, when the process 1400 proceeds to process 1300, the intermediary model (e.g., metadata 145 thereof) can be used to initialize and train a new predictive model that excludes the one or more top-ranked, bias-contributing variables.”)
Regarding claim 4, the combination of Bhide, Lehr and Cabrera discloses all the limitations of claim 3 (as shown in the rejections above).
Bhide in view of Lehr and Cabrera further discloses:
wherein the adjustment comprises modifying how the one or more features are bucketed by the ML model. (Lehr, ¶[0163]: “The intermediary predictive model can generate two or more sets of predictive outcomes 136.Each set of predictive outcomes 136 can be generated by executing the intermediary predictive model with one of the two or more groupings (e.g., and the corresponding input dataset thereof). In other words, the intermediary predictive model can be duplicated into two or more additional intermediary predictive models ( e.g., one predictive model per grouping).”) [Examiner’s note: The highlight discloses modifying how features are grouped or bucketed (groupings) to create multiple versions of intermediary predictive models, demonstrating how feature is adjusted]
Regarding claim 5, the combination of Bhide, Lehr and Cabrera discloses all the limitations of claim 1 (as shown in the rejections above).
Bhide in view of Lehr and Cabrera further discloses:
wherein the instructions further cause the system to perform operations comprising: determining that the bias is caused by bias in training data of the ML model; (Bhide, ¶[0008]: “The starting point for the inventive approach that solves problems in the art is an individual bias detector, which finds samples whose model prediction changes when the protected attributes change, leaving all other features constant.”, ¶[0009]: “In an exemplary embodiment, the present invention can provide a post-processing computer-implemented method for post-hoc improvement of instance-level and group-level prediction metrics, the post-processing method including training a bias detector that learns to detect a sample that has an individual bias greater than a predetermined individual bias threshold value with constraints on a group bias, applying the bias detector on a run-time sample to select a biased sample in the run-time sample having a bias greater than the predetermined individual bias threshold bias value, and suggesting a de-biased prediction for the biased sample.”)
updating the training data to eliminate the bias; and (Lehr, ¶[0141]: “In various embodiments, when the process 1400 proceeds to process 1300, the intermediary model (e.g., metadata 145 thereof) can be used to initialize and train a new predictive model that excludes the one or more top-ranked, bias-contributing variables.”)
retraining the ML model with the updated training data. (Lehr, ¶[0108]: “With reference to FIG. 13, shown is a model generation and training process 1300.”, ¶[0124]: “In one or more embodiments, if any of the one or more bias thresholds is exceeded, the process 1300 proceeds to bias removal process 1400”, ¶[0141]: “In various embodiments, when the process 1400 proceeds to process 1300, the intermediary model (e.g., metadata 145 thereof) can be used to initialize and train a new predictive model that excludes the one or more top-ranked, bias-contributing variables. Thus, in at least one embodiment, the predictive model generated based on each variable compartment can be a new, novel treatment of the dataset that we are comparing to the treatment of the primary dataset (e.g., in previous iterations of the predictive model). The new predictive model can be analyzed for validity and bias, and the processes 1300 and 1400 can be repeated as required to produce an iteration of the predictive model that does not violate validity or bias thresholds.”) [Examiner’s note: The highlight explains a process where a predictive model is retrained by modifying its features to address bias. If the intermediary model fails to meet validity or bias thresholds, certain high-ranking features identified as contributing to bias are excluded. Using this adjusted dataset (without the bias contributing features), a new predictive model is trained. The retraining process is repeated iteratively to refine the model until it meets the require thresholds for validity and bias]
Regarding claim 6, the combination of Bhide, Lehr and Cabrera discloses all the limitations of claim 1 (as shown in the rejections above).
Bhide in view of Lehr and Cabrera further discloses:
wherein the first group comprises a protected group and the second group comprises a complement group of the first group. (Lehr, ¶[0156]: “At step 1503, the process 1500 includes defining two or more groupings of protected data based on a protected data category or class. The protected data is associated with input data (e.g., and corresponding subjects, such as users) utilized by an initial predictive model (e.g., a trained, validity and bias threshold-compliant performance model). In one example, the category is sex and the two or more groupings include a female grouping and a male grouping.”) [Examiner’s note: “a protected group” i.e., a female grouping, “a complement group of the first group” i.e., a male grouping]
Regarding claim 8, the combination of Bhide, Lehr and Cabrera discloses all the limitations of claim 1 (as shown in the rejections above).
Bhide in view of Lehr and Cabrera further discloses:
wherein the UT comprises a chart for group disparity metrics showing a Difference in Means (DM) as a difference between the means of ML model scores for the first group and the second group. (Cabrera, Page 50, Figure 4: “In the Subgroup Overview users can see how different subgroups compare to one another according to various performance metrics. As more metrics are selected at the top, additional strip plots are added to the interface. Here, a user has pinned the Female subgroup and hovers over the Male subgroup.”
PNG
media_image3.png
379
1000
media_image3.png
Greyscale
) [Examiner’s note: In Figure 4, the “disparity metrics” showing a Difference in Means can be interpreted as the differences in the performance metrics (e.g., accuracy, precision, recall) between the female group and male group, “Difference in Means” is being interpreted as the difference between average values of the performance metrics of the 2 groups]
Regarding claim 9, the combination of Bhide, Lehr and Cabrera discloses all the limitations of claim 1 (as shown in the rejections above).
Bhide in view of Lehr and Cabrera further discloses:
wherein the UI comprises a table for a plurality of features causing bias and an influence of each feature on the first group and the second group. (Cabrera, Page 47, Col. 2, ¶2]: “Once a subgroup for which a model has poor performance has been identified, it can be useful to look at similar subgroups to compare their values and performance. We use similarity in the form of statistical divergence between feature distributions to find subgroups that are statistically similar. Users can then compare similar groups to discover which value differences impact performance or to form more general subgroups of fewer features.”, Page 52, Figure 5: “Here we can see the Suggested and Similar Subgroup View for both suggested and similar subgroups. Users can hover over any card to see detailed feature and performance information in the Detailed Comparison View.”
PNG
media_image4.png
398
1262
media_image4.png
Greyscale
) [Examiner’s note: Figure 5 discloses the table of multiple features causing bias (e.g., marital-status, sex, relationship, education etc.) and the impact of them to the female and male subgroups]
Regarding claim 11, Bhide explicitly discloses:
determining bias by the ML model based on a difference of performance between the first group and the second group, wherein determining the bias further comprises determining that a difference in performance of the first group and the second group exceeds a predetermined threshold; (Bhide, ¶[0008]: “The starting point for the inventive approach that solves problems in the art is an individual bias detector, which finds samples whose model prediction changes when the protected attributes change, leaving all other features constant.”, ¶[0009]: “In an exemplary embodiment, the present invention can provide a post-processing computer-implemented method for post-hoc improvement of instance-level and group-level prediction metrics, the post-processing method including training a bias detector that learns to detect a sample that has an individual bias greater than a predetermined individual bias threshold value with constraints on a group bias, applying the bias detector on a run-time sample to select a biased sample in the run-time sample having a bias greater than the predetermined individual bias threshold bias value, and suggesting a de-biased prediction for the biased sample.”) [Examiner’s note: ¶[0008] describes a system that detects individual bias in samples by observing how predictions change when protected attributes (such as race or gender) are altered while keeping other features constant. This process identifies instances where bias may arise, contributing to understanding disparities in performance between groups. ¶[0009] discloses a bias detector that identifies samples with bias exceeding a predefined threshold. This threshold provides a concrete measure for determining when the difference in performance between groups qualifies as bias.]
identifying one or more features of the ML model that cause the bias by the ML model between the first group and the second group; (Bhide, ¶[0045]: “This generalizes from a computationally expensive individual bias checker to create a model that identifies new samples that likely have individual bias and to alter these samples first to achieve group fairness metric requirements.”, ¶[0047]: “Each sample from the unprivileged group (di=0) is tested for individual bias and if it is likely to have individual bias (i.e., bi=1), then this sample is assigned the outcome it would have received if it were in the favorable class, (i.e., yi=y(xk, 1)”, ¶[0048]: “At periodic intervals, an individual bias check is conducted on test samples and generalized to the entire feature space. When a new unlabeled sample comes in, the model scores it and the generalized individual bias checker predicts whether it will have individual bias.”) [Examiner’s note: The highlights discloses how an individual bias check is conducted on test samples and generalized to an entire feature space, which illustrates that specific features contributing to bias are identified during this process since the bias detector predicts whether new samples will have individual bias]
determining an adjustment to the identified one or more features that cause the bias between the first group and the second group; (Bhide, ¶[0050]: “The de-biased predictions 350 is performed in post-processing by de-biasing procedure for each sample point by perturbing the protected attribute(s) in a training set (e.g., in 303 of FIG. 3), run the perturbed examples through customer model 320, and picking the most likely prediction for the perturbed data as suggested values to modify.”, ¶[0051]: “Thereby, samples predicted to have highest individual biases (among the 'unprivileged group') by the detector are prioritized for correction, suggested correction involves running perturbed examples through the customer model and picking the most likely prediction, and an arbiter can decide whether to choose the original or the suggested de-biased prediction.”) [Examiner’s note: The de-biasing process involves systematically altering (or “perturbing”) the features associated with protected attributes and running them through the model. By running perturbed examples through the model and selecting the “most likely prediction” for the perturbed data, the process effectively determines the adjustments needed to address the bias in the model’s outputs]
calculating one or more statistical metrics associated with the bias and the one or more features causing the bias (Bhide, ¶[0057]: “During the training stage of the bias detector, the invention implements an individual bias Checker that perturbs the protected attribute in the payload data for the unprivileged group samples, and computes the individual bias scores for them by finding the difference between the probability of favorable outcomes for the perturbed and the original data.”) [Examiner’s note: The highlight illustrates the process of calculating a statistical metric (the probability difference) that quantifies the bias associated with the protected feature.]
Bhide fails to disclose:
determining a performance of a machine learning (ML) model for a first group and a
second group;
the determining of the bias comprising determining feature influence in the ML model using Quantitative Input Influence (QII) that measures a degree of influence that each input feature exerts on outputs of the ML model,
retraining the ML model based on the adjustment to the one or more features causing the bias in the ML model to obtain a retrained ML model without the bias
utilizing the retrained model to make an inference without the bias between the first group and the second group;
providing a user interface (UI) comprising a display of information about the bias between the first group and the second group, the one or more features causing the bias, and the one or more statistical metrics.
However, Cabrera explicitly discloses:
determining a performance of a machine learning (ML) model for a first group and a
second group; (Cabrera, Page 50, Figure 4: “In the Subgroup Overview users can see how different subgroups compare to one another according to various performance metrics. As more metrics are selected at the top, additional strip plots are added to the interface. Here, a user has pinned the Female subgroup and hovers over the Male subgroup.
PNG
media_image1.png
316
851
media_image1.png
Greyscale
”, Page 50, Col. 1, ¶2]: “When a user clicks the “Generate Subgroups” button (Fig. 3), FAIRVIS splits the data into the specified subgroups and calculates various performance metrics for them. These groups are then represented in the multiple strip plots as lines corresponding to their performance for the respective metric.”, Page 50, Col. 1, ¶[5]: “In total, users can select from the following metrics: Accuracy, Recall, Specificity, Precision, Negative Predictive Value, False Negative Rate, False Positive Rate, False Discovery Rate, False Omission Rate, and F1 score. These metrics were selected as they are typically the most common metrics used for evaluating the equity and performance of classification models.”) [Examiner’s note: a first group i.e., female group, a second group i.e., male group, a performance of machine learning model for these groups is determined by the following metrics: accuracy, recall, precision, etc. and shown in figure 4]
providing a user interface (UI) comprising a display of information about the bias between the first group and the second group, the one or more features causing the bias, and the one or more statistical metrics.
(Cabrera, Page 52, Figure 6:
PNG
media_image2.png
538
442
media_image2.png
Greyscale
, Page 52, Col. 1, Section 5.5, ¶[2-3]: “A user is able to see the details for two groups in the Detailed Comparison View, the pinned and hovered group. A group can be pinned when a user clicks on it in the Subgroup Overview or Suggested and Similar Subgroup View, and is designated by a light red across the UI. The hovered group is designated by a light blue across the UI. These two distinct colors allow users to see a selected group’s information across various different views. There are three primary components in the Detailed Comparison View, as seen in Fig. 6. The topmost component is a bar chart displaying how a group performs for selected performance metrics.”, Col. 1, Section 5.5, ¶[4]: “The second component in the Detailed Comparison View is a bar chart for the ground truth label balance of both selected subgroups. The label imbalance is important because it can often explain extreme values for metrics like recall and precision and can suggest reasons for bias (C6)”) [Examiner’s note: a user interface i.e., the Detailed Comparison View UI, information about the bias i.e., the label imbalance between the statistical metrics (i.e., accuracy, precision, recall), the features i.e., sex, race, marital status, relationship]
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Bhide and Cabrera. Bhide teaches a bias detector used to prioritize data samples in a bias mitigation algorithm aiming to improve the group fairness measure of disparate impact. Cabrera teaches a mixed-initiative visual analytics system that integrates a novel subgroup discovery technique for users to audit the fairness of machine learning models. One of ordinary skill would have motivation to combine Bhide and Cabrera to help data scientists and the general public understand and create more equitable algorithmic systems by the interactive visualization and enable users to audit the fairness of machine learning models (Cabrera, Abstract)
However, Datta explicitly discloses:
the determining of the bias comprising determining feature influence in the ML model using Quantitative Input Influence (QII) that measures a degree of influence that each input feature exerts on outputs of the ML model, (Datta, Pg. 599, Col. 1, ¶[3]: “We formalize transparency reports by introducing a family of Quantitative Input Influence (QII) measures that capture the degree of influence of inputs on outputs of the system.”)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Bhide and Datta. Bhide teaches a bias detector used to prioritize data samples in a bias mitigation algorithm aiming to improve the group fairness measure of disparate impact. Datta teaches a family of Quantitative Input Influence (QII) measures that capture the degree of influence of inputs on outputs of systems. One of ordinary skill would have motivation to combine Bhide and Datta because MPEP 2143 sets forth the Supreme Court rationales for obviousness including: (D) Applying a known technique to a known device (method, or product) ready for improvement to yield predictable results; (E): “Obvious to try” choosing from a finite number of identified, predictable solutions, with a reasonable expectation of success; (F) Known work in one field of endeavor may prompt variations of it for use in either the same field or a different one based on design incentives or other market forces if the variations are predictable to one of the ordinary skill in the art.
However, Lehr explicitly discloses:
retraining the ML model based on the adjustment to the one or more features causing the bias in the ML model to obtain a retrained ML model without the bias; (Lehr, ¶[0108]: “With reference to FIG. 13, shown is a model generation and training process 1300.”, ¶[0124]: “In one or more embodiments, if any of the one or more bias thresholds is exceeded, the process 1300 proceeds to bias removal process 1400”, ¶[0141]: “In various embodiments, when the process 1400 proceeds to process 1300, the intermediary model (e.g., metadata 145 thereof) can be used to initialize and train a new predictive model that excludes the one or more top-ranked, bias-contributing variables. Thus, in at least one embodiment, the predictive model generated based on each variable compartment can be a new, novel treatment of the dataset that we are comparing to the treatment of the primary dataset (e.g., in previous iterations of the predictive model). The new predictive model can be analyzed for validity and bias, and the processes 1300 and 1400 can be repeated as required to produce an iteration of the predictive model that does not violate validity or bias thresholds.”) [Examiner’s note: The highlight explains a process where a predictive model is retrained by modifying its features to address bias. If the intermediary model fails to meet validity or bias thresholds, certain high-ranking features identified as contributing to bias are excluded. Using this adjusted dataset (without the bias contributing features), a new predictive model is trained. The retraining process is repeated iteratively to refine the model until it meets the require thresholds for validity and bias]
utilizing the retrained model to make an inference without the bias between the first group and the second group; (Lehr, ¶[0121]: “At step 1309, the process 1300 includes generating predicted outcomes 136 using the trained, validated, and error threshold-satisfying iteration of the predictive model… The predictive model can generate a set of predicted outcomes 136 based on the input data and the weighted variables”) [Examiner’s note: the final predictive model used for generating predicted outcomes (i.e., making an inference) using the validated and error threshold-satisfying model (i.e., the model excluding bias feature)]
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Bhide and Lehr. Bhide teaches a bias detector used to prioritize data samples in a bias mitigation algorithm aiming to improve the group fairness measure of disparate impact. Lehr teaches systems and processes for bias removal in a predictive performance model. One of ordinary skill would have motivation to combine Bhide and Lehr to promote fairness and equity in training prediction models. Removing bias ensures that the mode’s predictions are fair and do not disproportionately favor or disadvantage any specific group. A bias-free model provides more reliable and accurate predictions, enabling better and more informed decisions based on actual performance rather than systemic or historical disparities. (Lehr, ¶[0115])
Regarding claim 12, the combination of Bhide, Lehr and Cabrera discloses all the limitations of Claim 11 (as shown in the rejections above).
Bhide in view of Lehr and Cabrera further discloses:
wherein the adjustment comprises discarding the one or more features from the ML model; (Lehr, ¶[0141]: “In various embodiments, when the process 1400 proceeds to process 1300, the intermediary model (e.g., metadata 145 thereof) can be used to initialize and train a new predictive model that excludes the one or more top-ranked, bias-contributing variables.”)
Regarding claim 14, the combination of Bhide, Lehr and Cabrera discloses all the limitations of Claim 11 (as shown in the rejections above).
Bhide in view of Lehr and Cabrera further discloses:
the adjustment comprises modifying how the one or more features are bucketed by the ML model. (Lehr, ¶[0163]: “The intermediary predictive model can generate two or more sets of predictive outcomes 136.Each set of predictive outcomes 136 can be generated by executing the intermediary predictive model with one of the two or more groupings (e.g., and the corresponding input dataset thereof). In other words, the intermediary predictive model can be duplicated into two or more additional intermediary predictive models ( e.g., one predictive model per grouping).”) [Examiner’s note: The highlight discloses modifying how features are grouped or bucketed (groupings) to create multiple versions of intermediary predictive models, demonstrating how feature is adjusted]
Regarding claim 15, the combination of Bhide, Lehr and Cabrera discloses all the limitations of Claim 11 (as shown in the rejections above).
Bhide in view of Lehr and Cabrera further discloses:
determining that the bias is caused by bias in training data of the ML model; (Bhide, ¶[0008]: “The starting point for the inventive approach that solves problems in the art is an individual bias detector, which finds samples whose model prediction changes when the protected attributes change, leaving all other features constant.”, ¶[0009]: “In an exemplary embodiment, the present invention can provide a post-processing computer-implemented method for post-hoc improvement of instance-level and group-level prediction metrics, the post-processing method including training a bias detector that learns to detect a sample that has an individual bias greater than a predetermined individual bias threshold value with constraints on a group bias, applying the bias detector on a run-time sample to select a biased sample in the run-time sample having a bias greater than the predetermined individual bias threshold bias value, and suggesting a de-biased prediction for the biased sample.”)
updating the training data to eliminate the bias; and (Lehr, ¶[0141]: “In various embodiments, when the process 1400 proceeds to process 1300, the intermediary model (e.g., metadata 145 thereof) can be used to initialize and train a new predictive model that excludes the one or more top-ranked, bias-contributing variables.”)
retraining the ML model with the updated training data. (Lehr, ¶[0108]: “With reference to FIG. 13, shown is a model generation and training process 1300.”, ¶[0124]: “In one or more embodiments, if any of the one or more bias thresholds is exceeded, the process 1300 proceeds to bias removal process 1400”, ¶[0141]: “In various embodiments, when the process 1400 proceeds to process 1300, the intermediary model (e.g., metadata 145 thereof) can be used to initialize and train a new predictive model that excludes the one or more top-ranked, bias-contributing variables. Thus, in at least one embodiment, the predictive model generated based on each variable compartment can be a new, novel treatment of the dataset that we are comparing to the treatment of the primary dataset (e.g., in previous iterations of the predictive model). The new predictive model can be analyzed for validity and bias, and the processes 1300 and 1400 can be repeated as required to produce an iteration of the predictive model that does not violate validity or bias thresholds.”) [Examiner’s note: The highlight explains a process where a predictive model is retrained by modifying its features to address bias. If the intermediary model fails to meet validity or bias thresholds, certain high-ranking features identified as contributing to bias are excluded. Using this adjusted dataset (without the bias contributing features), a new predictive model is trained. The retraining process is repeated iteratively to refine the model until it meets the require thresholds for validity and bias]
Regarding claim 16, Bhide explicitly discloses:
A non-transitory machine-readable storage medium including instructions that, when executed by a machine, cause the machine to perform operations comprising: (Bhide, ¶[0103]: “The computer program product may include a computer readable storage medium ( or media) having computer readable program instructions thereon for causing a processor to carry, out aspects of the present invention.”)
determining bias by the ML model based on a difference of performance between the first group and the second group, wherein determining the bias further comprises determining that a difference in performance of the first group and the second group exceeds a predetermined threshold; (Bhide, ¶[0008]: “The starting point for the inventive approach that solves problems in the art is an individual bias detector, which finds samples whose model prediction changes when the protected attributes change, leaving all other features constant.”, ¶[0009]: “In an exemplary embodiment, the present invention can provide a post-processing computer-implemented method for post-hoc improvement of instance-level and group-level prediction metrics, the post-processing method including training a bias detector that learns to detect a sample that has an individual bias greater than a predetermined individual bias threshold value with constraints on a group bias, applying the bias detector on a run-time sample to select a biased sample in the run-time sample having a bias greater than the predetermined individual bias threshold bias value, and suggesting a de-biased prediction for the biased sample.”) [Examiner’s note: ¶[0008] describes a system that detects individual bias in samples by observing how predictions change when protected attributes (such as race or gender) are altered while keeping other features constant. This process identifies instances where bias may arise, contributing to understanding disparities in performance between groups. ¶[0009] discloses a bias detector that identifies samples with bias exceeding a predefined threshold. This threshold provides a concrete measure for determining when the difference in performance between groups qualifies as bias.]
identifying one or more features of the ML model that cause the bias by the ML model between the first group and the second group; (Bhide, ¶[0045]: “This generalizes from a computationally expensive individual bias checker to create a model that identifies new samples that likely have individual bias and to alter these samples first to achieve group fairness metric requirements.”, ¶[0047]: “Each sample from the unprivileged group (di=0) is tested for individual bias and if it is likely to have individual bias (i.e., bi=1), then this sample is assigned the outcome it would have received if it were in the favorable class, (i.e., yi=y(xk, 1)”, ¶[0048]: “At periodic intervals, an individual bias check is conducted on test samples and generalized to the entire feature space. When a new unlabeled sample comes in, the model scores it and the generalized individual bias checker predicts whether it will have individual bias.”) [Examiner’s note: The highlights discloses how an individual bias check is conducted on test samples and generalized to an entire feature space, which illustrates that specific features contributing to bias are identified during this process since the bias detector predicts whether new samples will have individual bias]
determining an adjustment to the identified one or more features that cause the bias
between the first group and the second group; (Bhide, ¶[0050]: “The de-biased predictions 350 is performed in post-processing by de-biasing procedure for each sample point by perturbing the protected attribute(s) in a training set (e.g., in 303 of FIG. 3), run the perturbed examples through customer model 320, and picking the most likely prediction for the perturbed data as suggested values to modify.”, ¶[0051]: “Thereby, samples predicted to have highest individual biases (among the 'unprivileged group') by the detector are prioritized for correction, suggested correction involves running perturbed examples through the customer model and picking the most likely prediction, and an arbiter can decide whether to choose the original or the suggested de-biased prediction.”) [Examiner’s note: The de-biasing process involves systematically altering (or “perturbing”) the features associated with protected attributes and running them through the model. By running perturbed examples through the model and selecting the “most likely prediction” for the perturbed data, the process effectively determines the adjustments needed to address the bias in the model’s outputs]
calculating one or more statistical metrics associated with the bias and the one or more features causing the bias (Bhide, ¶[0057]: “During the training stage of the bias detector, the invention implements an individual bias Checker that perturbs the protected attribute in the payload data for the unprivileged group samples, and computes the individual bias scores for them by finding the difference between the probability of favorable outcomes for the perturbed and the original data.”) [Examiner’s note: The highlight illustrates the process of calculating a statistical metric (the probability difference) that quantifies the bias associated with the protected feature.]
Bhide fails to disclose:
determining a performance of a machine learning (ML) model for a first group and a second group;
the determining of the bias comprising determining feature influence in the ML model using Quantitative Input Influence (QII) that measures a degree of influence that each input feature exerts on outputs of the ML model,
retraining the ML model based on the adjustment to the one or more features causing the bias in the ML model to obtain a retrained ML model without the bias;
utilizing the retrained model to make an inference without the bias between the first group and the second group;
providing a user interface (UI) comprising a display of information about the bias between the first group and the second group, the one or more features causing the bias, and the one or more statistical metrics
However, Cabrera explicitly discloses:
determining a performance of a machine learning (ML) model for a first group and a second group; (Cabrera, Page 50, Figure 4: “In the Subgroup Overview users can see how different subgroups compare to one another according to various performance metrics. As more metrics are selected at the top, additional strip plots are added to the interface. Here, a user has pinned the Female subgroup and hovers over the Male subgroup.
PNG
media_image1.png
316
851
media_image1.png
Greyscale
”, Page 50, Col. 1, ¶2]: “When a user clicks the “Generate Subgroups” button (Fig. 3), FAIRVIS splits the data into the specified subgroups and calculates various performance metrics for them. These groups are then represented in the multiple strip plots as lines corresponding to their performance for the respective metric.”, Page 50, Col. 1, ¶[5]: “In total, users can select from the following metrics: Accuracy, Recall, Specificity, Precision, Negative Predictive Value, False Negative Rate, False Positive Rate, False Discovery Rate, False Omission Rate, and F1 score. These metrics were selected as they are typically the most common metrics used for evaluating the equity and performance of classification models.”) [Examiner’s note: a first group i.e., female group, a second group i.e., male group, a performance of machine learning model for these groups is determined by the following metrics: accuracy, recall, precision, etc. and shown in figure 4]
providing a user interface (UI) comprising a display of information about the bias between the first group and the second group, the one or more features causing the bias, and the one or more statistical metrics (Cabrera, Page 52, Figure 6:
PNG
media_image2.png
538
442
media_image2.png
Greyscale
, Page 52, Col. 1, Section 5.5, ¶[2-3]: “A user is able to see the details for two groups in the Detailed Comparison View, the pinned and hovered group. A group can be pinned when a user clicks on it in the Subgroup Overview or Suggested and Similar Subgroup View, and is designated by a light red across the UI. The hovered group is designated by a light blue across the UI. These two distinct colors allow users to see a selected group’s information across various different views. There are three primary components in the Detailed Comparison View, as seen in Fig. 6. The topmost component is a bar chart displaying how a group performs for selected performance metrics.”, Col. 1, Section 5.5, ¶[4]: “The second component in the Detailed Comparison View is a bar chart for the ground truth label balance of both selected subgroups. The label imbalance is important because it can often explain extreme values for metrics like recall and precision and can suggest reasons for bias (C6)”) [Examiner’s note: a user interface i.e., the Detailed Comparison View UI, information about the bias i.e., the label imbalance between the statistical metrics (i.e., accuracy, precision, recall), the features i.e., sex, race, marital status, relationship]
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Bhide and Cabrera. Bhide teaches a bias detector used to prioritize data samples in a bias mitigation algorithm aiming to improve the group fairness measure of disparate impact. Cabrera teaches a mixed-initiative visual analytics system that integrates a novel subgroup discovery technique for users to audit the fairness of machine learning models. One of ordinary skill would have motivation to combine Bhide and Cabrera to help data scientists and the general public understand and create more equitable algorithmic systems by the interactive visualization and enable users to audit the fairness of machine learning models (Cabrera, Abstract)
However, Datta explicitly discloses:
the determining of the bias comprising determining feature influence in the ML model using Quantitative Input Influence (QII) that measures a degree of influence that each input feature exerts on outputs of the ML model, (Datta, Pg. 599, Col. 1, ¶[3]: “We formalize transparency reports by introducing a family of Quantitative Input Influence (QII) measures that capture the degree of influence of inputs on outputs of the system.”)
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Bhide and Datta. Bhide teaches a bias detector used to prioritize data samples in a bias mitigation algorithm aiming to improve the group fairness measure of disparate impact. Datta teaches a family of Quantitative Input Influence (QII) measures that capture the degree of influence of inputs on outputs of systems. One of ordinary skill would have motivation to combine Bhide and Datta because MPEP 2143 sets forth the Supreme Court rationales for obviousness including: (D) Applying a known technique to a known device (method, or product) ready for improvement to yield predictable results; (E): “Obvious to try” choosing from a finite number of identified, predictable solutions, with a reasonable expectation of success; (F) Known work in one field of endeavor may prompt variations of it for use in either the same field or a different one based on design incentives or other market forces if the variations are predictable to one of the ordinary skill in the art.
However, Lehr explicitly discloses:
retraining the ML model based on the adjustment to the one or more features causing the bias in the ML model to obtain a retrained ML model without the bias; (Lehr, ¶[0108]: “With reference to FIG. 13, shown is a model generation and training process 1300.”, ¶[0124]: “In one or more embodiments, if any of the one or more bias thresholds is exceeded, the process 1300 proceeds to bias removal process 1400”, ¶[0141]: “In various embodiments, when the process 1400 proceeds to process 1300, the intermediary model (e.g., metadata 145 thereof) can be used to initialize and train a new predictive model that excludes the one or more top-ranked, bias-contributing variables. Thus, in at least one embodiment, the predictive model generated based on each variable compartment can be a new, novel treatment of the dataset that we are comparing to the treatment of the primary dataset (e.g., in previous iterations of the predictive model). The new predictive model can be analyzed for validity and bias, and the processes 1300 and 1400 can be repeated as required to produce an iteration of the predictive model that does not violate validity or bias thresholds.”) [Examiner’s note: The highlight explains a process where a predictive model is retrained by modifying its features to address bias. If the intermediary model fails to meet validity or bias thresholds, certain high-ranking features identified as contributing to bias are excluded. Using this adjusted dataset (without the bias contributing features), a new predictive model is trained. The retraining process is repeated iteratively to refine the model until it meets the require thresholds for validity and bias]
utilizing the retrained model to make an inference without the bias between the first group and the second group; (Lehr, ¶[0121]: “At step 1309, the process 1300 includes generating predicted outcomes 136 using the trained, validated, and error threshold-satisfying iteration of the predictive model… The predictive model can generate a set of predicted outcomes 136 based on the input data and the weighted variables”) [Examiner’s note: the final predictive model used for generating predicted outcomes (i.e., making an inference) using the validated and error threshold-satisfying model (i.e., the model excluding bias feature)]
It would have obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Bhide and Lehr. Bhide teaches a bias detector used to prioritize data samples in a bias mitigation algorithm aiming to improve the group fairness measure of disparate impact. Lehr teaches systems and processes for bias removal in a predictive performance model. One of ordinary skill would have motivation to combine Bhide and Lehr to promote fairness and equity in training prediction models. Removing bias ensures that the mode’s predictions are fair and do not disproportionately favor or disadvantage any specific group. A bias-free model provides more reliable and accurate predictions, enabling better and more informed decisions based on actual performance rather than systemic or historical disparities. (Lehr, ¶[0115])
Regarding claim 17, the combination of Bhide, Lehr and Cabrera discloses all the limitations of Claim 16 (as shown in the rejections above).
Bhide in view of Lehr and Cabrera further discloses:
wherein the adjustment comprises discarding the one or more features from the ML model. (Lehr, ¶[0141]: “In various embodiments, when the process 1400 proceeds to process 1300, the intermediary model (e.g., metadata 145 thereof) can be used to initialize and train a new predictive model that excludes the one or more top-ranked, bias-contributing variables.”)
Regarding claim 19, the combination of Bhide, Lehr and Cabrera discloses all the limitations of Claim 16 (as shown in the rejections above).
Bhide in view of Lehr and Cabrera further discloses:
wherein the adjustment comprises modifying how the one or more features are bucketed by the ML model. (Lehr, ¶[0163]: “The intermediary predictive model can generate two or more sets of predictive outcomes 136.Each set of predictive outcomes 136 can be generated by executing the intermediary predictive model with one of the two or more groupings (e.g., and the corresponding input dataset thereof). In other words, the intermediary predictive model can be duplicated into two or more additional intermediary predictive models ( e.g., one predictive model per grouping).”) [Examiner’s note: The highlight discloses modifying how features are grouped or bucketed (groupings) to create multiple versions of intermediary predictive models, demonstrating how feature is adjusted]
Regarding claim 20, the combination of Bhide, Lehr and Cabrera discloses all the limitations of Claim 16 (as shown in the rejections above).
Bhide in view of Lehr and Cabrera further discloses:
wherein the machine further performs operations comprising: determining that the bias is caused by bias in training data of the ML model; (Bhide, ¶[0008]: “The starting point for the inventive approach that solves problems in the art is an individual bias detector, which finds samples whose model prediction changes when the protected attributes change, leaving all other features constant.”, ¶[0009]: “In an exemplary embodiment, the present invention can provide a post-processing computer-implemented method for post-hoc improvement of instance-level and group-level prediction metrics, the post-processing method including training a bias detector that learns to detect a sample that has an individual bias greater than a predetermined individual bias threshold value with constraints on a group bias, applying the bias detector on a run-time sample to select a biased sample in the run-time sample having a bias greater than the predetermined individual bias threshold bias value, and suggesting a de-biased prediction for the biased sample.”)
updating the training data to eliminate the bias; and (Lehr, ¶[0141]: “In various embodiments, when the process 1400 proceeds to process 1300, the intermediary model (e.g., metadata 145 thereof) can be used to initialize and train a new predictive model that excludes the one or more top-ranked, bias-contributing variables.”)
retraining the ML model with the updated training data. (Lehr, ¶[0108]: “With reference to FIG. 13, shown is a model generation and training process 1300.”, ¶[0124]: “In one or more embodiments, if any of the one or more bias thresholds is exceeded, the process 1300 proceeds to bias removal process 1400”, ¶[0141]: “In various embodiments, when the process 1400 proceeds to process 1300, the intermediary model (e.g., metadata 145 thereof) can be used to initialize and train a new predictive model that excludes the one or more top-ranked, bias-contributing variables. Thus, in at least one embodiment, the predictive model generated based on each variable compartment can be a new, novel treatment of the dataset that we are comparing to the treatment of the primary dataset (e.g., in previous iterations of the predictive model). The new predictive model can be analyzed for validity and bias, and the processes 1300 and 1400 can be repeated as required to produce an iteration of the predictive model that does not violate validity or bias thresholds.”) [Examiner’s note: The highlight explains a process where a predictive model is retrained by modifying its features to address bias. If the intermediary model fails to meet validity or bias thresholds, certain high-ranking features identified as contributing to bias are excluded. Using this adjusted dataset (without the bias contributing features), a new predictive model is trained. The retraining process is repeated iteratively to refine the model until it meets the require thresholds for validity and bias]
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as
set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AMY TRAN whose telephone number is (571)270-0693. The examiner can normally be reached Monday - Friday 7:30 am - 5:00 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David Yi can be reached at (571) 270-7519. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AMY TRAN/Examiner, Art Unit 2126
/DAVID YI/Supervisory Patent Examiner, Art Unit 2126
Prosecution Insights