DETAILED ACTION
This non-final office action is responsive to application 18/658,656 as submitted 08 May 2024.
Claim status is currently pending and under examination for claims 1-20 of which independent claims are 1, 8 and 15.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claims 1, 7-8 and 15 are objected to because of the following informalities:
Claims 1, 8 and 15 limitation of selecting a subset recites “machine learning model models” the term ‘model models’ employs redundancy of terminology.
Claims 1, 8 and 15 limitation of defining recites “for each samples” should read ‘for each of the samples’ or ‘for each sample’
Claim 7 preamble “further complying” should be ‘further comprising’ similar to claim 14
Appropriate correction or clarification is requested.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. In determining whether the claims are subject matter eligible, the examiner applies guidance set forth under MPEP 2106.
Step 1: Is the claim to a process, machine, manufacture, or composition of matter? Yes—all claims fall within one of the four statutory categories: claims 1-7 are a method/process, claims 8-14 are a computer readable medium/article of manufacture, and claims 15-20 are a system/machine. Thus, the claims are to statutory subject matter and the analysis should proceed per MPEP 2106.03.
Step 2A, prong one: Does the claim recite an abstract idea, law of nature or natural phenomenon? Yes—the claims, under the broadest reasonable interpretation, recites an abstract idea. In this case, claims fall within the enumerated grouping of abstract idea being “Mental Processes” with “Mathematical Concepts” under MPEP 2106.04(a). In particular, claims recite:
“selecting, […], one of the first machine learning models with a smallest loss using the loss function” (Mental judgment with evaluation)
“computing, […], a residual for each of the plurality of samples” (Mathematical calculation, e.g. instant specification [0051-52])
“defining, […], a new dataset comprising the plurality of samples and the residual for each samples” (Mental judgment with evaluation)
generating… “wherein a number of second machine learning models is equal to the first number multiplied by the number of boosting iterations” (Math calculation, =multiplied)
“selecting, […], a subset of the second plurality of machine learning model models having a specified property” (Mental judgment)
Focus of the claims concern selecting models, calculating residuals, defining data, multiplication and selecting models having some specified property. These functions do not preclude mental performance by human as choice of model subject to constraint by evaluation which may include mathematical calculations. As such, the claims recite at least mental processes with mathematical concepts as the abstract idea.
Step 2A, prong two: Does the claim recite additional elements that integrate the judicial exception into a practical application? No—a practical application is not integrated by the judicial exception because the additional elements are as follows:
“receiving, by a computer program, a dataset comprising a plurality of samples and a loss function” MPEP 2106.05(g) adding insignificant extra-solution activity to a judicial exception, e.g. mere data gathering
“training, by the computer program, a first number of a plurality of first machine learning models using the dataset, wherein each of the plurality of first machine learning models has a similar performance as measured by the loss function” and “training, by the computer program, the first machine learning model with the new dataset” MPEP 2106.05(h) generally linking the use of judicial exception to a particular technological environment or field of use
“generating, by the computer program, a second plurality of machine learning models by repeating the selecting, the computing, the defining, and training for a number of boosting iterations” MPEP 2106.05(g) adding insignificant extra-solution activity to the judicial exception, e.g. repetitive processing
“deploying, by the computer program, the subset of second machine learning models to a downstream task” MPEP 2106.05(f) adding the words ‘apply-it’ (or an equivalent) with the judicial exception
Balance of the claim concern receiving data, training models, generating by repeating, and deploying models to some task. The task is unspecified without real-world context and amounts to deployed model as necessary output generated by repetitive task to include training based on loss being somehow similar in performance to other models, and receiving or gathering data. These functions do not meaningfully limit the claim beyond generally linking the use of the judicial exception to a technological environment, such that the claim as a whole amounts to a drafting effort to monopolize the judicial exception. Accordingly, claims remain drawn to the judicial exception and the additional elements do not integrate the judicial exception into a practical application.
Step 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception? No—the claims do not include additional elements that amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea in to a practical application, the additional elements are identified with respect to MPEP 2106.05 and do not demonstrate an inventive concept. Particularly, the additional elements are as follows:
“receiving, by a computer program, a dataset comprising a plurality of samples and a loss function” MPEP 2106.05(g) adding insignificant extra-solution activity to a judicial exception, e.g. mere data gathering. Particularly, said extra-solution activity is a well-understood, routine and conventional (WURC) activity under MPEP 2106.05(d)(II)(i) receiving data.
“training, by the computer program, a first number of a plurality of first machine learning models using the dataset, wherein each of the plurality of first machine learning models has a similar performance as measured by the loss function” and “training, by the computer program, the first machine learning model with the new dataset” MPEP 2106.05(h) generally linking the use of judicial exception to a particular technological environment or field of use. The trained models can be established ensemble models such as XGboost known1 a decade prior to effective filing date, and there is no indication how the loss is to measure a likeness in performing an unspecified task so as to satisfy a test of particular transformation under MPEP 2106.05(c).
“generating, by the computer program, a second plurality of machine learning models by repeating the selecting, the computing, the defining, and training for a number of boosting iterations” MPEP 2106.05(g) adding insignificant extra-solution activity to the judicial exception, e.g. repetitive processing. Particularly, said extra-solution activity is a well-understood, routine and conventional activity MPEP 2106.05(d)(II)(ii) repetitive calculations.
“deploying, by the computer program, the subset of second machine learning models to a downstream task” MPEP 2106.05(f) adding the words ‘apply-it’ (or an equivalent) with the judicial exception. No meaningful limitation is imposed by the task and model deployment amounts to output necessary for model to have any relevance.
Significantly more is not satisfied for at least the reasons above. If the claim language provides only a result-oriented solution, with insufficient detail for how a computer accomplishes it, then the claims do contain an inventive concept. Taken alone, their additional elements do not amount to significantly more than the above-identified judicial exception. Looking at the claim as a whole with limitations as an ordered combination adds nothing that is not already present when looking at the elements taken individually. There is no evidence that the combination of elements improves the functioning of a computer or improves any other technology. Their collective functions merely provide conventional computer implementation.
Independent claim 8 recites limitations similar to claim 1 and further recites “A non-transitory computer readable storage medium, including instructions stored thereon, which when read and executed by one or more computer processors”. This is considered as additional elements which amounts to mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea under MPEP 2106.05(f). Particularly, the computer elements do not qualify as a particular machine under MPEP 2106.05(b). As such, the claim remains drawn to the judicial exception and additional elements do not integrate the judicial exception into a practical application or amount to significantly more.
Independent claim 15 recites limitations similar to claim 1 and further recites “A system, comprising: a database storing a dataset; a user electronic device; and an electronic device executing a computer program that is configured to” cont’d “with different initializations or different random seeds to fit the dataset”. The limitations are considered additional elements. Particularly, the system devices fall under MPEP 2106.05(f) mere use of a computer to perform an abstract idea, and which does not qualify as particular machine under MPEP 2106.05(b). Further, the database storing dataset amounts to adding insignificant extra-solution activity under MPEP 2106.05(g), and which is a well-understood, routine and conventional activity under MPEP 2106.05(d)(II)(iv) storing and retrieving data in memory. Finally, the various initializations or random seeds further describes training which remains considered as generally linking the use of the judicial exception to a particular technological environment or field of use. At least some initial or first values on which to evaluate data is requisite and does not meaningfully limit the claim through technical solution or reveal particular transformation to impart inventive concept. Accordingly, the claim remains drawn to the judicial exception and additional elements do not integrate the judicial exception into a practical application or amount to significantly more.
Dependent claim 2, 9 and 16 disclose receiving the number of boosting iterations. The limitation is considered as additional elements which fall under MPEP 2106.05(g) adding insignificant extra-solution activity to the judicial exception. Particularly, said extra-solution is a well-understood, routine and conventional activity under MPEP 2106.05(d)(II)(i) receiving data. Therefore, the claims remain drawn to the judicial exception and additional elements fail to integrate the judicial exception into a practical application or amount to significantly more.
Dependent claims 3, 10 and 17 disclose receiving a hypothesis space comprising one of sparse decision-trees, linear models, and neural networks. A hypothesis space of linear models e.g. regression does not preclude mental processes or mathematical concepts and thus may be considered part of the abstract idea. The sparse decision trees and neural networks are considered additional elements which fall under MPEP 2106.05(h) generally linking the use of the judicial exception to a particular technological environment or field of use. Such models are not structurally modified to impart an inventive concept. Accordingly, the claim remains drawn to the abstract idea and additional elements fail to integrate the abstract idea into a practical application or amount to significantly more.
Dependent claims 4 and 11 disclose limitation already addressed at claim 15 discussed above.
Dependent claims 5-6, 12-13, and 18-19 disclose wherein the specified property comprises fairness measured using statistical parity, and interpretability measured using shapley values. These properties are not new properties (e.g. shapley introduced by Lundberg) and simply point to the use of established metrics to be included as part of the abstract idea. There are no additional elements.
Dependent claims 7, 14 and 20 disclose computing a predictive multiplicity metric measuring conflicting predictions. The limitation is considered part of the abstract idea including math multiplicity with conflicting predictions evaluated mentally such as pairwise disagreement. The measurement by metric is not specified by technical solution with particularity, and evidence of such measure being already established is by Gomez detailed below. Thus, even a narrow interpretation would not lead to finding of inventive concept. Rather, the limitation embellishes the abstract idea and claims remain drawn to the judicial exception without integration into practical application or significantly more.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-4, 6, 8-11, 13, 15-17 and 19 are rejected under 35 USC 103 as being unpatentable over:
Donnelly et al., “The Rashomon Importance Distribution: Getting RID of Unstable, Single Model-based Variable Importance” hereinafter Donnelly (arXiv: 2309.13775v4), in view of
Zuin, Gianlucca Lodron “Ensemble Learning through Rashomon Sets” hereinafter Zuin.
With respect to claim 1, Donnelly teaches:
A method {Donnelly [P.4 Sect.3] “Methods… we propose using the entire Rashomon set”}, comprising:
receiving, by a computer program, a dataset comprising a plurality of samples and a loss function {Donnelly [P.4 Sect.3] Let D(n) denote dataset, received in Eq.1 Rashomon set including loss l for function f of model class F, described [P.6 ¶2] “we sample observations from an observed dataset”};
training, by the computer program, a first number of a plurality of first machine learning models using the dataset, wherein each of the plurality of first machine learning models has a similar performance as measured by the loss function {Donnelly [P.24 ¶1] “trained and evaluated all models” at [P.26 ¶2,1] trained models via algorithms including e.g. Random Forest (RF), and wherein [P.4 Sect.3] “empirical losses are within some bound” bounded loss provides similar performance, measuring may entail RLD Rashomon loss distribution [P.5 ¶3] and/or error analysis Fig 11 [P.26]};
selecting, by the computer program, one of the first machine learning models with a smallest loss using the loss function {Donnelly [P.4] Eq.1 loss minimizer ‘ ≤ min l (f ‘ minimized being a smallest loss l argument of function f for class of models F, selected by ‘ ≤ ‘ threshold operand, e.g. [P.3] Fig 2 “Rashomon threshold; all models whose loss is under a threshold” again at Fig 1};
computing, by the computer program, a residual for each of the plurality of samples using the one first machine learning model {Donnelly [P.6 ¶4] Eq.3 computes residual as the “difference between RIDj and RID(hat)j is within some pre-specified error” where RID Rashomon importance distribution with respective arguments and subscript represents the samples and models};
defining, by the computer program, a new dataset comprising the plurality of samples and the residual for each samples {Donnelly [P.6 ¶2] “new datasets” as “We estimate RIDj for each variable j by leveraging bootstrap sampling to draw new datasets…dataset sampled with replacement from D(n) in iteration b=1, …, B” comprises empirical estimate of RID(hat)j, also termed replicate e.g. Fig 1, [P.7 ¶1] “DGP ϕ(sub) calculations on 500 new datasets from the DGP” data generation process per [P.4 Sect. 3.1] see also [P.5 ¶2] “considering all possible resamples”};
However, Donnelly does not appear to disclose the following limitations which are met by Zuin:
training, by the computer program, the first machine learning model with the new dataset {Zuin [P.36 ¶1] “we train models on data Dtrain coming from a distribution T” further [P.42 ¶2] “data drawn from T consists of adding Gaussian noise to the input features” thus T adds noise to arrive at new dataset, see e.g. [P.38 ¶2] “models to address new data distribution” similar at [P.37 ¶2] model for new data distribution correlates to unseen distribution, e.g. generalization};
generating, by the computer program, a second plurality of machine learning models by repeating the selecting, the computing, the defining, and training for a number of boosting iterations, wherein a number of second machine learning models is equal to the first number multiplied by the number of boosting iterations {Zuin [P.120 Last¶] “we employed our Rashomon Ensemble technique with gradient-boosting machines” where boosting iterations are [P.30 ¶1] Let “i be an iteration of the gradient boosting algorithm… boosting is a gradient descent algorithm for combining and training” particularly using Algorithm 1 at [P.43] for-do with while-do loops (iterations) minimizes expected loss for candidate model expansion/generation, the multiplied may comprise factorial combinatorial space “F! combinatorial space of feature subsets to select the best-performing specialized model” introduced [P.38-39], and where second plurality models are described e.g. [P.34 ¶1] “two models: f’1 in the hypothesis space F1 and f*2 in hypothesis space F2 …expected loss of optimal model f*2 ∈ F2”. Further, residuals are made express at [P.30 ¶1]};
selecting, by the computer program, a subset of the second plurality of machine learning model models having a specified property {Zuin [P.43 ¶1] “select the best-performing specialized models and build the Rashomon ensemble” again at [P.94 ¶2], the selected models are specified by searching space of feature subsets and clustering by explanatory factors/properties like Shapley feature described throughout, e.g. [P.46 ¶1]}; and
deploying, by the computer program, the subset of second machine learning models to a downstream task {Zuin [P.50 ¶1] “deployment model” e.g. [P.75 ¶2] “our model was deployed into the APEEAM South America and used to predict heating slivers” is a downstream task, as is COVID-19 diagnosis [P.112-13 ¶2] “deploy our ensembling approach” which is returned at Alg.1 [P.43]}.
Zuin is directed to ensemble learning with gradient boosting for Rashomon sets thus being analogous. A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to combine the teachings of Zuin with Donnelly to arrive at the invention as claimed for a stated motivation [Abst] “key motivation is to study the Rashomon Effect… Our approach to deal with the Rashomon Effect is to stratify, during training, models into groups that are either coherent or contrasting. From these Rashomon groups, we can select models that increase the robustness of the production responses along with means to gauge data drift” with Contributions [P.20-21] “Our main premise is that explicability can be employed to induce diversity in ensembles” and further [P.30 ¶1-2] “gradient boosting approach attempt to fit h to the residual loss…leads to a reduction in correlation between trees and prevents over-fitness.”
With respect to claim 2, the combination of Donnelly and Zuin teaches the method of claim 1, wherein the computer program further
receives the number of boosting iterations {Zuin [P.30 ¶1] Let… “i be an iteration of the gradient boosting” received by estimator h, as h(x) = y – Fi(x) where Fi subscript i denotes the iterations of boosting}.
With respect to claim 3, the combination of Donnelly and Zuin teaches the method of claim 1, further comprising:
receiving, by the computer program, a hypothesis space comprising one of sparse decision-trees, linear models, and neural networks {Zuin [P.32 ¶2] Hypothesis spaces F1 and F2 comprising subset of model functions f1 and f2, exemplified [P.140 ¶3] “hypothesis space of all decision trees with depth up to some value” where depth constraint suggests some sparsity e.g. where [P.121 ¶2] “Gradient-Boosted Decision Tree estimators with a maximum tree depth of 50 and a maximum number of leaves of 50” such that [P.92 ¶2] “models are assigned to the same sparse grouping” and/or pruning [P.43]. Further, disclosed are [P.48 Tbl.4.1] MLP (i.e. neural network), and linear models Fig. 6.10}.
With respect to claim 4, the combination of Donnelly and Zuin teaches the method of claim 1, wherein
the first plurality of machine learning models are trained with different initializations or different random seeds to fit the dataset {Zuin [P.45 ¶1] “different random seed” to [P.41 ¶1] “fit the source data” e.g. [P.56 ¶1] “random seeds 114 and 3452” and/or [P.43] “initialize pool P with n models containing random combination of features”. See also Donnelly [P.24 ¶4] “we used random seed 0 for dataset generation, model training, and evaluation”}.
With respect to claim 6, the combination of Donnelly and Zuin teaches the method of claim 1, wherein
the specified property comprises interpretability, and interpretability is measured using a SHapley Additive exPlanations value for each of the plurality of second machine learning models {Zuin [P.46 ¶1] “Shapley Additive Explanations (or simply SHAP), is the usage of Shapley values to interpret a prediction model. We represent how model f ’ explains the data” where f regards the models applied in Alg.1 [P.43], illustrated Fig 4.9 SHAP of target and anchor models, described e.g. [P.42 ¶2] “we find instances of models with similar SHAP…as well as contrasting SHAP” e.g. as per [P.37 ¶4] “we cluster models in F’ based on the distance between their explanation vectors (i.e., SHAP values)”}.
With respect to claim 8, the rejection of claim 1 is incorporated. The difference in scope being a non-transitory computer readable storage medium storing instructions to be read and executed by computer processor to perform limitations of method claim 1. Donnelly discloses computer cluster of processors [P.25 ¶2,1] as computing environment with requisite memory for implementation with scikit-learn software and provides github code [P.3 ¶1]. The remainder of this claim is rejected for the same rationale as claim 1.
With respect to claim 9, the combination of Donnelly and Zuin teaches the non-transitory computer readable storage medium of claim 8, and further teaches the limitation of claim 2. Therefore, the rejection of claim 2 is applied to claim 9.
With respect to claim 10, the combination of Donnelly and Zuin teaches the non-transitory computer readable storage medium of claim 8, and further teaches the limitation of claim 3. Therefore, the rejection of claim 3 is applied to claim 10.
With respect to claim 11, the combination of Donnelly and Zuin teaches the non-transitory computer readable storage medium of claim 8, and further teaches the limitation of claim 4. Therefore, the rejection of claim 4 is applied to claim 11.
With respect to claim 13, the combination of Donnelly and Zuin teaches the non-transitory computer readable storage medium of claim 8, and further teaches the limitation of claim 6. Therefore, the rejection of claim 6 is applied to claim 13.
With respect to claim 15, the rejection of claim 1 is incorporated. The difference in scope being a system comprising database storing dataset and user electronic device executing a computer program that is configured to perform limitations similar to method claim 1, and further recites limitation of claim 4. Zuin discloses [P.116 ¶1] “SQL database… Once stored in the database, the result is made available” and “All users have a username and password” emphasis database for users, a requisite device of system for implementing pseudo-code [P.43] with scikit-learn python package [P.26 ¶2] conveys computer environment such as that disclosed by Donnelly [P.25 ¶2,1]. Further, the limitation of claim 4 is taught by the combination of Donnelly and Zuin. A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to specify database per Zuin in combination with Donnelly to arrive at the invention as claimed for the motivation that it handles tabular data [P.44 ¶2] “our approach is algorithm-agnostic, and reproducible in any model that handles tabular data.” Furthermore, the system with device such as Donnelly’s would be obvious to try in choosing from a finite number of identified, predictable solutions for implementing computer environment with a reasonable expectation of success.
With respect to claim 16, the combination of Donnelly and Zuin teaches the system of claim 15, and further teaches the limitation of claim 2. Therefore, the rejection of claim 2 is applied to claim 16.
With respect to claim 17, the combination of Donnelly and Zuin teaches the system of claim 15, and further teaches the limitation of claim 3. Therefore, the rejection of claim 3 is applied to claim 17.
With respect to claim 19, the combination of Donnelly and Zuin teaches the system of claim 15, and further teaches the limitation of claim 6. Therefore, the rejection of claim 6 is applied to claim 19.
Claims 5, 12 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Donnelly and Zuin, and further in view of Aivodji et al., “Characterizing the risk of fairwashing” hereinafter Aivodji (arXiv: 2106.07504v3).
With respect to claim 5, the combination of Donnelly and Zuin teaches the method of claim 1. Aivodji teaches wherein
the specified property comprises fairness, and fairness is measured using a statistical parity for the plurality of second machine learning models {Aivodji [P.3 ¶2-4] “Fairness In The Rashomon Set (FAIRS)… we consider four different statistical notions of fairness, namely statistical parity” ΔSP Table 1 defines Statistical Parity fairness metric, and further details Rashomon set Eq.2 [P.9] that seeks to minimize unfairness Eq.3, see results e.g. Table 3 ΔSP applied to XgBoost “model agnostic technique to train any classifier under fairness constraints”}.
Aivodji is directed to machine learning model training with Rashomon sets thus being analogous. A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to employ Aivodji’s statistical parity for fairness in machine learning models in a Rashomon set in combination with the models in Rashomon set of Donnelly and Zuin to arrive at the invention as claimed as simple substitution of specified properties according to known techniques to obtain predictable result in which “statistical notions of fairness, which require a model to exhibit approximate parity according to a statistical measure across the different groups” [P.3 ¶4] thus “adopting robust fairness-enhancing techniques” [P.7 ¶4].
With respect to claim 12, the combination of Donnelly and Zuin teaches the non-transitory computer readable storage medium of claim 8, and further combination with Aivodji teaches the limitation of claim 5. Therefore, the rejection of claim 5 with equal motivation is applied to claim 12.
With respect to claim 18, the combination of Donnelly and Zuin teaches the system of claim 15, and further combination with Aivodji teaches the limitation of claim 5. Therefore, the rejection of claim 5 with equal motivation is applied to claim 18.
Claims 7, 14 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Donnelly and Zuin in view of Gomez et al., “Algorithmic Arbitrariness in Content Moderation” hereinafter Gomez (arXiv: 2402.16979v1).
With respect to claim 7, the combination of Donnelly and Zuin teaches the method of claim 1. Gomez teaches further complying:
computing, by the computer program, a predictive multiplicity metric for the second plurality of machine learning models, wherein the predictive multiplicity metric measures conflicting predictions among the second plurality of machine learning models {Gomez [P.10 ¶3 - P.11 ¶4] “predictive multiplicity when models in the Rashomon set assign conflicting predictions to the same data point, formally defined… we use the following two metrics: arbitrariness…pairwise disagreement” respectively Definitions 1 and 2, computed using Eqs. 3-5 for Rashomon set of models}.
Gomez is directed to trained machine learning models for Rashomon set thus being analogous. A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to compute predictive multiplicity measuring conflicting predictions per Gomez in combination to arrive at the invention as claimed for a motivation being [P.11 ¶3] “We select the above metrics because they quantify two important aspects of predictive multiplicity: (i) the fraction of samples in a dataset for which predictions are arbitrary (Defn. 1), in that a competing model would have assigned a different prediction, and (ii) the extent to which models disagree on individual (Defn. 2).”
With respect to claim 14, the combination of Donnelly and Zuin teaches the non-transitory computer readable storage medium of claim 8, and further combination with Gomez teaches the limitation of claim 7. Therefore, the rejection of claim 7 with equal motivation is applied to claim 14.
With respect to claim 20, the combination of Donnelly and Zuin teaches the system of claim 15, and further combination with Gomez teaches the limitation of claim 7. Therefore, the rejection of claim 7 with equal motivation is applied to claim 20.
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Leventi-Peetz et Weber, “Rashomon Effect and Consistency in Explainable Artificial Intelligence (XAI)” details XGboost [P.799 Sect. 3.1]
Li et al., “Exploring the Cloud of Feature Interaction Scores in a Rashomon Set” ICLR,
arXiv: 2305.10181v1 discloses Mask-based Rashomon
Xin et al., “Exploring the Whole Rashomon Set of Sparse Decision Trees” NeurIPS,
arXiv: 2209.08040v2 discloses sparse decision trees, providing algorithms
Hu et al., US PG Pub No 2024/0029154 Wells Fargo [0090] “M is indicative of the maximum number of boosting iterations” again at [0119], where M is tunable hyperparameter [0167]
McNaughton et al., US Patent No 11,853,906 see Figs 8-9
Paliwal et al., US PG Pub No 2023/0325886A1 see [0067]
Wang et al., US PG Pub No 2025/0190817A1 see Figs 13-15
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Chase P Hinckley whose telephone number is (571)272-7935. The examiner can normally be reached M-F 9:00 - 5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Miranda M. Huang can be reached at 571-270-7092. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHASE P. HINCKLEY/Examiner, Art Unit 2124
1 Chen et Guestrin, “XGBoost: A Scalable Tree Boosting System” arXiv:1603.02754v3, ~95k citations Google Scholar
Prosecution Insights