DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendment filed 10/22/2025 has been entered. Claims 1, 4, 6, 10, 13-15, 17-18, 20, 22, 24 and 26-33 remain pending in the application. Claims 32-33 are new.
Response to Arguments
Applicant’s arguments, filed 10/22/2025, with respect to the rejections of the claims under 103 have been fully considered and are persuasive because of the amendments. Therefore, the rejection has been withdrawn.
Applicant’s arguments, filed 10/22/2025, with respect to the rejections of the claims under 101 have been fully considered and are not persuasive.
Applicant argues (pages 14-17)
A. Claims 1, 4, 6, 10, 13-15, 17-18, 20, 22, 24 and 26-31 Are Patent Eligible Under USPTO Step 2A Because They Are Integrated Into a Practical Application.
Under Alice step one, claims are "consider[ed] ... as a whole ... and read ... in light of the specification." Packet Intelligence LLC v. NetScout Sys., Inc., 965 F.3d 1299, 1309 (Fed. Cir. 2020). In other words, "[t]he 'directed to' inquiry ... cannot simply ask whether the claims involve a patent-ineligible concept, because essentially every routinely patent-eligible claim involving physical products and actions involves a law of nature and/or natural phenomenon." Enfish, LLC v. Microsoft Corp., 822 F.3d 1327, 1335 (Fed. Cir. 2016) (emphasis in original). Importantly, the Federal Circuit has recognized that "software-based innovations can make 'non-abstract improvements to computer technology' and be deemed patent-eligible subject matter at step I." Packet Intelligence, 965 F.3d at 1309 (quoting Finjan, Inc. v. Blue Coat Sys., Inc., 879 F.3d 1299, 1304 (Fed. Cir. 2018) (quoting Enfish, 822 F.3d at 1335-36)).
The Manual of Patent Examining Procedure (MPEP) explains that Step 2A of the subject matter eligibility analysis is divided into two prongs: Prong One asks whether the claim recites a judicial exception. See MPEP § 2106.04(11). Prong Two asks whether "the claim recite[s] additional elements that integrate the judicial exception into a practical application." Id As explained by the MPEP, "mere recitation of a judicial exception does not mean that the claim is 'directed to' that judicial exception under Step 2A Prong Two." MPEP § 2106.04(II)(A)(2). Accordingly, a claim that recites a judicial exception may not be directed to the judicial exception if "the claim as a whole integrates the recited judicial exception into a practical application of the exception." Id.
To determine whether the claims integrate a judicial exception into a practical application, the Office must analyze "the limitations containing the judicial exception as well as the additional elements in the claim besides the judicial exception." MPEP § 2106.04(d)(III). "Accordingly, the additional limitations should not be evaluated in a vacuum, completely separate from the recited judicial exception .... Instead, the analysis should take into consideration all the claim limitations and how those limitations interact and impact each other." Id Significantly, the MPEP explains that the "routine and conventional" consideration should not be applied under Step 2A Prong Two. "[T]he claimed invention may integrate the judicial exception into a practical application by demonstrating that it improves the relevant existing technology although it may not be an improvement over well-understood, routine, conventional activity." MPEP § 2106.04(d)(l).
Applicant respectfully submits that the claims are patent-eligible because the claims integrate any alleged abstract idea by reciting a system that includes various computing components in a "meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception." MPEP § 2106.04(d). The pending claims are directed to a particular practical application as they provide a technical solution to a technological problem relating to automated generation of explanation information for complex predictive machine learning models based on a particular, detailed process using particular inputs, feature contribution values, and a similarity metric analysis.
The Office has asserted that each of the additional elements of the independent claims is either data gathering, extra-solution activity, recited at a high level of generality, or merely indicates a field of use or technological environment such that they do not integrate the claimed invention into a practical application. Amended independent claim 18, for example, recites, among other limitations:
invoking one or more functions of the modeling system to obtain model access information comprising at least a tree structure of the predictive machine learning model;
for each of a plurality of features identified in the evaluation input rows, determine a distribution of feature contribution values across a set of model outputs using the tree structure, the evaluation input rows, and the reference input rows, wherein the distribution of feature contribution values for each of the identified features represents an influence of the identified feature on first scores for the evaluation input rows;
for each pair of features among the identified features, determine a similarity metric value based on a difference between one of the distributions of feature contribution values determined for a first feature of the pair of features and another of the distributions of feature contribution values determined for a second feature of the pair of features, wherein the similarity metric value quantifies a similarity between the first and second features for each of the pair of features;
construct a graph that comprises nodes representing each of the identified features and edges representing each of the similarity metric values;
perform a node clustering process to identify node clusters of the graph based on at least one of the similarity metric values assigned to each of the graph edges, wherein each of the node clusters represents a feature group and each of the feature groups comprises a subset of the features
The above-identified claim limitations recite particular, specific limitations for mapping explanation information to groups of features of a predictive machine learning model to improve explanations for model output of the machine learning model. Invoking a function of a modeling system, determining distributions of contribution values using a tree structure of a predictive machine learning model, constructing graphs with particular nodes and edges, and performing a node clustering process on the graph, for example, cannot reasonably be performed in the human mind or with pen and paper.
Further, the associated limitations are not recited at a high level of generality and the claims have been amended herein to provide a significant amount of additional detail regarding the identification of feature groups, as just one example. The USPTO stated in the "Reminders on evaluating subject matter eligibility of claims under 35 U.S.C. 101" issued August 4, 2025 ("2025 Guidance") that "Examiners should be careful to distinguish claims that recite an exception (which require further eligibility analysis) from claims that merely involve an exception (which are eligible and do not require further eligibility analysis)." The pending amendment independent claims simply do not recite, and are not directed to, any judicial exception, and instead provide a significant amount of technical specificity.
The limitations of the amended independent claims cannot reasonably be said to compare with the limitations in Bilski, for example, which limited the claimed invention to the commodities and energy markets. Instead, these limitations are central to the claimed invention and further provide the method by which node clustering and graphs are used to identify features groups. Applicant believes that the amended independent claims are narrowly drawn and directed to a particular, practical application.
In response
As stated in the 101 rejections below, the judicial exception is not integrated into a practical application. The amended claim recites the additional elements of (among others) “invoking one or more functions of the modeling system to obtain model access information comprising at least a tree structure of the predictive machine learning model” and “applying the predictive machine learning model to the input row to generate a model output”. These limitations are recited at a high-level of generality (i.e., as a generic device performing the generic computer functions) such that they amount to no more than mere instructions to apply the exception using the generic computer components (MPEP 2106.05(f)). Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The Applicant then argues that the limitations of “determining distributions of contribution values … constructing graphs with particular nodes and edges … performing a node clustering process on the graph, for example, cannot reasonably be performed in the human mind or with pen and paper.” The examiner respectfully disagrees.
A human can determine/assign scores/values to the items/features based on some factors. The human can generate/draw a graph comprising nodes and connections between the nodes, and the human can group some nodes together based on some certain features.
The Applicant further argues that “The above-identified claim limitations recite particular, specific limitations for mapping explanation information to groups of features of a predictive machine learning model to improve explanations for model output of the machine learning model”. However, as stated in the 101 rejections below, the limitation of generating/selecting explanations for model output recites a mental process, and an improvement in an abstract idea itself is not an improvement in technology. The claim must recite additional elements which provide the improvement.
Applicant argues (pages 17-19)
The independent claims have also been amended herein to recite a machine learning platform system that requires steps performed by several devices including an application server, a modelling system, and a model evaluation system. An application server provides an input row received from an operator device to a modelling system, which applies a predictive machine learning model to the input row to generate model output that is then provided to a model evaluation system that generates explanation information based on a tree structure for the machine learning model obtained from the modeling system and evaluation and reference input rows, features, feature contribution values, similarity metric values, and feature groups. Thus, each of the recited devices is essential to the operation of the claimed invention and not merely a technological environment.
The Office has asserted on page 18 of the Office Action that these limitations "amount to merely indicating a field of use or technological environment in which to apply a judicial exception" and therefore do not satisfy Step 2A, Prong Two. However, MPEP 2106.05(h), which was cited by the Office for this proposition, recites "[f]or claim limitations that generally link the use of the judicial exception to a particular technological environment or field of use, examiners should explain in an eligibility rejection why they do not meaningfully limit the claim." The Office still has not identified any field of use or technological environment for which these limitations are examples.
The 2025 Guidance on page 4 reiterates that a claim is eligible in Step 2A Prong Two when it "reflects an improvement to ... another technology or technical field" and that an important consideration as to whether a claim improves technology or a technical field "is the extent to which the claim covers a particular solution to a problem or a particular way to achieve a desired outcome, as opposed to merely claiming the idea of a solution or outcome." Applicant respectfully submits that the amended claims are patent-eligible because they integrate any alleged abstract idea by providing a particular technical solution to a technical problem relating to automated explanation of complex machine learning model decisioning, and not just the idea of such a solution.
When viewed as a whole, the limitations are integrated into a practical application related to adverse action decisioning and explanation for complex machine learning models. Additionally, the problems solved by the claimed technology are technical and the claimed method is "necessarily rooted in computer technology" and requires interactions of multiple computing devices to analyze and explain machine learning model decisioning, which is a problem that only exists in computer networks. See DDR Holdings, LLC v. Hotels.com, L.P., 773 F.3d 1245, 1257 (Fed. Cir. 2014).
Even further, the claimed technology is clearly directed to an improvement to machine learning technology, which was confirmed by the Federal Circuit to qualify as a practical application. See, contra, Recentive Analytics Inc. v. Fox Corp. et al., Case No. 23-2437, 18 (Fed. Cir., April 18, 2025). The claimed technology does more than apply established methods of machine learning to a new data environment or field of use, and the amended claims are in fact silent as to any particular data environment or field of use. Instead, the pending claims optimize and improve machine learning technology by more effectively generating explanations of complex machine learning model decisioning.
The claimed invention does not "perform a task previously undertaken by humans with greater speed and efficiency than could previously be achieved." See, contra, Recentive, 15. For example, before the claimed invention, humans could not analyze the contribution of features for black box, complex, tree-based machine learning models to explain decisions output by such models and such improvements were simply not undertaken by humans irrespective of speed and efficiency. As the Federal Circuit stated in Recentive, "[m]achine learning is a burgeoning and increasingly important field and may lead to patent-eligible improvements in technology." Recentive, 18. The claimed invention that is the subject of the pending claims is one such patenteligible technological improvement.
Applicant has not attempted to patent and preempt any particular abstract idea, but has instead recited in claims 1, 4, 6, 10-11, 13-18, 20-22, 24, and 26-31 a particular, practical application and technical solution that solves a technical problem as required by Prong Two of the revised Step 2A recited in the 2019 Guidelines. As a result, the claimed subject matter is patent eligible.
In response
The Applicant first stated that “The independent claims have also been amended herein to recite …”, then argues that “The Office has asserted on page 18 of the Office Action that these limitations "amount to merely indicating a field of use or technological environment”. This argument is confusing. Further, the claim limitations such as (among others) “determining a distribution of feature contribution values across a set of model outputs using the tree structure, the evaluation input rows, and the reference input rows … wherein the similarity metric value quantifies a similarity between the first and second features for each of the pair of features … wherein each of the node clusters represents a feature group and each of the feature groups comprises a subset of the features” only describes how to determine a contribution values , how to determine the similarity metric value and how to grouping the nodes using certain data/features (the evaluation input rows, the reference input rows, the first and second features …), and therefore, those limitations do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Also, the claim limitations do not show an improvement to the functioning of a computer or to any other technology or technical field. The claim does not recite how the machine learning model is trained or operated to implement the process such that the machine learning model is improved. The claim only recites using a machine learning technology as a tool to perform a mental process of “generating decision and an explanation”, wherein, the machine learning model is trained using a certain training data and computer components such as a server, modelling and evaluation model system to maybe improve the generating process. However, as mentioned above, an improvement in an abstract idea itself is not an improvement in technology. The claim must recite additional elements which provide the improvement.
Applicant argues (pages 19-22)
B. Claims 1, 4, 6, 10, 13-15, 17-18, 20, 22, 24 and 26-31 Are Patent Eligible Under USPTO Step 2B Because They Recite a Combination of Elements that Is Significantly More than an Abstract Idea
Since the currently pending claims are patent eligible under Prong Two of Step 2A of the 2019 Guidelines, Step 2B need not be addressed. However, assuming arguendo that the currently pending claims were not patent eligible under Prong Two of Step 2A, the currently pending claims would still be patent eligible under Step 2B set forth in the 2014 Guidelines.
The Step 2B inquiry asks whether "the additional elements recited in the claims contribute an inventive concept." MPEP § 2106.05. Applicant notes that a relevant consideration for evaluating whether the claims provide an inventive concept is "adding a specific limitation other than what is well-understood, routine, conventional activity in the field." Id.
The Federal Circuit reinforced this notion by establishing that "[t]he question of whether a claim element or combination of elements is well-understood, routine and conventional to a skilled artisan in the relevant field is a question of fact" that "must be proven by clear and convincing evidence." Berkheimer v. HP, Inc., 881 F.3d 1360, 1368 (Fed. Cir. 2018) (citing Microsoft Corp. v. i4i Ltd P 'ship, 564 U.S. 91, 95 (2011)). "As such, an examiner should determine that an element (or combination of elements) is well-understood, routine, conventional activity only when the examiner can readily conclude, based on their expertise in the art, that the element is widely prevalent or in common use in the relevant industry." MPEP § 2106.05.
Applicant respectfully submits that the amended claims involve more than performance of well-understood, routine, and conventional activities previously known to the industry. For example, the independent claims recite at least "for each of a plurality of features identified in the evaluation input rows, determine a distribution of feature contribution values across a set of model outputs using the tree structure, the evaluation input rows, and the reference input rows, wherein the distribution of feature contribution values for each of the identified features represents an influence of the identified feature on first scores for the evaluation input rows," "for each pair of features among the identified features, determine a similarity metric value based on a difference between one of the distributions of feature contribution values determined for a first feature of the pair of features and another of the distributions of feature contribution values determined for a second feature of the pair of features, wherein the similarity metric value quantifies a similarity between the first and second features for each of the pair of features," "construct a graph that comprises nodes representing each of the identified features and edges representing each of the similarity metric values," and "perform a node clustering process to identify node clusters of the graph based on at least one of the similarity metric values assigned to each of the graph edges, wherein each of the node clusters represents a feature group and each of the feature groups comprises a subset of the features." These limitations cannot reasonable be said to represent well understood, routine, or conventional activities.
The independent claims with these amended limitations provide an improvement to machine learning model evaluation technology by improving the accuracy and efficiency with which machine learning model explanations are provided. The determining limitations of the independent claims are not generic computer functions performed by generic computers. The Office's Step 2B analysis again focuses on a subset of the claim elements and fails to consider the claims as a whole, which clearly recites much more than accessing and storing information.
Further, the amended independent claims recite particular computing devices including an application server that provides an input row received from an operator device to a modelling system and provides explanation information received from a model evaluation system and a decision generated based on the output received from the modelling system to the operator device in response to the input row. The modelling system applies the credit model to the input row to generate the output and provides the output to the application server and the model evaluation system. A model evaluation system, inter alia, identifies a feature group related the output, responsive to the input row and determined based on similarity metric values and provides to the application server explanation information for the feature group. The explanation information comprising human-readable explanatory text associated with an impact on the output of at least one feature of the identified feature group.
Throughout the Office Action, the Office has asserted that the pending claim limitations amount to no more than adding the words "apply it" with a judicial exception. The 2025 Guidance on page 4 cautions Examiners "not to oversimply claim limitations and expand the application of the 'apply it' consideration." Specifically, the 2025 Guidance makes clear that a claim that recites only the idea of a solution or outcome is patent ineligible, as is a claim that merely invokes a computer as a tool to perform an existing process.
The independent claims do not recite only the idea of a solution or outcome but instead recite a particular practical application that recites details of how the improvement to machine learning model technology is achieved. Those details include generating distributions of feature contribution values across a set of model outputs using a tree structure, evaluation input rows, and reference input rows and performing a node clustering process to identify node clusters of a constructed graph based on at least one similarity metric value assigned to each of the graph edges, wherein each of the node clusters represents a feature group and each of the feature groups comprises a subset of the features. The claims recite a particular, detailed solution to the technical problem of explaining complex machine learning model decisioning.
Moreover, the pending claims do not merely invoke computers as a tool to perform an existing process. The claimed improvement to machine learning technology is not a process that could possibly have existed absent computers. The amended claims do not invoke a generic computer to generate a credit score for a credit applicant, for example, but instead use particular distributions of feature contribution values, similarity metric values, and constructed graphs that comprise nodes representing each of the identified features and edges representing each of the similarity metric values, among other steps, to improve accuracy of machine learning model explainability and analysis. Thus, the claims clearly improve an existing technology (i.e., machine learning model technology).
The independent claims are directed to issues rooted in computer technology and include significant and meaningful limitations for generating explanation information for machine learning models. These limitations are not well-understood, routine, or conventional in the field, and add unconventional steps that confine the claims to a particular useful application. See Mayo Collaborative Serv. v. Prometheus Labs., Inc., 132 S. Ct. 1289, 1302 (2012). The amended independent claims also make clear that this technology does not merely implement conventional functions, and instead requires specially programmed computing devices with particular input data, determined feature contribution and similarity metric values, graphs, and graph processing methods.
The claims recite an inventive concept such that the claims are directed to patent eligible subject matter under step two of Alice. Accordingly, for at least the reasons discussed, Applicant respectfully submits that claims 1, 4, 6, 10, 13-15, 17-18, 20, 22, 24 and 26-31 are directed to patent eligible subject matter and respectfully requests that the Examiner withdraw the patent eligibility rejections of claims 1, 4, 6, 10, 13-15, 17-18, 20, 22, 24 and 26-31.
In response
As explained in the 101 rejections section below, beside the limitations which recite mental processes, the claim does not include additional limitations that are sufficient to amount to significantly more than the judicial exception.
the additional elements of “a machine learning platform system”, “an application server”, “a modelling system” and “a model evaluation system” to perform the computer functions amount to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept.
The additional elements of “invoking one or more functions of the modeling system to obtain model access information comprising at least a tree structure of the predictive machine learning model”, “applying the predictive machine learning model to the input row to generate a model output” amount to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept.
The additional elements of “determining a distribution of feature contribution values across a set of model outputs using the tree structure, the evaluation input rows, and the reference input rows, wherein the distribution of feature contribution values for each of the identified features represents an influence of the identified feature on first scores for the evaluation input rows … wherein the similarity metric value quantifies a similarity between the first and second features for each of the pair of features … wherein each of the node clusters represents a feature group and each of the feature groups comprises a subset of the features” amount to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitation that amounts to merely indicating a field of use or technological environment in which to apply a judicial exception does not amount to significantly more than the judicial exception (see MPEP 2106.05(h)).
The additional element of “storing explanation information correlated to each of the feature groups in a storage device of the machine learning platform system” is recited at a high level of generality and amounts to insignificant extra-solution activity of storing data. The courts have found limitations directed to storing information, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), "electronic record keeping," and "storing and retrieving information in memory").
The additional elements of “receiving from an operator device an input row … providing to the modelling system the input row … providing the model output to the application server and the model evaluation system … providing to the application server the explanation information corresponding to the one of the feature groups and comprising human-readable explanatory text associated with an impact on the model output of at least one feature of the one of the feature groups … providing the adverse decision and the explanation information corresponding to the one of the feature groups to the operator device in response to the input row” are recited at a high level of generality and amounts to insignificant extra-solution activity related to mere data gathering and transmitting (MPEP 2106.05(g)). The courts have found limitations directed to receiving and transmitting information electronically, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), “receiving or transmitting data over a network”).
The Applicant then argues that “The independent claims with these amended limitations provides an improvement to machine learning model evaluation technology by improving the accuracy and efficiency with which machine learning model explanations are provided … Further, the amended independent claims recite particular computing devices including an application server that provides an input row received from an operator device to a modelling system … The modelling system applies the credit model to the input row to generate the output and provides the output to the application server and the model evaluation system. A model evaluation system, inter alia, identifies a feature group related the output …”. The examiner respectfully disagrees. Even though the claim recites how the machine learning platform is operated, using multiple computer components such as an application server, the modelling system and the model evaluation system, to generate the decision and the explanation for the model result, however, the claim does not recite how the machine learning platform is operated to improve the machine learning technology. As mentioned above and in the 101 rejections section below, an improvement in an abstract idea itself (generating the decision and the explanation) is not an improvement in technology.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 1, 4, 6, 10, 13-15, 17-18, 20, 22, 24 and 26-33 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites the limitation of “the predictive machine learning model”. There is insufficient antecedent basis for this limitation in the claim.
Claim 1 recites the limitation of “the modeling evaluation system”. There is insufficient antecedent basis for this limitation in the claim.
Claims 14-15 and 17 recite the limitation of “the predictive machine learning model”. There is insufficient antecedent basis for this limitation in the claim.
Claim 18 recites the limitation of “the predictive machine learning model”. There is insufficient antecedent basis for this limitation in the claim.
Claim 22 recites the limitation of “the predictive machine learning model”. There is insufficient antecedent basis for this limitation in the claim.
Claim 22 recites the limitation of “the modeling evaluation system”. There is insufficient antecedent basis for this limitation in the claim.
Claims 32-33 recite the limitation of “the predictive machine learning model”. There is insufficient antecedent basis for this limitation in the claim.
Claims 4, 6, 10, 13 and 17 are rejected for being dependent on a rejected base claim, namely claim 1.
Claim 20, 29-31 and 33 are rejected for being dependent on a rejected base claim, namely claim 18.
Claims 24, 26-28 and 32 are rejected for being dependent on a rejected base claim, namely claim 22.
Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 4, 6, 10, 13-15, 17-18, 20, 22, 24 and 26-33 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
Claim 1 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1:
The claim recites a method which falls within at least one of the four statutory categories of patent eligible subject matter.
Step 2:
Step 2A (prong 1):
The limitation of “upon determining that the predictive machine learning model has been re-trained, with the modeling evaluation system”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, “determining” in the context of this claim encompasses the user determines to train a model.
The limitation of “obtaining evaluation input rows, each representing a positive decision by the predictive machine learning model, and reference input rows, each representing a negative decision by the predictive machine learning model”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, “obtaining” in the context of this claim encompasses the user collecting data to analyze.
The limitation of “for each of a plurality of features identified in the evaluation input rows, determining a distribution of feature contribution values across a set of model outputs”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, “determining” in the context of this claim encompasses the user assigning values to the elements/features.
The limitation of “for each pair of features among the identified features, determining a similarity metric value based on a difference between one of the distributions of feature contribution values determined for a first feature of the pair of features and another of the distributions of feature contribution values determined for a second feature of the pair of features”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, “determining” in the context of this claim encompasses the user, for each pair of features, identifying a value that is similar.
The limitation of “constructing a graph that comprises nodes representing each of the identified features and edges representing each of the similarity metric values”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, “constructing” in the context of this claim encompasses the user creates a graph having nodes and edges connecting the nodes.
The limitation of “identify node clusters of the graph based on at least one of the similarity metric values assigned to each of the graph edges”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, “identify” in the context of this claim encompasses the user grouping the nodes.
The limitation of “identifying one of the feature groups related to the model output, responsive to the input row, and determined based on the similarity metric values”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, “identifying” in the context of this claim encompasses the user grouping data based on the similarity values.
The limitation of “retrieving from the storage device the explanation information corresponding to the one of the feature groups”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, “retrieving” in the context of this claim encompasses the user extracting/selecting information corresponding to a certain class.
The limitation of “generating an adverse decision based on the model output”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, “generating” in the context of this claim encompasses the user makes a decision based on some factors.
The above steps are based on observations, evaluations, judgments or opinion that are performable in the human mind or with the aid of pencil and paper (see MPEP 2106.04(a)(2)(III). That is, other than reciting “a machine learning platform system”, “an application server”, “a modelling system” and “a model evaluation system”, nothing in the claim element precludes the steps from practically being performed in the mind. Therefore, the claim recites an abstract idea.
Step 2A (prong 2):
This judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of “a machine learning platform system”, “an application server”, “a modelling system” and “a model evaluation system”. These additional elements are recited at a high-level of generality (i.e., as a generic device performing the generic computer functions) such that they amount no more than mere instructions to apply the exception using the generic computer components (MPEP 2106.05(f)). Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claim recites the additional elements of “invoking one or more functions of the modeling system to obtain model access information comprising at least a tree structure of the predictive machine learning model”, “applying the predictive machine learning model to the input row to generate a model output”. These limitations are recited at a high-level of generality (i.e., as a generic device performing the generic computer functions) such that they amount to no more than mere instructions to apply the exception using the generic computer components (MPEP 2106.05(f)). Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claim recites the additional elements of “determining a distribution of feature contribution values across a set of model outputs using the tree structure, the evaluation input rows, and the reference input rows, wherein the distribution of feature contribution values for each of the identified features represents an influence of the identified feature on first scores for the evaluation input rows … wherein the similarity metric value quantifies a similarity between the first and second features for each of the pair of features … wherein each of the node clusters represents a feature group and each of the feature groups comprises a subset of the features”. These limitations amount to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitation that amounts to merely indicating a field of use or technological environment in which to apply a judicial exception does not integrate into a practical application (see MPEP 2106.05(h)).
The claim recites the additional element of “storing explanation information correlated to each of the feature groups in a storage device of the machine learning platform system” The store step is recited at a high level of generality and amounts to mere data storing, which has been identified as a type of limitation that is considered an insignificant extra-solution activity. As described in MPEP 2106.05(g), limitations that amount to merely adding insignificant extra-solution activity to a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
The additional elements of “receiving from an operator device an input row … providing to the modelling system the input row … providing the model output to the application server and the model evaluation system … providing to the application server the explanation information corresponding to the one of the feature groups and comprising human-readable explanatory text associated with an impact on the model output of at least one feature of the one of the feature groups … providing the adverse decision and the explanation information corresponding to the one of the feature groups to the operator device in response to the input row” amount to insignificant extra-solution activities of data gathering and transmitting which do not amount to significantly more than the abstract idea (MPEP 2106.05(g)). Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of “a machine learning platform system”, “an application server”, “a modelling system” and “a model evaluation system” to perform the computer functions amount to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept.
The additional elements of “invoking one or more functions of the modeling system to obtain model access information comprising at least a tree structure of the predictive machine learning model”, “applying the predictive machine learning model to the input row to generate a model output” amount to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept.
The additional elements of “determining a distribution of feature contribution values across a set of model outputs using the tree structure, the evaluation input rows, and the reference input rows, wherein the distribution of feature contribution values for each of the identified features represents an influence of the identified feature on first scores for the evaluation input rows … wherein the similarity metric value quantifies a similarity between the first and second features for each of the pair of features … wherein each of the node clusters represents a feature group and each of the feature groups comprises a subset of the features” amount to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitation that amounts to merely indicating a field of use or technological environment in which to apply a judicial exception does not amount to significantly more than the judicial exception (see MPEP 2106.05(h)).
The additional element of “storing explanation information correlated to each of the feature groups in a storage device of the machine learning platform system” is recited at a high level of generality and amounts to insignificant extra-solution activity of storing data. The courts have found limitations directed to storing information, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), "electronic record keeping," and "storing and retrieving information in memory").
The additional elements of “receiving from an operator device an input row … providing to the modelling system the input row … providing the model output to the application server and the model evaluation system … providing to the application server the explanation information corresponding to the one of the feature groups and comprising human-readable explanatory text associated with an impact on the model output of at least one feature of the one of the feature groups … providing the adverse decision and the explanation information corresponding to the one of the feature groups to the operator device in response to the input row” are recited at a high level of generality and amounts to insignificant extra-solution activity related to mere data gathering and transmitting (MPEP 2106.05(g)). The courts have found limitations directed to receiving and transmitting information electronically, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), “receiving or transmitting data over a network”).
Claim 4 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1:
The claim recites the method which falls within at least one of the four statutory categories of patent eligible subject matter.
Step 2:
Step 2A (prong 1):
The limitation of “determine one or more of the similarity metric values”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, “determine” in the context of this claim encompasses the user determining if the two values are the same.
Step 2A (prong 2):
This judicial exception is not integrated into a practical application. The claim recites an additional element of “performing a Kolmogorov- Smirnov test or computing at least one Pearson correlation coefficient to determine …”. This limitation amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitation that amounts to merely indicating a field of use or technological environment in which to apply a judicial exception does not integrate into a practical application (see MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
The additional element of “performing a Kolmogorov- Smirnov test or computing at least one Pearson correlation coefficient to determine …” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitation that amounts to merely indicating a field of use or technological environment in which to apply a judicial exception does not amount to significantly more than the judicial exception (see MPEP 2106.05(h)).
Claim 6 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1:
The claim recites the method which falls within at least one of the four statutory categories of patent eligible subject matter.
Step 2:
Step 2A (prong 1):
The limitation of “sampling the evaluation or reference input rows from at least one dataset”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, “sampling” in the context of this claim encompasses the user analyzing the data from a dataset.
Step 2A (prong 2):
This judicial exception is not integrated into a practical application. The claim recites an additional element of “iteratively sampling the evaluation or reference input rows … until a sampling metric computed for a current sample indicates that results generated by using the current sample are likely to have an accuracy above an accuracy threshold”. This limitation amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitation that amounts to merely indicating a field of use or technological environment in which to apply a judicial exception does not integrate into a practical application (see MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
The additional element of “iteratively sampling the evaluation or reference input rows … until a sampling metric computed for a current sample indicates that results generated by using the current sample are likely to have an accuracy above an accuracy threshold” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitation that amounts to merely indicating a field of use or technological environment in which to apply a judicial exception does not amount to significantly more than the judicial exception (see MPEP 2106.05(h)).
Claim 10 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1:
The claim recites the method which falls within at least one of the four statutory categories of patent eligible subject matter.
Step 2:
Step 2A (prong 1):
The limitation of “identifying other features”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, “identifying” in the context of this claim encompasses the user determining other features that are different with the identified features. Similarly, the limitation of “identifying the human-readable explanatory text for another one or more of the feature groups that include the identified other features”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, “identifying” in the context of this claim encompasses the user identifies a text created by a human that explains why the loan application was denied for example. Similarly, the limitation of “generating the explanation information by using the identified human-readable explanatory text”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, “generating” in the context of this claim encompasses the user provides an explanation why the loan application was denied.
Step 2A (prong 2):
This judicial exception is not integrated into a practical application. The claim recites an additional element of “other features having other feature contribution values generated for the output that exceed one or more associated thresholds”. This limitation amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitation that amounts to merely indicating a field of use or technological environment in which to apply a judicial exception does not integrate into a practical application (see MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
The additional element of “other features having other feature contribution values generated for the output that exceed one or more associated thresholds” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitation that amounts to merely indicating a field of use or technological environment in which to apply a judicial exception does not amount to significantly more than the judicial exception (see MPEP 2106.05(h)).
Claim 13 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1:
The claim recites the method which falls within at least one of the four statutory categories of patent eligible subject matter.
Step 2:
Step 2A (prong 2):
This judicial exception is not integrated into a practical application. The claim recites an additional element of “wherein the node clustering process comprises a hierarchical agglomerative clustering process or identifying a clique in the graph in which each associated edge has one of the similarity metrics above a threshold”. This limitation amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitation that amounts to merely indicating a field of use or technological environment in which to apply a judicial exception does not integrate into a practical application (see MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
The additional element of “wherein the node clustering process comprises a hierarchical agglomerative clustering process or identifying a clique in the graph in which each associated edge has one of the similarity metrics above a threshold” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitation that amounts to merely indicating a field of use or technological environment in which to apply a judicial exception does not amount to significantly more than the judicial exception (see MPEP 2106.05(h)).
Claim 14 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1:
The claim recites the method which falls within at least one of the four statutory categories of patent eligible subject matter.
Step 2:
Step 2A (prong 2):
This judicial exception is not integrated into a practical application. The claim recites an additional element of “the predictive machine learning model includes at least a gradient boosted tree forest (GBM) coupled to base signals and a smoothed approximate empirical cumulative distribution function (ECDF) coupled to other output of the GBM and output values of the GBM are transformed by using the ECDF”. This limitation amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitation that amounts to merely indicating a field of use or technological environment in which to apply a judicial exception does not integrate into a practical application (see MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
The additional element of “the predictive machine learning model includes at least a gradient boosted tree forest (GBM) coupled to base signals and a smoothed approximate empirical cumulative distribution function (ECDF) coupled to other output of the GBM and output values of the GBM are transformed by using the ECDF” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitation that amounts to merely indicating a field of use or technological environment in which to apply a judicial exception does not amount to significantly more than the judicial exception (see MPEP 2106.05(h)).
Claim 15 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1:
The claim recites the method which falls within at least one of the four statutory categories of patent eligible subject matter.
Step 2:
Step 2A (prong 2):
This judicial exception is not integrated into a practical application. The claim recites an additional element of “determine the distribution of feature contribution values across the set of model outputs by generating permutations of input values and observing score changes based on applications of the predictive machine learning model to the input values, computing one or more gradients, computing one or more SHapley Additive exPlanations (SHAP) values, or determining contribution values at model discontinuities identified based on the tree structure”. This limitation amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitation that amounts to merely indicating a field of use or technological environment in which to apply a judicial exception does not integrate into a practical application (see MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
The additional element of “determine the distribution of feature contribution values across the set of model outputs by generating permutations of input values and observing score changes based on applications of the predictive machine learning model to the input values, computing one or more gradients, computing one or more SHapley Additive exPlanations (SHAP) values, or determining contribution values at model discontinuities identified based on the tree structure” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitation that amounts to merely indicating a field of use or technological environment in which to apply a judicial exception does not amount to significantly more than the judicial exception (see MPEP 2106.05(h)).
Claim 17 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1:
The claim recites the method which falls within at least one of the four statutory categories of patent eligible subject matter.
Step 2:
Step 2A (prong 2):
This judicial exception is not integrated into a practical application. The claim recites an additional element of “the predictive machine learning model includes at least a neutral network (NN), a gradient boosted tree forest (GBM), and a neural network ensembling module and another output of the neural network ensembling module is processed by a differentiable function”. This limitation amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitation that amounts to merely indicating a field of use or technological environment in which to apply a judicial exception does not integrate into a practical application (see MPEP 2106.05(h)).
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
The additional element of “the predictive machine learning model includes at least a neutral network (NN), a gradient boosted tree forest (GBM), and a neural network ensembling module and another output of the neural network ensembling module is processed by a differentiable function” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitation that amounts to merely indicating a field of use or technological environment in which to apply a judicial exception does not amount to significantly more than the judicial exception (see MPEP 2106.05(h)).
Claim 18 is rejected by the same reason as of claim 1, since these claims recite the similar limitations.
The claim recites the additional limitations of “one or more processors” and “memory”.
Step 2A (prong 2):
This judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of “an application server”, “a modelling system”, “a model evaluation system”, “one or more processors” and “memory”. These additional elements are recited at a high-level of generality (i.e., as a generic device performing the generic computer functions) such that they amount no more than mere instructions to apply the exception using the generic computer components (MPEP 2106.05(f)). Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of “an application server”, “a modelling system”, “a model evaluation system”, “one or more processors” and “memory” to perform the computer functions amount to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept.
Claims 20 and 24 are rejected by the same reason as of claim 10, since these claims recite the similar limitations.
Claim 22 is rejected by the same reason as of claim 18, since these claims recite the similar limitations.
Claims 26 and 29 are rejected by the same reason as of claim 4, since these claims recite the similar limitations.
Claims 27 and 30 are rejected by the same reason as of claim 6, since these claims recite the similar limitations.
Claims 28 and 31 are rejected by the same reason as of claim 13, since these claims recite the similar limitations.
Claims 32 and 33 are rejected by the same reason as of claim 15, since these claims recite the similar limitations.
Allowable Subject Matter
Claims 1, 4, 6, 10, 13-15, 17-18, 20, 22, 24 and 26-33 would be allowable if rewritten so that the 101 and 112 (b) rejections above were overcome.
The following is a statement of reasons for the indication of allowable subject matter:
Claim 1 is allowable for disclosing
A method for automated explanation of complex machine learning model decisioning, the method implemented by a machine learning platform system comprising an application server, a modelling system, and a model evaluation system coupled together via one or more networks, the method comprising:
upon determining that the predictive machine learning model has been re-trained, with the modeling evaluation system:
obtaining evaluation input rows, each representing a positive decision by the predictive machine learning model, and reference input rows, each representing a negative decision by the predictive machine learning model;
invoking one or more functions of the modeling system to obtain model access information comprising at least a tree structure of the predictive machine learning model;
for each of a plurality of features identified in the evaluation input rows, determining a distribution of feature contribution values across a set of model outputs using the tree structure, the evaluation input rows, and the reference input rows, wherein the distribution of feature contribution values for each of the identified features represents an influence of the identified feature on first scores for the evaluation input rows;
for each pair of features among the identified features, determining a similarity metric value based on a difference between one of the distributions of feature contribution values determined for a first feature of the pair of features and another of the distributions of feature contribution values determined for a second feature of the pair of features, wherein the similarity metric value quantifies a similarity between the first and second features for each of the pair of features;
constructing a graph that comprises nodes representing each of the identified features and edges representing each of the similarity metric values;
performing a node clustering process to identify node clusters of the graph based on at least one of the similarity metric values assigned to each of the graph edges, wherein each of the node clusters represents a feature group and each of the feature groups comprises a subset of the features; and
storing explanation information correlated to each of the feature groups in a storage device of the machine learning platform system;
with the application server:
receiving from an operator device an input row; and
providing to the modelling system the input row;
with the modelling system:
applying the predictive machine learning model to the input row to generate a model output; and
providing the model output to the application server and the model evaluation system;
with the model evaluation system:
identifying one of the feature groups related to the model output, responsive to the input row, and determined based on the similarity metric values;
retrieving from the storage device the explanation information corresponding to the one of the feature groups; and
providing to the application server the explanation information corresponding to the one of the feature groups and comprising human-readable explanatory text associated with an impact on the model output of at least one feature of the one of the feature groups; and
with the application server:
generating an adverse decision based on the model output; and
providing the adverse decision and the explanation information corresponding to the one of the feature groups to the operator device in response to the input row.
The closest references found
Lahrichi et al. (US Pub. 2020/0134716) in Figs. 4-6, abstract, paragraphs 0097-0130, teaches a system and method of predicting whether the loan application is denied or approved using the machine learning algorithm, wherein, the system comprises a server system, a predictive model and an evaluation model.
Pai et al. (US Pub. 2020/0279140) in abstract and paragraphs 0023-0024 teaches a process of using a credit model to generate the scores for the features that indicating how important particular features are to a machine learning model determining a particular output.
Zhu et al. (US Pub. 2014/0172371) in paragraphs 0026-0043 teaches a process of generating a graph comprising nodes and edges representing features and similarity metric values.
Marwah et al. (US Pub. 2019/0303716) in paragraphs 0040-0042 teaches “The raw contribution of the value of each input feature of the point of interest to the output score provided by the machine learning model for the point of interest is determined (302), using the feature contribution coefficient for the input feature in question”.
Pai et al. (US Pub. 2020/0193234) in paragraph 0100 teaches “an output of the machine learning model(s) may be coefficients associated with each of the input data types (e.g., the features of the input data), where the coefficients represent the relative contribution of the input data types with respect to the output data and/or the corresponding performance”
However, the prior art of record do not teach or suggest, individually or in combination the claim limitations as a whole, especially the limitations:
invoking one or more functions of the modeling system to obtain model access information comprising at least a tree structure of the predictive machine learning model;
for each of a plurality of features identified in the evaluation input rows, determining a distribution of feature contribution values across a set of model outputs using the tree structure, the evaluation input rows, and the reference input rows, wherein the distribution of feature contribution values for each of the identified features represents an influence of the identified feature on first scores for the evaluation input rows;
Therefore, the combination of features is considered to be allowable.
Claims 4, 6, 10, 13-15 and 17 are considered to be allowable because they are dependent on claim 1.
Claims 18 and 22 are considered to be allowable for disclosing the similar subject matter to claim 1.
Claims 20, 29-31 and 33 are considered to be allowable because they are dependent on claim 18.
Claims 24, 26-28 and 32 are considered to be allowable because they are dependent on claim 22.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TRI T NGUYEN whose telephone number is 571-272-0103. The examiner can normally be reached M-F, 8 AM-5 PM, (CT).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, OMAR FERNANDEZ can be reached at 571-272-2589. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TRI T NGUYEN/Examiner, Art Unit 2128
/OMAR F FERNANDEZ RIVAS/Supervisory Patent Examiner, Art Unit 2128