Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 9/19/2022 and 4/19/2023 are in compliance with the provisions of 37 CFR 1.97 and 1.98. Accordingly, the information disclosure statements are considered by the examiner.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 6-7 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim 6 recites the limitation of “multiplying the integrated gradient of the determination processor with respect to the plurality of outputs by (the integrated gradient of the event processor with respect to the plurality of data elements divided by the plurality of outputs)”. According to [0031] and [0035] of applicant’s specification, integrated gradients is an algorithm used to identify feature importance of features in AI models. The determination processor and event process are not described as AI models and the specification does not provide written description of a process for feature selection of the determination and event processors.
According to the specification, the event processor groups outputs into events (see [0040]). The specification does not provide any details on the implementation of the event processor. Therefore, the specification fails to provide written description for an event processor implemented as an AI model to group outputs of AI models into events and fails to provide written description for determining the integrated gradient of the event processor.
Next, the specification describes the determination processor as receiving a plurality of events and determining a probability of the first data structure being associated with characterization output. The specification also fails to provide written description for a determination processor implemented as an AI model to determine a probability of the first data structure being associated with characterization output and fails to provide written description for determining the integrated gradient of the determination processor.
The recitation of an integrated gradient of the determination processor and event processor fails to comply with written description requirement. For the purpose of examination, these features are interpreted as integrated gradient of the one or more models and integrated gradients of models with subpopulation of the plurality of data elements (consistent with the features of claim 8).
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 10 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 10 recites the following mathematical formula for calculating the integrated gradient of a model:
PNG
media_image1.png
94
286
media_image1.png
Greyscale
However, claim 10 does not define the variables of the recited formula. For example, the variables w, x, t, aw, ax, dx, and dt are not defined in the claim. Since the variables are not defined, the metes and bounds of the claimed invention are not clear. Examiner notes that the specification of the instant application also does not define the variables and only discloses the recited formula without further explanation.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-17 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
MPEP 2106 (III) sets out steps for evaluating whether a claim is drawn to patent-eligible subject matter. The analysis of claims 1-7, in accordance with these steps, follows.
Step 1 Analysis:
Claims 1 – 10 are directed to a method, claims 11 – 17 are directed to a system comprising of hardware processor and memory. Therefore, claims 1-17 fall into one of four statutory categories (i.e., process, machine, article of manufacture, or composition of matter).
As to claim 1,
Step 2A Prong 1: this claim recites the following abstract ideas:
using a cascade of models with integrated gradients to identify a feature importance value for each of the plurality of features (this limitation is interpreted as using integrated gradients (i.e. a calculated value) of cascade models to identify feature importance values, which is a mental process implementable in a human mind);
based on the feature importance value identified for each feature included in the plurality of features, determining a feature importance metric level (determining a feature importance metric level based on the feature importance value is a mental process implementable in a human mind);
based on the feature importance value identified for each feature included in the plurality of features, removing one or more features, from the plurality of features, that are assigned a feature importance value that is less than the feature importance metric level to form a revised AI-based model (determining features to remove based on the metric level and a threshold is a mental process of comparison and evaluation).
Step 2A Prong 2 and 2B: this claim recites the following additional elements:
receiving an AI-based model, said AI-based model being trained with a
plurality of training data elements, said AI-based model identifying a plurality of features
from the plurality of training data elements, said AI-based model executing with respect to a
first input (This limitation covers receiving (i.e. accessing) trained AI model, the function of the model and executing the model using a first input. These are generic computer functions of accessing the trained AI model from memory and executing the AI model to perform a function of identifying features based on input. The use of the AI model to identify features is also directed to mere instruction to apply an abstract idea on a generic computer. The additional limitation does not integrate the judicial exception into a practical application and does not amount to significantly more than the judicial exception. See MPEP 2106.05(f) for mere instruction to apply an exception and 2106.05(d)(II)(iv) which describes retrieving information from memory as well-understood, routine, conventional activity);
executing the revised AI-based model with respect to a second input (This is also directed to mere instruction to apply the abstract idea on a generic computer. The additional limitation does not integrate the judicial exception into a practical application and does not amount to significantly more than the judicial exception. See MPEP 2106.05(f)).
As to claim 6
Step 2A Prong 1: this claim recites the following abstract ideas:
identifying a plurality of data elements associated with the first data structure (the limitation of identify data is a mental process implementable in a human mind);
grouping the plurality of outputs into a plurality of events at the event processor (The step of grouping data is a mental process of sorting data based on a criterion, which is a mental process. The use of an event processor is interpreted as mere instruction to apply the abstract idea on a generic computer.);
determining, at the determination processor, a probability of the first data structure being associated with the characterization output (The step of determining a probability is a mental process and the use of a determination processor is directed to mere instructions to apply the abstract idea on a generic computer);
in order to remove a predetermined number of data elements from the plurality of data elements, said predetermined number of data elements that are detrimental to the characterization output (This is directed to a mental process of identifying a number of data elements to remove.):
multiplying the integrated gradient of the determination processor with respect to the plurality of outputs by (the integrated gradient of the event processor with respect to the plurality of data elements divided by the plurality of outputs), which results in a vector of:
a subset of the plurality of data elements; and
a probability that each data element, included in the subset of data elements, contributed to the characterization output (The step of multiplying gradients and the resulting vector is directed to a mathematical concept.);
removing one or more data elements from the subset of the plurality of data elements that are associated with a probability that is less than a probability threshold to form an updated subset of the plurality of data elements (determining features to remove based on the metric level and a threshold is a mental process of comparison and evaluation);
re-grouping the plurality of outputs into the plurality of events at the event processor (The step of re-grouping data is a mental process of sorting data based on a criterion, which is a mental process. The use of an event processor is interpreted as mere instruction to apply the abstract idea on a generic computer.);
re-determining, at the determination processor, the probability of the first data structure being associated with the characterization output (The step of re-determining a probability is a mental process and the use of a determination processor is directed to mere instructions to apply the abstract idea on a generic computer).
Step 2A Prong 2 and 2B: this claim recites the following additional elements:
on a first iteration:
receiving a characterization output characterizing a first data structure (This limitation directed to retrieving information from memory, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(iv).);
feeding the plurality of data elements into one or more models (This limitation is directed to transmitting data into the models, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i));
processing the plurality of data elements at the one or more models (This limitation is directed to mere instruction to apply the abstract idea on a generic computer to process the data elements, see MPEP 2106.05(f));
identifying a plurality of outputs from the one or more models (This limitation directed to retrieving output from memory, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(iv).);
feeding the plurality of outputs into an event processor (This limitation is directed to transmitting data into the event processor, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i));
processing the plurality of outputs at the event processor (This limitation is directed to mere instruction to apply the abstract idea on a generic computer to process the data elements, see MPEP 2106.05(f));
inputting the plurality of events into a determination processor (This limitation is directed to transmitting data into the determination processor, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i));
on a second iteration (The recitation of the first and second iteration is directed to performing repetitive calculation, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(ii));
re-feeding the updated subset of the plurality of data elements into the one or more models (This limitation is directed to transmitting data into the models, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i));
re-processing the plurality of data elements at the one or more models (This limitation is directed to mere instruction to apply the abstract idea on a generic computer to process the data elements, see MPEP 2106.05(f));
re-identifying the plurality of outputs from the one or more models (This limitation directed to retrieving output from memory, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(iv).);
re-feeding the plurality of outputs into the event processor (This limitation is directed to transmitting data into the event processor, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i));
re-processing the plurality of outputs at the event processor (This limitation is directed to mere instruction to apply the abstract idea on a generic computer to process the data elements, see MPEP 2106.05(f));
re-inputting the plurality of events into the determination processor (This limitation is directed to transmitting data into the determination processor, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i));
utilizing the one or more models to characterize unlabeled data elements (This limitation is directed to mere instruction to apply the abstract idea on a generic computer to process the data elements, see MPEP 2106.05(f)).
The additional limitations do not integrate the judicial exception into a practical application and do not amount to significantly more than the judicial exception.
As to Claim 8
Step 2A Prong 1: this claim recites the following abstract ideas:
identifying a plurality of data elements associated with the first data structure (the
limitation of identify data is a mental process implementable in a human mind);
determining a probability of the first data structure being associated with the characterization output (The step of determining a probability is a mental process);
in order to remove a predetermined number of data elements from the plurality of data elements, said predetermined number of data elements that are detrimental to the characterization output (This is directed to a mental process of identifying a number of data elements to remove.):
multiplying the integrated gradient of the one or more models with respect to the plurality of outputs by (the integrated gradient of the one or more models with respect to the plurality of data elements divided by the plurality of outputs), which results in a vector of: a subset of the plurality of data elements; and a probability that each data element, included in the subset of
data elements, contributed to the characterization output (The step of multiplying gradients and the resulting vector is directed to a mathematical concept.);
removing one or more data elements from the subset of the plurality of data elements that are associated with a probability that is less than a probability threshold to generate an updated subset of the plurality of data elements (determining features to remove based on the metric level and a threshold is a mental process of comparison and evaluation).
Step 2A Prong 2 and 2B: this claim recites the following additional elements:
on a first iteration:
receiving a characterization output characterizing a first data structure (This limitation
directed to retrieving information from memory, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(iv).);
feeding the plurality of data elements into one or more models (This limitation is directed
to transmitting data into the models, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i));
processing the plurality of data elements at the one or more models (This limitation is directed to mere instruction to apply the abstract idea on a generic computer to process the data elements, see MPEP 2106.05(f));
identifying a plurality of outputs from one or more models (This limitation directed to
retrieving output from memory, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(iv).);
on a second iteration (The recitation of the first and second iteration is directed to performing repetitive calculation, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(ii)):
re-feeding the updated subset of the plurality of data elements into the one or more models (This limitation is directed to transmitting data into the models, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i));
re-processing the plurality of data elements at the one or more models (This limitation is
directed to mere instruction to apply the abstract idea on a generic computer to process the data elements, see MPEP 2106.05(f));
re-identifying the plurality of outputs from one or more models (This limitation
directed to retrieving output from memory, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(iv).); and
utilizing the one or more models to characterize unlabeled data elements (This limitation is directed to mere instruction to apply the abstract idea on a generic computer to process the data elements, see MPEP 2106.05(f)).
The additional limitations do not integrate the judicial exception into a practical application and do not amount to significantly more than the judicial exception.
As to claim 11,
Step 2A Prong 1: this claim recites the following abstract ideas:
identify a plurality of features that characterize a data element set as being associated with the predetermined label (the limitation of identify data is a mental process implementable in a human mind);
create, using the plurality of features, an artificially-intelligent model that can characterize an unlabeled data element set as being associated with the predetermined label (This limitation is directed to defining and generating a model using identified features, which is a mental process implementable with the aide of pen and paper.);
assign, using an algorithm, a value to each feature included in the plurality of features (determining a value to each feature is a mental process implementable in a human mind);
remove, from the artificially-intelligent model, features that have been assigned a value that is less than a predetermined threshold to form a revised artificially-intelligent model (determining features to remove based on the metric level and a threshold is a mental process of comparison and evaluation); and
recreate the revised artificially-intelligent model that can characterize an unlabeled data element set as being associated with the predetermined label (This limitation is directed to modifying and re-generating a model using modified feature set, which is a mental process implementable with the aide of pen and paper.).
Step 2A Prong 2 and 2B: this claim recites the following additional elements:
a priming model module operating on a hardware processor and a memory, the priming model module operable to (The module, hardware processor and memory are directed to generic computer components for implementing the abstract ideas identified in the second above. This limitation is directed to mere instruction to apply the abstract idea on a generic computer to process the data elements, see MPEP 2106.05(f)):
receive a training data set, said training data set comprising a plurality of data element sets and a predetermined label associated with each of the data elements sets (This limitation
directed to retrieving information from memory, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(iv).);
a refining model module operating on the hardware processor and the memory, the refining model module operable to (The module, hardware processor and memory are directed to generic computer components for implementing the abstract ideas identified in the second above. This limitation is directed to mere instruction to apply the abstract idea on a generic computer to process the data elements, see MPEP 2106.05(f)).
The additional limitations do not integrate the judicial exception into a practical application and do not amount to significantly more than the judicial exception.
As to claims 2 and 14
The limitation further defines the type of information used (i.e. “a percentage of the plurality of features”) determine features to remove. Therefore, claims 2 and 14 also recites a mental process. The claims do not recite any additional elements.
As to claims 3 and 15
The limitation further defines the type of information used (i.e. “predetermined number of the plurality of features”) determine features to remove. Therefore, claims 3 and 15 also recites a mental process. The claims do not recite any additional elements.
As to claims 4 and 16
The limitation further defines the type of information used (i.e. “predetermined value assigned to the plurality of features”) determine features to remove. Therefore, claims 4 and 16 also recites a mental process. The claims do not recite any additional elements.
As to claims 5 and 17
The limitation further defines the type of information used (i.e. “predetermined value corresponds to a negative value”) determine features to remove. Therefore, claims 5 and 17 also recites a mental process. The claims do not recite any additional elements.
As to claims 7, 9, and 13
The claims do not recite any additional abstract ideas. The limitation, “wherein the first iteration is re-executed until all of the data elements are assigned a probability that is greater than the probability threshold” is directed to repetitive calculation/computing, which is a well‐understood, routine, and conventional functions (see MPEP 2106.05(d)(II)(ii)).
As to claim 10
The limitation “wherein an equation for determining the integrated gradient of the one or more models with respect to the plurality of outputs is:”
PNG
media_image2.png
66
250
media_image2.png
Greyscale
is directed to a mathematical formula. The claim does not recite any additional elements.
As to claim 12
The limitation “wherein the algorithm is Integrated Gradients, Cascaded Integrated Gradients, SHAP or TreeSHAP” are directed to various calculation used to determine feature importance. These various algorithm of calculating feature importance are mental processes performed with the aide of pen and paper. The claim does not recite any additional elements.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-3 and 11 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by US 20220405623 A1 (hereinafter Cheng).
As to claim 1, Cheng teaches a method for harnessing an explainable artificial intelligence system to execute computer-aided feature selection, the method comprising (see [0003], “a query-driven computing platform for generating feature attributions and other model explanation data.”);
receiving an AI-based model, said AI-based model being trained with a plurality of training data elements, said AI-based model identifying a plurality of features from the plurality of training data elements, said AI-based model executing with respect to a first input (see [0039], “The preprocessing engine 110 of the platform 100 can be configured for preprocessing data selected from the storage devices 150. For example, preprocessing can include data normalization and formatting to bring the selected data to a form suitable for processing by the training engine 120. The preprocessing engine 110 can also be configured for feature selection/engineering, and/or removing or adding features to the input data according to any of a variety of different approaches. Parameters for feature selection and/or engineering can be received from user input, for example for preprocessing training data before training a model. The preprocessing engine 110 can encode categorical features, e.g., using one-hot encoding, dummy encoding, and/or target coding, etc. In some examples, the preprocessing engine 110 can add embedding layers to a received machine learning model.”);
using a cascade of models with integrated gradients to identify a feature importance value for each of the plurality of features (see [0088] and [0090], “The shard explanation engine 330 can execute one or more explainers 335A-N. In FIG. 3, the shard explanation engine 330 is shown as including two explainers A, N 335A, N, although in other examples the shard explanation engine 330 can include fewer or more explainers. An explainer can be implemented in software and/or hardware and be configured to process the machine learning model and input data to generate local or global explanations, for example as described herein with reference to FIG. 1 and the explanation engine 130. The shard explanation engine 330 can receive model data, and from the model data and corresponding parameters, generate each explainer, and generate model explanation data for received input data.”, see also [0091] for discussion of explainers implementing integrated gradients);
based on the feature importance value identified for each feature included in the plurality of features, determining a feature importance metric level (see [0058], “Integrated gradients can have the property that the feature attributions sum to the prediction difference between the input score and the baseline score.”);
based on the feature importance value identified for each feature included in the plurality of features, removing one or more features, from the plurality of features, that are assigned a feature importance value that is less than the feature importance metric level to form a revised AI-based model (see [0062], “In some examples, the evaluation engine 140 can automatically select the top feature attributions that explain some predetermined threshold, e.g., 80%, of the model prediction.”); and
executing the revised AI-based model with respect to a second input (see [0063], “The evaluation engine 140 can implement a graphical user interface, e.g., as one or more web pages, as an application installed on a user device, etc., for presenting and receiving data from a user device. In response to providing the model predictions and model explanations, the evaluation engine 140 can receive additional query statements, e.g., for re-training the model or for generating model explanation data according to different approaches or parameters than what was previously specified”).
As to claim 11, Cheng teaches a computing resource conservation system comprising (see [0003], “a query-driven computing platform for generating feature attributions and other model explanation data”):
a priming model module operating on a hardware processor and a memory, the priming model module operable to (see [0066] “The server computing device 215 can include one or more processors 213 and memory 214” and see [0034], “The platform 100 can include server devices communicating with each other and one or more user devices over a network”):
receive a training data set, said training data set comprising a plurality of data element sets and a predetermined label associated with each of the data elements sets (see [0029], “In addition, the platform can receive, through the one or more query statements, parameters for generating model explanation data. Model explanation data can include local and global explanations. An explanation can be any data that at least partially characterizes a relationship between the output of the model, with either the input data used to generate the model, or with the model itself”);
identify a plurality of features that characterize a data element set as being associated with the predetermined label (see [0039], “For example, preprocessing can include data normalization and formatting to bring the selected data to a form suitable for processing by the training engine 120. The preprocessing engine 110 can also be configured for feature selection/engineering, and/or removing or adding features to the input data according to any of a variety of different approaches.”);
create, using the plurality of features, an artificially-intelligent model that can characterize an unlabeled data element set as being associated with the predetermined label (see [0036] and [0040], “The platform 100 can implement a number of different machine learning models, which the platform 100 can train and process data at inference from data stored on the one or more storage devices 150”);
a refining model module operating on the hardware processor and the memory (see [0034], “The platform 100 can also implement one or more processing shards 135”), the refining model module operable to (see [0089], “In those examples, in retrieving the model data, the shard explanation engine 330 is configured to retrieve the individual pieces of the model data stored at the multiple locations, and to reconstruct the pieces in the correct order prior to processing the model as described herein”):
assign, using an algorithm, a value to each feature included in the plurality of features (see [0045], “Example approaches include permutation feature importance, partial dependence plots, Shapley values, SHAP (Shapley Additive Explanations), KernelSHAP, TreeSHAP, and integrated gradients. The explanation engine 130 can be configured to use some approaches over others depending on whether the explanation engine 130 is generating local or global explanations. For example, the explanation engine 130 may use permutation feature importance and partial dependence plots for generating global explanations, and Shapley values, SHAP, and integrated gradients for generating both local and global explanations”);
remove, from the artificially-intelligent model, features that have been assigned a value that is less than a predetermined threshold to form a revised artificially-intelligent model (see [0062], “In some examples, the evaluation engine 140 can automatically select the top feature attributions that explain some predetermined threshold, e.g., 80%, of the model prediction”); and recreate the revised artificially-intelligent model that can characterize an unlabeled data element set as being associated with the predetermined label (see [0063] and [0127], “The evaluation engine 140 can implement a graphical user interface, e.g., as one or more web pages, as an application installed on a user device, etc., for presenting and receiving data from a user device. In response to providing the model predictions and model explanations, the evaluation engine 140 can receive additional query statements, e.g., for re-training the model or for generating model explanation data according to different approaches or parameters than what was previously specified”).
As to claim 2, Cheng teaches the feature importance metric level corresponds to a percentage of the plurality of features (see [0062], “In some examples, the evaluation engine 140 can automatically select the top feature attributions that explain some predetermined threshold, e.g., 80%, of the model prediction”).
As to claim 3, Cheng teaches the feature importance metric level corresponds to a predetermined number of the plurality of features (see [0101], “The option top_k_features specifies the number of features whose attributions are returned. The features returned can be returned sorted according to the absolute value of their feature attributions. When a number is not provided, the default number of top features returned can be predetermined, e.g., set to the top 5 features”).
As to claim 12, Cheng teaches wherein the algorithm is Integrated Gradients, Cascaded Integrated Gradients, SHAP or TreeSHAP (see [0045], “Example approaches include permutation feature importance, partial dependence plots, Shapley values, SHAP (Shapley Additive Explanations), KernelSHAP, TreeSHAP, and integrated gradients. The explanation engine 130 can be configured to use some approaches over others depending on whether the explanation engine 130 is generating local or global explanations. For example, the explanation engine 130 may use permutation feature importance and partial dependence plots for generating global explanations, and Shapley values, SHAP, and integrated gradients for generating both local and global explanations”).
As to claim 14, Cheng teaches wherein the predetermined threshold is a percentage of the plurality of features (see [0062], “In some examples, the evaluation engine 140 can automatically select the top feature attributions that explain some predetermined threshold, e.g., 80%, of the model prediction”).
As to claim 15, Cheng teaches the predetermined threshold corresponds to a predetermined number of the plurality of features (see [0101], “The option top_k_features specifies the number of features whose attributions are returned. The features returned can be returned sorted according to the absolute value of their feature attributions. When a number is not provided, the default number of top features returned can be predetermined, e.g., set to the top 5 features”).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 8 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Cheng in view of “Gradient-Based Attribution Methods” (Publish Year: 2019, hereinafter Ancona).
As to claim 8, Cheng teaches a method for harnessing an explainable artificial intelligence system to execute computer-aided feature selection (see [0003], “a query-driven
computing platform for generating feature attributions and other model explanation data.”), the method comprising:
on a first iteration: receiving a characterization output characterizing a first data structure (see [0029], “In addition, the platform can receive, through the one or more query statements, parameters for generating model explanation data. Model explanation data can include local and global explanations. An explanation can be any data that at least partially characterizes a relationship between the output of the model, with either the input data used to generate the model, or with the model itself”);
identifying a plurality of data elements associated with the first data structure (see
[0039], “For example, preprocessing can include data normalization and formatting to bring the selected data to a form suitable for processing by the training engine 120. The preprocessing engine 110 can also be configured for feature selection/engineering, and/or removing or adding features to the input data according to any of a variety of different approaches.”);
feeding the plurality of data elements into one or more models (see [0088] and [0090], “The shard explanation engine 330 can receive model data, and from the model data and corresponding parameters, generate each explainer, and generate model explanation data for received input data.”);
processing the plurality of data elements at the one or more models (see [0089], “In those examples, in retrieving the model data, the shard explanation engine 330 is configured to retrieve the individual pieces of the model data stored at the multiple locations, and to reconstruct the pieces in the correct order prior to processing the model as described herein”);
identifying a plurality of outputs from one or more models (see [0090], “An explainer can be implemented in software and/or hardware and be configured to process the machine learning model and input data to generate local or global explanations, for example as described herein with reference to FIG. 1 and the explanation engine 130”);
determining a probability of the first data structure being associated with the characterization output (see [0051] For classification models, the explanation engine 130 can generate a global explanation on a model-level and/or a class-level. Model-level explanations can measure the importance of a feature across all classes a machine learning model is trained to use in classifying input. Class-level explanations can measure the importance of a feature for a particular class”);
removing one or more data elements from the subset of the plurality of data elements that are associated with a probability that is less than a probability threshold to generate an updated subset of the plurality of data elements (see [0062], “In some examples, the evaluation engine 140 can automatically select the top feature attributions that explain some predetermined threshold, e.g., 80%, of the model prediction”);
on a second iteration (see [0063], “The evaluation engine 140 can implement a graphical user interface, e.g., as one or more web pages, as an application installed on a user device, etc., for presenting and receiving data from a user device. In response to providing the model predictions and model explanations, the evaluation engine 140 can receive additional query statements, e.g., for re-training the model or for generating model explanation data according to different approaches or parameters than what was previously specified”; Note: when the valuation engine 140 receives additional query statements for re-training the model or generating model explanation data, a second iteration will be performed):
re-feeding the updated subset of the plurality of data elements into the one or more models (see [0088] and [0090], “The shard explanation engine 330 can receive model data, and from the model data and corresponding parameters, generate each explainer, and generate model explanation data for received input data”, see also [0063] for additional query statements for re-training the model);
re-processing the plurality of data elements at the one or more models (see [0089], “In those examples, in retrieving the model data, the shard explanation engine 330 is configured to retrieve the individual pieces of the model data stored at the multiple locations, and to reconstruct the pieces in the correct order prior to processing the model as described herein”, see also [0063] for additional query statements for re-training the model);
re-identifying the plurality of outputs from one or more models (see [0090], “An explainer can be implemented in software and/or hardware and be configured to process the machine learning model and input data to generate local or global explanations, for example as described herein with reference to FIG. 1 and the explanation engine 130”); and
utilizing the one or more models to characterize unlabeled data elements (see [0117], “For a trained model, the platform can receive input data for generating new predictions using the model”).
Although the Cheng teaches generating global explanation, it does not disclose the claim limitation of, “in order to remove a predetermined number of data elements from the plurality of data elements, said predetermined number of data elements that are detrimental to the characterization output: multiplying the integrated gradient of the one or more models with respect to the plurality of outputs by (the integrated gradient of the one or more models with respect to the plurality of data elements divided by the plurality of outputs), which results in a vector of: a subset of the plurality of data elements; and a probability that each data element, included in the subset of data elements, contributed to the characterization output”.
However, Ancona teaches using gradient information to determine attribution information for neural networks (p. 170, 1st and 2nd paragraphs),
remove a predetermined number of data elements from the plurality of data elements, said predetermined number of data elements that are detrimental to the characterization output (see p. 171, Section 9.2, 3rd paragraph, “Formally, attribution methods aim at producing explanations by assigning a scalar attribution value, sometimes also called “relevance” or “contribution”, to each input feature of a network for a given input sample”): multiplying the integrated gradient of the one or more models with respect to the plurality of outputs by (the integrated gradient of the one or more models with respect to the plurality of data elements divided by the plurality of outputs) (See p. 173, “In terms of relation with the gradient, we can formulate attributions as the gradient multiplied element-wise by the input:
PNG
media_image3.png
68
542
media_image3.png
Greyscale
),
which results in a vector of: a subset of the plurality of data elements; and a probability that each data element, included in the subset of data elements (see p. 176, “An attribution method satisfies sensitivity-n [3] when the sum of the attributions for any subset of features of cardinality n is equal to the variation of the output Sc caused removing the features in the subset. In this context, removing a feature means setting it to a baseline value, often chosen to be zero as discussed in Sect. 9.2.3”), contributed to the characterization output” (see p. 171, “Given a single target output unit c, the goal of an attribution method is to determine the contribution Rc = [Rc1,...,RcN] ∈ RN of each input feature xi to the output Sc(x)”).
It would have been obvious to a person of ordinary skilled in the art before the effective filing date of the invention to modify the invention of Cheng to use the global attribution technique as taught by Ancona because the technique contributes to making attribution maps sharper (see p. 174 of Ancona). Cheng and Ancona are in the same field of generating global explanation for machine learning models.
As to claim 10, the combination of Cheng and Ancona teaches wherein an equation for determining the integrated gradient of the one or more models with respect to the plurality of outputs is:
PNG
media_image2.png
66
250
media_image2.png
Greyscale
(Note: Due to the 112(b) rejection for claim 10, the integrated formula is interpreted as the known calculation for integrated gradients, which is described in Table 9.1, p. 180 of Ancona.)
PNG
media_image4.png
358
702
media_image4.png
Greyscale
Claims 6-9 are rejected under 35 U.S.C. 103 as being unpatentable over Cheng in view of US 20210241115 A1 (hereinafter Ibrahim) and further in view of “Gradient-Based Attribution Methods” (Publish Year: 2019, hereinafter Ancona).
As to claim 6, Cheng teaches a method for harnessing an explainable artificial intelligence system to execute computer-aided feature selection (see [0003], “a query-driven computing platform for generating feature attributions and other model explanation data.”), the method comprising:
on a first iteration: receiving a characterization output characterizing a first data structure (see [0029], “In addition, the platform can receive, through the one or more query statements, parameters for generating model explanation data. Model explanation data can include local and global explanations. An explanation can be any data that at least partially characterizes a relationship between the output of the model, with either the input data used to generate the model, or with the model itself”);
identifying a plurality of data elements associated with the first data structure (see [0039], “For example, preprocessing can include data normalization and formatting to bring the selected data to a form suitable for processing by the training engine 120. The preprocessing engine 110 can also be configured for feature selection/engineering, and/or removing or adding features to the input data according to any of a variety of different approaches.”);
feeding the plurality of data elements into one or more models (see [0088] and [0090], “The shard explanation engine 330 can receive model data, and from the model data and corresponding parameters, generate each explainer, and generate model explanation data for received input data.”);
processing the plurality of data elements at the one or more models (see [0089], “In those examples, in retrieving the model data, the shard explanation engine 330 is configured to retrieve the individual pieces of the model data stored at the multiple locations, and to reconstruct the pieces in the correct order prior to processing the model as described herein”);
identifying a plurality of outputs from the one or more models (see [0090], “An explainer can be implemented in software and/or hardware and be configured to process the machine learning model and input data to generate local or global explanations, for example as described herein with reference to FIG. 1 and the explanation engine 130”);
removing one or more data elements from the subset of the plurality of data elements that are associated with a probability that is less than a probability threshold to form an updated subset of the plurality of data elements (see [0062], “In some examples, the evaluation engine 140 can automatically select the top feature attributions that explain some predetermined threshold, e.g., 80%, of the model prediction”);
on a second iteration (see [0063], “The evaluation engine 140 can implement a graphical user interface, e.g., as one or more web pages, as an application installed on a user device, etc., for presenting and receiving data from a user device. In response to providing the model predictions and model explanations, the evaluation engine 140 can receive additional query statements, e.g., for re-training the model or for generating model explanation data according to different approaches or parameters than what was previously specified”; Note: when the evaluation engine 140 receives additional query statements for re-training the model or generating model explanation data, a second iteration will be performed):
re-feeding the updated subset of the plurality of data elements into the one or more models (see [0088] and [0090], “The shard explanation engine 330 can receive model data, and from the model data and corresponding parameters, generate each explainer, and generate model explanation data for received input data”, see also [0063] for additional query statements for re-training the model);
re-processing the plurality of data elements at the one or more models (see [0089], “In those examples, in retrieving the model data, the shard explanation engine 330 is configured to retrieve the individual pieces of the model data stored at the multiple locations, and to reconstruct the pieces in the correct order prior to processing the model as described herein”, see also [0063] for additional query statements for re-training the model);
re-identifying the plurality of outputs from the one or more models (identifying a plurality of outputs from the one or more models (see [0090], “An explainer can be implemented in software and/or hardware and be configured to process the machine learning model and input data to generate local or global explanations, for example as described herein with reference to FIG. 1 and the explanation engine 130”);
utilizing the one or more models to characterize unlabeled data elements (see [0117], “For a trained model, the platform can receive input data for generating new predictions using the model”).
Although Cheng discusses a plurality of models generating a plurality of outputs and combining the outputs to generate a global explanation using baseline score (see [0090] and [0058]-[0060], Cheng does not provide specific teachings of an event processor and a determination processor.
However, Ibrahim teaches a method for harnessing an explainable artificial intelligence system to execute computer-aided feature selection, the method comprising (see [0017], “the GAM system 101 may be configured to determine feature importance across a population of a data set to a neural network”),
feeding the plurality of outputs into an event processor (see [0039], “In embodiments, the GAM system 101 identifies differences in explanations among subpopulations in contrast to global explanations via surrogate models, which produce a single global explanation” and [0022], “The local attribution logic 102 generates and processes each attribution as a vector in R.sup.n: an attribution ranks and weighs each feature's association with a particular prediction. In embodiments, the local attribution logic 102 may determine an importance between a feature and a particular prediction by treating each attribution s a vector”);
processing the plurality of outputs at the event processor (see [0031], “In embodiments, the GAM system 102 includes clustering logic 108 to generate clusters or groupings of data points or local attributions. For example, the clustering logic 108 may apply a clustering algorithm, such as a K-medoids clustering algorithm, to generate clusters of the local risk assessment attributions”);
grouping the plurality of outputs into a plurality of events at the event processor (see [0032], “Clustering algorithms can detect global patterns in a dataset by grouping similar data points into a cluster. The clustering logic 108 applies cluster in the context of local attribution vectors by transforming many local attributions”);
inputting the plurality of events into a determination processor (see [0038], “The global attributions logic 110 may form GAM's global explanations across subpopulations by grouping similar normalized attributions”);
determining, at the determination processor, a probability of the first data structure being associated with the characterization output (see [0037], “In embodiments, the GAM system 101 including the global attributions logic 110 may generate global attributions. For example, the global attributions logic 110 may generate global attributions, and each global attribution corresponds to one of the medoids of one of the clusters. Each of GAM's global explanations yields the most centrally located vector of feature importance for a subpopulation. The global attribution landscape described by GAM is then the collection of global explanations for each subpopulation”);
re-feeding the plurality of outputs into the event processor (see [0022] and [0039], “In embodiments, the GAM system 101 identifies differences in explanations among subpopulations in contrast to global explanations via surrogate models, which produce a single global explanation” and [0033] for discussion of execution multiple iterations of the clustering logic);
re-processing the plurality of outputs at the event processor (see [0031], “In embodiments, the GAM system 102 includes clustering logic 108 to generate clusters or groupings of data points or local attributions. For example, the clustering logic 108 may apply a clustering algorithm, such as a K-medoids clustering algorithm, to generate clusters of the local risk assessment attributions”);
re-grouping the plurality of outputs into the plurality of events at the event processor (see [0032], “Clustering algorithms can detect global patterns in a dataset by grouping similar data points into a cluster. The clustering logic 108 applies cluster in the context of local attribution vectors by transforming many local attributions”);
re-inputting the plurality of events into the determination processor (see [0038], “The global attributions logic 110 may form GAM's global explanations across subpopulations by grouping similar normalized attributions”);
re-determining, at the determination processor, the probability of the first data structure being associated with the characterization output (see [0037], “In embodiments, the GAM system 101 including the global attributions logic 110 may generate global attributions. For example, the global attributions logic 110 may generate global attributions, and each global attribution corresponds to one of the medoids of one of the clusters. Each of GAM's global explanations yields the most centrally located vector of feature importance for a subpopulation. The global attribution landscape described by GAM is then the collection of global explanations for each subpopulation”).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the invention of Cheng to include the clustering technique to generate global explanations as taught by Ibrahim because the clustering technique in Ibrahim has advantages over other algorithms by allowing the system to compare attributions as conjoined weighted rankings rather than simply vectors. Moreover, the K-medoid cluster technique takes advantage of both the rank and weight information in local attributions during clustering (see [0035] of Ibrahim).
Although the combination of Cheng and Ibrahim teaches generating global explanation, they do not disclose the claim limitation of, “in order to remove a predetermined number of data elements from the plurality of data elements, said predetermined number of data elements that are detrimental to the characterization output: multiplying the integrated gradient of the determination processor with respect to the plurality of outputs by (the integrated gradient of the event processor with respect to the plurality of data elements divided by the plurality of outputs), which results in a vector of: a subset of the plurality of data elements; and a probability that each data element, included in the subset of data elements, contributed to the characterization output”.
However, Ancona teaches using gradient information to determine attribution information for neural networks (p. 170, 1st and 2nd paragraphs),
remove a predetermined number of data elements from the plurality of data elements, said predetermined number of data elements that are detrimental to the characterization output (see p. 171, Section 9.2, 3rd paragraph, “Formally, attribution methods aim at producing explanations by assigning a scalar attribution value, sometimes also called “relevance” or “contribution”, to each input feature of a network for a given input sample”): multiplying the integrated gradient of the determination processor with respect to the plurality of outputs by (the integrated gradient of the event processor with respect to the plurality of data elements divided by the plurality of outputs) (Note: due the 112(a) rejection for the features of integrated gradients of the event and determination processor, these features are interpreted as integrated gradient of the one or more models and integrated gradients of models with subpopulation of the plurality of data elements (consistent with the features of claim 8). See p. 173, “In terms of relation with the gradient, we can formulate attributions as the gradient multiplied element-wise by the input:
PNG
media_image3.png
68
542
media_image3.png
Greyscale
), which results in a vector of: a subset of the plurality of data elements; and a probability that each data element, included in the subset of data elements (see p. 176, “An attribution method satisfies sensitivity-n [3] when the sum of the attributions for any subset of features of cardinality n is equal to the variation of the output Sc caused removing the features in the subset. In this context, removing a feature means setting it to a baseline value, often chosen to be zero as discussed in Sect. 9.2.3”), contributed to the characterization output (see p. 171, “Given a single target output unit c, the goal of an attribution method is to determine the contribution Rc = [Rc1,...,RcN] ∈ RN of each input feature xi to the output Sc(x)”).
It would have been obvious to a person of ordinary skilled in the art before the effective filing date of the invention to modify the invention of Cheng and Ibrahim to use the global attribution technique as taught by Ancona because the technique contributes to making attribution maps sharper (see p. 174 of Ancona). Cheng, Ibrahim, and Ancona are in the same field of generating global explanation for machine learning models.
As to claim 7, Cheng as modified by Ibrahim and Ancona teaches wherein the first iteration is re-executed until all of the data elements are assigned a probability that is greater than the probability threshold (see [0033] of Ibrahim, “The local attributions are reassigned until convergence or for a sufficiently large number of iterations, each taking O(n(n−K).sup.2)”).
As to claim 9, Cheng and Ancona do not teach the first iteration is re-executed until all of the data elements are assigned a probability that is greater than the probability threshold.
However, Ibrahim teaches wherein the first iteration is re-executed until all of the data elements are assigned a probability that is greater than the probability threshold (see [0033] of Ibrahim, “The local attributions are reassigned until convergence or for a sufficiently large number of iterations, each taking O(n(n−K).sup.2)”).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the invention of Cheng and Ancona to include the process of re-executed a first iteration until all of the data elements are assigned a probability that is greater than the probability threshold because the clustering technique in Ibrahim has advantages over other algorithms by allowing the system to compare attributions as conjoined weighted rankings rather than simply vectors. Moreover, the K-medoid cluster technique takes advantage of both the rank and weight information in local attributions during clustering (see [0035] of Ibrahim).
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Cheng in view of Ibrahim.
As to claim 13, Cheng does not teach wherein the refining model module is re-executed until all of the features are assigned a value that is greater than the predetermined threshold.
However, Ibrahim teaches wherein the first iteration is re-executed until all of the data elements are assigned a probability that is greater than the probability threshold (see [0033] of Ibrahim, “The local attributions are reassigned until convergence or for a sufficiently large number of iterations, each taking O(n(n−K).sup.2)”).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the invention of Cheng and Ancona to include the process of re-executed a first iteration until all of the data elements are assigned a probability that is greater than the probability threshold because the clustering technique in Ibrahim has advantages over other algorithms by allowing the system to compare attributions as conjoined weighted rankings rather than simply vectors. Moreover, the K-medoid cluster technique takes advantage of both the rank and weight information in local attributions during clustering (see [0035] of Ibrahim).
Claims 4, 5, 16 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Cheng in view of US 20210365189 A1 (hereinafter Masuda).
As to claim 4, Cheng does not teach wherein the feature importance metric level corresponds to a predetermined value assigned to the plurality of features.
However Masuda teaches wherein the feature importance metric level corresponds to a predetermined value assigned to the plurality of features (see [0074], “Outputs from the explanatory model depend on the type of the XAI technology which is used. There are outputs which are output totally as positive values as indicated in the columns of the rows 507 to 510 in FIG. 5 as described above and there are also outputs which are output as positive values and negative values in a mixed state as illustrated in FIG. 12”).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the invention of Cheng to include the technique of generating predetermined value in explanatory output as taught by Masuda because the allows for negative and positive values which can be used to represent negative and positive influence in the explanatory data (see [0074] of Masuda).
As to claim 5, the combination of Cheng and Masuda teaches wherein the predetermined value corresponds to a negative value (see [0074] of Masuda, “For example, in a case where, in the output values which are derived from the explanatory model, the positive value that the absolute value exceeds its threshold value and the negative value that the absolute value exceeds its threshold value are present in the mixed state”).
As to claim 16, the combination of Cheng and Masuda teaches wherein the predetermined threshold corresponds to a predetermined value assigned to the plurality of features (see [0074] of Masuda, “Outputs from the explanatory model depend on the type of the XAI technology which is used. There are outputs which are output totally as positive values as indicated in the columns of the rows 507 to 510 in FIG. 5 as described above and there are also outputs which are output as positive values and negative values in a mixed state as illustrated in FIG. 12”).
As to claim 17, the combination of Cheng and Masuda teaches wherein the predetermined threshold corresponds to a negative value (see [0074] of Masuda, “For example, in a case where, in the output values which are derived from the explanatory model, the positive value that the absolute value exceeds its threshold value and the negative value that the absolute value exceeds its threshold value are present in the mixed state”).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
20220067580 (Rho) discloses a system that flexibly and dynamically analyze a machine learning process, and that generate analytical output characterizing an operation of the machine learning process across multiple analytical periods
20220027986 (Hesami) discloses a system that analyzes the resulting model(s) with decomposition methods such as Generalized Integrated Gradients and allow analysts to understand how the inclusion of unlabeled rows influences how a model generates scores by comparing the input feature importances between models with and without these additional data points.
20220004923 (Kamkar) discloses a system that provides insight into operation of machine learning systems by evaluating operation of a machine learning system based on evaluation criteria (e.g., constraints on results generated by the machine learning system, constraints on comparisons among results generated by the machine learning system from one or more inputs, etc.).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LI B ZHEN whose telephone number is (571)272-3768. The examiner can normally be reached M-F, 8:30a-5p.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Li B. Zhen/Supervisory Patent Examiner, Art Unit 2121
Prosecution Insights