Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This Final Rejection is filed in response to Applicant Arguments/Remarks Made in an Amendment filed 05/26/2026.
Claims 1, 2, 5, 10, 14, and 18 are amended.
Claims 1-20 remain pending.
Response to Arguments
Argument 1, Applicant argues in Applicant Arguments/Remarks Made in an Amendment filed 05/26/2026 pg. 12-14, the primary claim limitation,
"receiving, by machine learning system, a request to start a monitoring job for the machine learning model after deployment of the machine learning model, wherein the monitoring job specifies a configuration for performing monitoring of feature attribution drift of the machine learning model according to the feature attribution requested in the training job;
determining, by the machine learning system, that the machine learning model of the request to start the monitoring job that is associated with the trial;
obtaining, by the machine learning system, the report, generated and stored for the trial as part of executing the training job according to the feature attribution instructed in the training job, to compute feature attribution drift of the machine learning model for the requested monitoring job",
Response to Argument 1, the examiner respectfully disagrees and notes that a combination of Raj, Coleman, and Cataltepe teach the above limitation.
Raj sets the stage for a machine learning system that receives a training job associated with a trial in that a model trainer receives a training job for a specific past data snapshot which is monitored for feature attribution by comparing the past data to current data as part of the trial of the machine learning model to determine how specific features influence the models behavior by fitting the model onto said data snapshots. Thus the BRI for, “receiving, by a machine learning system, a request to start a monitoring job for the machine learning model … wherein the monitoring job specifies a configuration for performing monitoring of feature attribution … of the machine learning model according to the feature attribution requested in the training job; determining, by the machine learning system, that the machine learning model of the request to start the monitoring job that is associated with the trial”, encompasses how a model starts a feature attribution trial by monitoring historical data compared to a current data snapshot in response to a user selection of specific past data.
The following paragraphs of Raj support this interpretation.
para. [0045], “a past data snapshot is retrieved (e.g., by variance server 120). In some embodiments, the past data snapshot may include data collected from a previous time frame such as the year 2018
para. [0048], the generated machine learning (or prediction) model is “scored” based on the past data snapshot retrieved in 310 along with a second data snapshot, such as one reflecting a different time frame. In some embodiments, that time frame may reflect a “current” time frame such as the current year or month
para. [0050], the feature attribution of each model behavior is determined using the scores of the prediction model… this step may involve deriving how each model behavior influences the model by assigning a feature attribution value to each model. In an example where the model is directed toward delinquency in the loan data, each model behavior (e.g., credit score, age segment) is assigned a feature attribution valvalueat reflects the influence of that feature on the behavior (e.g., delinquency) in the past data snapshot the current data snapshot
While Raj teaches a monitoring job that involves feature attribution drift, Raj may not explicitly teach a request to start a monitoring job for the machine learning model after deployment of the machine learning model that explicitly involves monitoring and detecting feature attribution drift in selected features.
However, Cataltepe teaches a live monitoring job that includes an online machine learning system that selects and evaluates features for a deployed model and includes a visual display explanation model of the relevance and drift of the selected features for a model task. It should be noted that Cataltepe specifically teaches calculating a difference in value between aggregated feature attributions for a staged model against a currently online model being continuously trained on live streaming data. Drift and anomaly detection methods are then used to generate a display of performance metrics that can be used to raise automated alerts. Thus the BRI for, “receiving, by a machine learning system, a request to start a monitoring job for the machine learning model after deployment of the machine learning model, wherein the monitoring job specifies a configuration for performing monitoring of feature attribution drift of the machine learning model according to the feature attribution requested in the training job;… the report, generated and stored for the trial as part of executing the training job according to the feature attribution instructed in the training job, to compute feature attribution drift of the machine learning model for the requested monitoring job”, encompasses the display of a monitoring job request that may be for select features in which the OMLS monitors a deployed model for feature drift among other vectors.
The following paragraphs of Cataltepe support this interpretation.
[0060], In some embodiments, users, such as experts or other decision-makers, are provided with a displayed interactive staging area. The staging area may be used to allow the decision-makers to interactively manipulate, filter, review and update models, for example, to increase model trustworthiness and accountability, or to compare models or model performances. In some embodiments, the OMLS or particular models may include alert modules designed to alert decision-makers to a need for their feedback and may prompt them with staged models.
[0085] At step 362, an OPrE receives streaming data including an instance including a vector of inputs including multiple continuous or categorical features. The OPrE is able to, and may, discretize features, impute missing feature values, normalize features, and detect drift or change in features.
[0190] Through the visual display of the explanation model, such as for example sunburst as in FIGS. 16-20, the user is able to understand how the machine learning has decided on its predictions in different regions of the input space and also provide model level feedback.
One would have been motivated to combine the monitoring of a deployed model for feature attribution drift in which a report is displayed to a user of Cataltepe with the feature attribution generation and training of a model by comparing user selected past snapshot data to current data as part of trial job of Raj-Coleman and would have had a reasonable expectation of success to save a user time by providing a remedy to such a detected performance decline and a quick analysis of the findings associated with the structural and/or generative shifts, which may help to do governance and/or backward analysis.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 5 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 5 recites the limitation
"as part of executing the training job according to the feature attribution instructed in the training job" in lines 31-32. There is insufficient antecedent basis for this limitation in the claim.
As claim 5 merely recites “a training job … that trains a machine learning model from a training data set that associates the machine learning model with a trial”, and that “the training job includes instruction to: determine feature attribution… ” in lines 2-6. It is unclear what feature attribution lines 31-32 refers to and for the sake of examination the limitation, “the feature attribution instructed in the training job” is being interpreted as the feature attribution determined by the instruction included in the training job.
Claim 1 is objected to for similar reasons.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-2, 4-7, 10, 12-13, 14-16, 18, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Application Publication NO. 20220067460 "Raj" and further in light of U.S. Patent Application Publication NO. 20190113973 "Coleman", and in further light of U.S. Patent Application Publication NO. 20190279102 "Cataltepe".
Claim 5:
Raj teaches a method, comprising:
receiving, by a machine learning system, a training job (i.e. para. [0041], “Model trainer 220 includes components for generating models to be used for performing the analysis by data processor 210”, wherein the BRI for a training job encompasses utilizing feature attribution values to train a model by iteratively adjusting the model to match the specified data) from a training data set that associates the machine learning model with a trial (i.e. para. [0045], “a past data snapshot is retrieved (e.g., by variance server 120). In some embodiments, the past data snapshot may include data collected from a previous time frame such as the year 2018”, wherein the BRI for training data encompasses a specific past data snapshot and wherein the BRI for a trial encompasses how the model trainer fits the model onto a specific data snapshot in order to derive how specific features influence model behavior), wherein the training job includes an instruction to
determine feature attribution as part of the trial of the machine learning model (i.e. para. [0062-0063], “a first model score is generated based on the fitted machine learning model and the first data snapshot… the first model score reflects a first probability between the values of data points in the first data snapshot with the calculated values provided by the fitted machine learning model… a first feature attribution of a first feature in the plurality of features is determined based on the first model score and a second feature attribution of the first feature is determined based on the second model score”, wherein at least a first and second feature attribution values are determined as part of the fitting trial for the model trainer fitting the model to a specific data snapshot); and
select a reference data set out of the training data set (i.e. para. [0045], Fig. 3, "In 310, a past data snapshot is retrieved (e.g., by variance server 120)", wherein it is noted that a data snapshot from the past may be specified out of all potential data snapshots for training, wherein the data snapshot that is specifically from the past is used to tune, train, and evaluate a model) used to generate a report for the trial (i.e. para. [0064], “the determined feature attributions are aggregated for each member of the data set. In some embodiments, the feature attributions for each member (e.g., account) are aggregated into a waterfall chart that indicates how each feature associated with that account is moving the prediction for the machine learning model”, wherein the BRI for a report encompasses the associated waterfall chart that accompanies the fitted model to the specific snapshot);
executing, by the machine learning system, the training job to train the machine learning model for the trial (i.e. para. [0039,0046], "the analysis may include generating and modify machine learning models, identifying similarities and differences between data sets, and predicting and analyzing the differences between the data sets using the generated and modified machine learning models... a machine learning (or prediction) model is fitted onto the retrieved data snapshot", wherein the user request for a data analysis of a data snapshot may be analyzed via a generated machine learning model) , wherein the executing comprises:
identifying a reference data set for determining the feature attribution of the machine learning model according to the request (i.e. para. [0040], "Examples of feature attribution values include Shapley values and are used to reflect a numerical measure of a feature's impact on the overall differences between compared data sets. For example, data processor 210 may be requested to analyze data sets from different years that have a variance between a feature", wherein an earlier data set may be used as a reference to determine a feature's impact);
determining the feature attribution of the trained machine learning model (i.e. para. [0050], each model behavior (e.g., credit score, age segment) is assigned a feature attribution value that that reflects the influence of that feature on the behavior (e.g., delinquency) in the past data snapshot); storing, a report associated with the trial (i.e. para. [0032], “Results may include a waterfall chart that depicts the cumulative effect of different values as they are added or subtracted. In some embodiments, a waterfall chart is produced by variance characterization server 120 based on the feature attributions that are assigned to different parameters associated with the analyzed data set”, wherein the BRI for a report encompasses a waterfall chart that displays the feature attribution impact on the model) that includes the feature attribution of the machine learning model (i.e. para. [0052], "This aggregation allows feature attributions assigned within the data snapshot to be grouped based on the specific feature. the feature attributions for these features may be aggregated across the separate members (e.g., loan accounts) to provide a single feature attribution value for each the feature", wherein the feature attributions are stored as they are aggregated into a single attribution value for a specific feature of the model);(i.e. para. [0048], “the generated machine learning (or prediction) model is “scored” based on the past data snapshot retrieved in 310 along with a second data snapshot, such as one reflecting a different time frame. In some embodiments, that time frame may reflect a “current” time frame such as the current year or month “, wherein the BRI for a monitoring job encompasses comparing a specific past data snapshot to a current snapshot)
determining, by the machine learning system, that the machine learning model of the request to start the monitoring job that is associated with the trial (i.e. para. [0050], “the feature attribution of each model behavior is determined using the scores of the prediction model… this step may involve deriving how each model behavior influences the model by assigning a feature attribution value to each model. In an example where the model is directed toward delinquency in the loan data, each model behavior (e.g., credit score, age segment) is assigned a feature attribution value that that reflects the influence of that feature on the behavior (e.g., delinquency) in the past data snapshot the current data snapshot”, wherein the BRI to start the monitoring job encompasses tracking the historical data compared to a current data snapshot);
obtaining, by the machine learning system, the report, generated and stored for the trial as part of executing the training job according to the feature attribution instructed in the training job, to compute feature attribution (i.e. para. [0064], “In some embodiments, the feature attributions for each member (e.g., account) are aggregated into a waterfall chart that indicates how each feature associated with that account is moving the prediction for the machine learning model”, wherein feature attributions for the data set are identified and stored in the waterfall chart as part of the model training and analysis); and
While Raj teaches determining feature attribution as part of trial of the machine learning model, Raj may not explicitly teach that the training job is for
A machine learning pipeline;
However, Coleman teaches
A training job for a machine learning pipeline that trains a machine learning model (i.e. para. [0150, 0206, 0218], "Searching for meaningful patterns may be used to predict whether a feature event belongs to one category Once a meaningful pattern has been discovered, the system or another platform may modify or create a pipeline to predict one of the feature event values from the other two", wherein it is noted that the BRI for a training job for a machine learning pipeline encompasses how a workflow in the cloud can apply machine learning to learn new pipelines in order to find a feature event value which describes a predictive pattern for a certain event).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to add a training job for a machine learning pipeline that trains a machine learning model, to the feature attribution and training models of Raj, with how finding a predictive score for an event of part of machine learning model is part of a machine learning pipeline, as taught by Coleman. One would have been motivated to combine Coleman with Raj and would have had a reasonable expectation of success the addition of a machine learning pipeline to the machine learning systems minimizes the efforts, number of steps, and time required to train different applications to function based on relevant features or attributes (Coleman, para. [0166]).
While Raj-Coleman teach a machine learning model for feature attribution that stores data that includes monitoring the feature attribution and stores a report for data about the selected features during training, Raj-Coleman may not explicitly teach
…performing monitoring of feature attribution drift …;
… to compute feature attribution drift of the machine learning model for the requested monitoring job
monitoring feature attribution drift of the machine learning model deployed and generating inferences on live data according to the configuration for performing feature attribution monitoring specified in the monitoring job, wherein the stored feature attribution is accessed as part of monitoring feature attribution drift.
However, Cataltepe teaches
receiving, by a machine learning system, a request to start a monitoring job for the machine learning model after deployment of the machine learning model (i.e. para. [0214], "When the individual models in the OMLS request feedback, the requests can be ordered based on the variance of all the model outputs for the feedback instance or the accuracy of the model that requests the instance ", wherein it is noted that a current model may be deployed and in use when a user may request to monitor and display feedback insights of the features in the current model), wherein the monitoring job specifies a configuration for performing monitoring of feature attribution drift of the machine learning model (i.e. para. [0058], "the OPrE may receive streaming data including an instance including a vector of inputs including multiple continuous or categorical features, and is able to discretize features, impute missing feature values, normalize features, and detect drift or change in features. The OFEE may produce features. The ORFSE may evaluate and select features", wherein a monitoring job request may be for select features in which the OMLS monitors a deployed model for feature drift. It is further noted in para. [0125] that those classes are learned by the OMLS in proportion to their importances, and that a class weight may be assigned to each class) according to the feature attribution requested in the training job (i.e. para. [0176-0177], “The current invention includes an Online (model) Explanation System (OES) that learns continuously while the online machine learning system (OMLS) keeps being updated based on the data stream and also according to the initial and changing preferences of the human domain experts… Let x denote the original features which the domain expert knows about and z denote the combination of the original and engineered features. While OMLS keeps learning continuously, its output g.sub.t(z) is taught to an ensemble of simpler explanation machine learning models, such as online decision trees or linear models. The explanation machine learning models h.sub.t(x) are trained using x, so that they are understandable by the human experts. In addition to x, features from z that are human understandable can also be included for training the explanation model”, wherein the BRI for the feature attribution requested in the training job encompasses an initial set of features that the OMLS system is trained on that m may be evaluated after deployment for feature attribution drift);
obtaining, by the machine learning system, the report (i.e. [0060], In some embodiments, users, such as experts or other decision-makers, are provided with a displayed interactive staging area. The staging area may be used to allow the decision-makers to interactively manipulate, filter, review and update models, for example, to increase model trustworthiness and accountability, or to compare models or model performances), generated and stored for the trial as part of executing the training job according to the feature attribution instructed in the training job and the selected data set, to compute feature attribution drift of the machine learning model for the requested monitoring job (i.e. para. [0087-0088, 0099], At step 364, an ORFSE evaluates and selects features… At step 365, the OMLE incorporates and utilizes one or more machine learning algorithms…. The OMLS also contains an Online Robust Feature Selection Engine (ORFSE) module where all the features are continuously and robustly evaluated in terms of how relevant they are for the particular machine learning task”, wherein the BRI for the report encompasses an understandable explanation that is an evaluation for the importance of the original features requested in an initial training job, which is then used as part of a monitoring job to find feature attribution drift of these original features); and
monitoring feature attribution drift of the machine learning model deployed and generating inferences on live data (i.e. para. [0084-0085], Fig. 3E, "Block 361 represents continuous operation of the OMLE, during which, and as part of which, operation the other steps take place. At step 362, an OPrE receives streaming data including an instance including a vector of inputs including multiple continuous or categorical features. The OPrE is able to, and may, discretize features, impute missing feature values, normalize features, and detect drift or change in features", wherein a deployed model is a current model being continuously monitored by the OMLS and the BRI for inferences encompasses the reports of detected drift or change in features) according to the configuration for performing feature attribution monitoring specified in the monitoring job, wherein the stored feature attribution is accessed as part of monitoring feature attribution drift (i.e. para. [0099], Fig. 16-20, "The OMLS also contains an Online Robust Feature Selection Engine (ORFSE) module where all the features are continuously and robustly evaluated in terms of how relevant they are for the particular machine learning task", wherein the model configuration may be specified to current features such that the current features are being continuously evaluated as part of the monitoring, wherein the evaluation may be a view as seen in fig. 18 which displays a model's current feature being evaluated).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to add receiving, by a machine learning system, a request to start a monitoring job for the machine learning model after deployment of the machine learning model, wherein the monitoring job specifies a configuration for performing monitoring of feature attribution drift of the machine learning model according to the feature attribution requested in the training job; obtaining, by the machine learning system, the report, generated and stored for the trial as part of executing the training job according to the feature attribution instructed in the training job, to compute feature attribution drift of the machine learning model for the requested monitoring job; monitoring feature attribution drift of the machine learning model deployed and generating inferences on live data according to the configuration for performing feature attribution monitoring specified in the monitoring job, wherein the stored feature attribution is accessed as part of monitoring feature attribution drift, to the feature attribution and training models of Raj-Coleman, with how a report with inferences on currently ingested data for drift in feature attribution is stored and displayed to users, as taught by Cataltepe. One would have been motivated to combine Cataltepe with Raj-Coleman and would have had a reasonable expectation of success to save a user time by providing a remedy to such a detected performance decline and a quick analysis of the findings associated with the structural and/or generative shifts, which may help to do governance and/or backward analysis.
Claim 6:
Raj, Coleman, and Cataltepe teach the method of claim 5.
Raj further teaches wherein the feature attribution is determined according to a specified feature attribution technique out of a plurality of feature attribution techniques supported by the machine learning system (i.e. para. [0042], "model trainer 220 generates a prediction model such as a gradient boosting machine (GBM) model as a prediction model, or other random forest techniques. In an alternative embodiment, model trainer 220 may employ a different model such as a neural network. In an example, the prediction model is a tree-based model that is utilized for detecting interactions between variables in a data set and making predictions of how those variables would impact the generated model", wherein different techniques, such as GMB or random forest models, determine feature attribution via a predictive model and would be supported by the model trainer of the variance characterization server).
Claim 7:
Raj, Coleman, and Cataltepe teach the method of claim 5.
Raj further teaches wherein the reference data set is identified according to one or more data values specified for the reference data set in the training job (i.e. para. [0041], Model trainer 220 may utilize the feature attribution values to generate machine learning models that are specific to predictions and analyses based on particular data sets and particular variables).
Claim 10:
Raj, Coleman, and Cataltepe teach the method of claim 5.
Raj further teaches wherein trial is an experiment trial executed as part of the training job (i.e. para. [0056], "method 400 may be used to generate a second prediction model for determining a performance change between the past data snapshot and the second data snapshot. Performance changes may represent a change (improvement or worsening) at a segment-level within the data set.", wherein the BRI for an experiment trial encompasses how the trial of fitting the model onto data snapshot is an experimental trial to derive how specific features influence the model behavior).
Claim 12:
Raj, Coleman, and Cataltepe teach the method of claim 5. Cataltepe further teaches wherein the training job i.e. para. [0011], "sharing may involve providing direct access to the individual data or results or the aggregated data or results or it may mean providing access to the data, results, or both through an application programming interface (API), or other known means", wherein after a machine learning process reviews discovered patterns, the workflows in the cloud can apply machine learning methods that can learn new pipelines, change which pipelines to apply to specific contexts, and learn new patterns that can help a User achieve their goals).
Cataltepe further teaches wherein the training job and monitoring job are specified according to one or more Application Programming Interfaces (APIs) of a fairness and explainability processing container offered by a machine learning service of a provider network (i.e. para. [0225-0226], Fig. 16-17, "The explanation models that are trained continuously are first copied and taken to a staging area (FIG. 16, FIG. 17) At the staging area, the user is able to examine (for an example, see FIG. 18) the details of the model by means of different filtering mechanisms, such as visualizing only the nodes that have a certain training/test accuracy or confidence", wherein the BRI for wherein the BRI for the training job encompasses the selection of a certain batch of streaming data and the BRI for the monitoring job being specified encompasses the specification of the current model for monitoring and display of feature information that may include detected drift or change in features)
Claim 13:
Raj, Coleman, and Cataltepe teach the method of claim 5.
Coleman further teaches wherein the machine learning system is implemented on one or more training nodes of a machine learning service offered by a provider network and wherein the feature attribution is stored as part of a report in a data storage service offered by the provider network (i.e. para. [0048], "FIG. 1, in an embodiment of the system there is shown a trusted network of local client devices 113 and a software as a service (SAAS) platform 200 operating at one or more computer servers", wherein it is noted in para. [0219] that the system platform may maintain a database store of all of the combination of feature events).
Claim 1:
Claim 1 is the system claim reciting similar limitations to Claim 5 and is rejected for similar reasons.
Claim 2:
Claim 2 is the system claim reciting similar limitations to Claim 10 and is rejected for similar reasons.
Claim 4:
Claim 4 is the system claim reciting similar limitations to Claim 12 and is rejected for similar reasons.
Claim 14:
Claim 14 is the machine claim reciting similar limitations to Claim 5 and is rejected for similar reasons.
Claim 15:
Claim 14 is the machine claim reciting similar limitations to Claim 6 and is rejected for similar reasons.
Claim 16:
Claim 14 is the machine claim reciting similar limitations to Claim 7 and is rejected for similar reasons.
Claim 18:
Claim 18 is the machine claim reciting similar limitations to Claim 10 and is rejected for similar reasons.
Claim 20:
Claim 20 is the machine claim reciting similar limitations to Claim 12 and is rejected for similar reasons.
Claim(s) 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Application Publication NO. 20220067460 "Raj", in light of U.S. Patent Application Publication NO. 20190113973 "Coleman", and in further light of U.S. Patent Application Publication NO. 20190279102 "Cataltepe", as applied to Claim 5 above, and further in light of U.S. Patent Application Publication NO. 9779362 "Gold".
Claim 9:
Raj, Coleman, and Cataltepe teach the method of claim 5.
Raj-Coleman-Cataltepe may not explicitly teach further comprising:
receiving, by the machine learning system, a request for a particular feature attribution for a specific inference generated by the trained machine learning model; determining, by the machine learning system, the particular feature attribution for the specific inference according to the identified reference data set; and sending, by the machine learning system, the particular feature attribution for the specific inference in response to the request.
However, Gold teaches
receiving, by the machine learning system (i.e. Col. 3, lines 18-21, features are then analyzed using a machine learning technique to determine weighted values representative of their respective contribution to causation), a request for a particular feature attribution for a specific inference generated by the trained machine learning model (i.e. Col. 11, lines 43-53, "An inference can be employed to identify a specific context or action. Such an inference can result in the construction of new events or actions from a set of observed events and/or stored event data" wherein a user may request a specific context or action in which an inference page explaining the contribution of the specific context is generated after being analyzed by a machine learning system); determining, by the machine learning system, the particular feature attribution for the specific inference according to the identified reference data set (i.e. Col. 11, lines 40-45, "inference component 208 can examine the entirety or a subset of the data to which it is granted access and can provide for reasoning about or infer states of the system, environment, etc. from a set of observations", wherein an inference for a specific context would be identified by the machine learning system according to the respective data); and sending, by the machine learning system, the particular feature attribution for the specific inference in response to the request (i.e. Col. 11, lines 24-28, "inference component 208 can facilitate analysis component 108 by inferring weights to associate with quality features based on an inferred level of contribution Inference component 208 can further facilitate recommendation component 206 with inferring changes to quality feature to recommend.", wherein the display of an analysis of feature contribution to a user).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to add receiving, by the machine learning system, a request for a particular feature attribution for a specific inference generated by the trained machine learning model; determining, by the machine learning system, the particular feature attribution for the specific inference according to the identified reference data set; and sending, by the machine learning system, the particular feature attribution for the specific inference in response to the request, to the feature attribution and training models of Raj-Coleman-Cataltepe, with an inference for a specific feature may be determined, calculated, and displayed to a user, as taught by Gold. One would have been motivated to combine Gold with Raj-Coleman-Cataltepe and would have had a reasonable expectation of success in order to analyze features and select ones that could predictively improve a system if the changes were implemented (Gold, Col. 3, lines 24-27).
Claim(s) 11 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Application Publication NO. 20220067460 "Raj", in light of U.S. Patent Application Publication NO. 20190113973 "Coleman", and in further light of U.S. Patent Application Publication NO. 20190279102 "Cataltepe", as applied to Claim 5 above, and further in light of U.S. Patent Application Publication NO. 20210241115 "Ibrahim".
Claim 11:
Raj, Coleman, and Cataltepe teach the method of claim 5.
Raj may not explicitly
teach wherein the training job further specifies determining bias metrics at one or more stages of the machine learning pipeline and wherein the executing further comprises: determining the one or more bias metrics at the one or more stages of the machine learning model; and storing the one or more bias metrics for the machine learning model.
However, Ibrahim teaches wherein the training job further specifies determining bias metrics at one or more stages of the machine learning pipeline and wherein the executing further comprises:
determining the one or more bias metrics at the one or more stages of the machine learning model (i.e. para. [0040], GAM insight logic 112 may determine global explanations to individual samples, supplement other techniques, determine bias in the neural network); and
storing the one or more bias metrics for the machine learning model (i.e. para. [0041], "The system 150 includes the GAM system 101, a storage system 103 including one or more data sets 114, and one or more neural networks", wherein insights related to determining bias may be stored for use in training the neural network).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to add determining the one or more bias metrics at the one or more stages of the machine learning model; and storing the one or more bias metrics for the machine learning model, to the feature attribution and training models Raj-Coleman-Cataltepe Raj, with how insights related to determining training bias may be stored and used, as taught by Ibrahim. One would have been motivated to combine Ibrahim with Raj-Coleman-Cataltepe and would have had a reasonable expectation of success as the combination creates more transparent predictions, which can help ensure neural network decisions are generated for the right reasons (Ibrahim, para. [0011]).
Claim 19:
Claim 19 is the machine claim reciting similar limitations to Claim 11 and is rejected for similar reasons.
Claim(s) 3, 8, and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Application Publication NO. 20220067460 "Raj", in light of U.S. Patent Application Publication NO. 20190113973 "Coleman", and in further light of U.S. Patent Application Publication NO. 20190279102 "Cataltepe", as applied to Claim 5 above, and further in light of U.S. Patent Application Publication NO. 20200183035 "Liu".
Claim 8:
Raj, Coleman, and Cataltepe teach the method of claim 5.
While Raj further teaches wherein the machine learning system comprises a cluster of nodes, and wherein determining the feature attribution of the trained machine learning model (i.e. para. [0035], "server 120 may be implemented as a plurality of servers that function collectively as a distributed database", wherein the distributed group of servers may be used to perform the feature attribution as variance characterization functions between data sets retrieved from data sources as part of the machine learning process), Raj may not explicitly teach:
wherein the machine learning system comprises a cluster of nodes, and wherein determining the feature attribution of the trained machine learning model as part of the machine learning pipeline dividing, by a leader node of the cluster of nodes, an input data set into different portions; assigning, by the leader node, the different portions to different worker nodes of the cluster of nodes; calculating, by the different worker nodes, respective feature attribution measurements for the different portions of the input data set using a respective copy of the reference data set at the worker nodes; combining, by the leader node, the respective feature attribution measurements into the feature attribution for the trained machine learning model.
However, Liu teaches wherein
the machine learning system comprises a cluster of nodes, and wherein determining the feature attribution of the trained machine learning model as part of the machine learning pipeline (i.e. para. [0061], Fig. 2, "Method 200 may be designed in a distributed, asynchronous workflow wherein the BRI for a machine learning pipeline encompasses the workflow for augmenting a ML system in Fig. 2) comprises:
dividing, by a leader node of the cluster of nodes, an input data set into different portions (i.e. para. [0061], During ML training, the master node may load the original seismic volume image and labels into its main memory (at block 212). The master node may randomly extract some 3-D patches (at block 270). The master node may put the patches into a queue system.); assigning, by the leader node, the different portions to different worker nodes of the cluster of nodes (i.e. para. [0061], Each of the patches in the queue may be dispatched to one of the worker nodes to perform transformation); calculating, by the different worker nodes, respective feature attribution measurements for the different portions of the input data set using a respective copy of the reference data set at the worker nodes (i.e. para. [0061], Once a worker node receives the assigned patches, it runs the transformation routine, and returns the augmented data to the queuing system of the master node); combining, by the leader node, the respective feature attribution measurements into the feature attribution for the trained machine learning model (i.e. para. [0061], runs the transformation routine, and returns the augmented data to the queuing system of the master node. The master node may then use the augmented data for ML trainings).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to add dividing, by a leader node of the cluster of nodes, an input data set into different portions; assigning, by the leader node, the different portions to different worker nodes of the cluster of nodes; calculating, by the different worker nodes, respective feature attribution measurements for the different portions of the input data set using a respective copy of the reference data set at the worker nodes; combining, by the leader node, the respective feature attribution measurements into the feature attribution for the trained machine learning model, to the feature attribution techniques that may be executed by across the distributed servers of Raj-Coleman-Cataltepe with how a leader node may calculate how to distribute a machine learning task among a cluster of nodes, as taught by Liu. One would have been motivated to combine Liu with Raj-Coleman-Cataltepe and would have had a reasonable expectation of success the combination results a distributed computing system may be utilized to improve the efficiency of a ML system through higher throughput with parallel input/output (Liu, para. [0055]).
Claim 3:
Claim 3 is the system claim reciting similar limitations to Claim 8 and is rejected for similar reasons.
Claim 17:
Claim 17 is the machine claim reciting similar limitations to Claim 8 and is rejected for similar reasons.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
U.S. Patent Application Publication NO. 20210383304 "Aquilizan" teaches in para. [0091], teaches that according to exemplary embodiments, the ITCMM 406 may be configured to review production data to identify features with potential ‘predictive power;’ create prediction hypotheses, for example, “do the number of project tracking systems and the number of applications impacted by a change impacts the success or failure of that change?” etc. The ITCMM 406 may further be configured to select a machine learning (ML) model and develop a ML pipeline locally using high-level general-purpose programming.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID H TAN whose telephone number is (571)272-7433. The examiner can normally be reached M-F 7:30-4:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Cesar Paula can be reached at (571) 272-4128. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/D.T./ Examiner, Art Unit 2145
/CESAR B PAULA/ Supervisory Patent Examiner, Art Unit 2145
Prosecution Insights