Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Examiner’s Note
Examiner notes that the claims were analyzed under the guidance of MPEP 2106 as possible judicial exceptions to eligibility under 35 U.S.C. 101 and found eligible. Examiner finds that claim 1 recites steps that can be performed as mental processes:
“performing, […], using historical data, the ML model estimation for identifying one or more parameters of the ML model”
‘determining, […], whether a distribution mismatch between the first distribution and the second distribution is present or not’
‘when the distribution mismatch is determined to be present’
‘comparing the calculated value against a reference data distribution”
“determining a difference between the calculated value and the reference data distribution and comparing the difference against a reference threshold’
“when the determined difference is greater than the reference threshold’
“determining an offset value based on the pessimistic penalty applied to the training dataset’
Other steps in claim 1 recites mathematical concepts:
“determining, […], a first distribution for the ML model based on training dataset”
“determining, […], a second distribution for the ML model based on the historical data”
“calculating a value for an individual state-action pair in the training dataset”
Thus, the claim recites an abstract idea.
The further step “receiving, by a processor, a machine learning (ML) model” recites mere data gathering, and the phrase “by the processor” in certain limitations merely indicates that a mental process or mathematical concept is performed by a generic computer.
However, Examiner finds that the steps ”removing the individual state-action pair from the training dataset as a pessimistic penalty” and “generating a modified ML model based on the determined offset value” are meaningful limitations that integrate the abstract idea into a practical application at Step 2A prong 2. The claim is therefore eligible under 35 U.S.C. 101.
Claim Objections
Claims 1–20 objected to because of the following informalities.
Claim 1 recites the step “determining, by the processor, a first distribution for the ML model based on training dataset.” Examiner respectfully suggests that “on training dataset” is intended to be read as “on a training dataset.” In further examination below, the limitation will be so interpreted. Further, claim 1 recites “performing, by the processor and using historical data, the ML model estimation for identifying one or more parameters of the ML model.” The wording suggests “the ML model estimation” is previously referenced in the claim but the phrase has no antecedent. In further examination below, the phrase will be interpreted as “an estimation of the ML model.”
Claims 11 and 20 are analogous to claim 1 and contain the same “on training dataset” and “the ML model estimation” phrases. Claims 2–10 and 12–19 are dependent on claim 1 or 11 and therefore contain the phrases.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1–20 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 1:
Claim 1 recites the method step “generating a modified ML model based on the determined offset value without retraining the ML model.” Examiner finds that a person having ordinary skill in the art would be unable to determine an unambiguous meaning for “without retraining the ML model” in the context of this claim, for example, whether the limitation merely requires a modification to a model based on “the determined offset value” that is, in some sense, separable from any retraining, or whether the limitation precludes any retraining. Further, Examiner notes that while the claim recites the use of data labelled as the “training dataset,” the claim does not include any initial training step, therefore the meaning of “retraining” itself is unclear; for example, if an initial training step were performed on a model after the model’s modification by a determined offset value, whether that would describe retraining, or mere training, and therefore whether the training step would describe “generating a modified ML model based on the determined offset value without retraining the ML model.” A person having ordinary skill in the art would thus be unable to determine the metes and bounds of the claim.
Regarding claims 11 and 20:
Claims 11 and 20 are analogous to claim 1 and are rejected by the same reasoning.
Regarding claims 2–9 and 12–19:
Claims 2–9 and 12–19 are dependent on claim 1 or claim 11 and are rejected by the same reasoning.
Regarding claim 10:
Claim 10 is dependent on claim 1 and is rejected by the same reasoning. Further, claim 10 recites “wherein the historical data is a closed dataset.” Examiner finds that the phrase “closed dataset” renders the claim indefinite. Neither the claim nor the specification provide a definition for “closed dataset.” In the mathematical and computing arts, the phrase has several meanings, including, but not limited to, a set that contains its boundary points, a finite set, or a set of data that is proprietary, non-public, or under copyright. A person having ordinary skill in the art would be unable to ascertain the precise meaning of the phrase to determine the metes and bounds of the claim.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1–3, 5, 8, 11–13, 15, 18, and 20 rejected under 35 U.S.C. 103 over Smuc et al., US Pre-Grant Publication No. 2010/0116658 (hereafter Smuc) in view of Harsha et al., US Pre-Grant Publication No. 2022/0207413 (hereafter Harhsa) and Chakraborty et al., “STEERING: Stein Information Directed Exploration for Model-Based Reinforcement Learning,” September 2023, arXiv:2301.12038v2 (hereafter Chakraborty).
Regarding claim 1 and analogous claims 11 and 20:
Smuc teaches:
“A method for performing information-directed pessimism in offline reinforcement learning for reduction of distribution mismatch, the method comprising”: Smuc, paragraph 0001, “The present invention relates to an analyser for determining the relative importance of fractions of biological mixtures, a method [method] of determining the relative importance of fractions of biological mixtures [reduction of distribution mismatch], a computer program comprising instructions which, when executed, cause an analyser to perform the method, a computer-readable medium comprising the computer program and a signal carrying the computer program.”
“determining, by the processor, a first distribution for the ML model based on training dataset”: Smuc, paragraph 0012, “filter the data set in the second attribute space using a feature selection method to determine which components of the data set are most relevant for determining the different physiological conditions by comparing for each individual component, the distribution of values for that component relating to the first physiological condition [a first distribution for the ML model based on training dataset, training dataset interpreted as a dataset used in the development of a model] and the distribution of values for that component relating to the second physiological condition and discarding those components where the difference between the distribution of values in respect of the first and second physiological conditions is low, to provide a filtered data set.”
“determining, by the processor, a second distribution for the ML model based on the historical data”: Smuc, paragraph 0012, “filter the data set in the second attribute space using a feature selection method to determine which components of the data set are most relevant for determining the different physiological conditions by comparing for each individual component, the distribution of values for that component relating to the first physiological condition and the distribution of values for that component relating to the second physiological condition [a second distribution for the ML model based on the historical data] and discarding those components where the difference between the distribution of values in respect of the first and second physiological conditions is low, to provide a filtered data set.”
“determining, by the processor, whether a distribution mismatch between the first distribution and the second distribution is present or not”: Smuc, paragraph 0012, “filter the data set in the second attribute space using a feature selection method to determine which components of the data set are most relevant for determining the different physiological conditions by comparing for each individual component, the distribution of values for that component relating to the first physiological condition and the distribution of values for that component relating to the second physiological condition and discarding those components where the difference between the distribution of values in respect of the first and second physiological conditions is low [whether a distribution mismatch between the first distribution and the second distribution is present or not], to provide a filtered data set.”
(bold only) “when the distribution mismatch is determined to be present: calculating a value for an individual state-action pair in the training dataset and comparing the calculated value against a reference data distribution”: Smuc, paragraph 0012, “filter the data set in the second attribute space using a feature selection method to determine which components of the data set are most relevant for determining the different physiological conditions by comparing for each individual component, the distribution of values for that component [calculating a value for an individual … in the training dataset] relating to the first physiological condition and the distribution of values for that component relating to the second physiological condition [comparing the calculated value against a reference data distribution] and discarding those components where the difference between the distribution of values in respect of the first and second physiological conditions is low [when the distribution mismatch is determined to be present], to provide a filtered data set.”
(bold only) “determining a difference between the calculated value and the reference data distribution and comparing the difference against a reference threshold; when the determined difference is greater than the reference threshold, removing the individual state-action pair from the training dataset as a pessimistic penalty” and (bold only) “determining an offset value based on the pessimistic penalty applied to the training dataset”: Smuc, paragraph 0012, “filter the data set in the second attribute space using a feature selection method to determine which components of the data set are most relevant for determining the different physiological conditions by comparing for each individual component, the distribution of values for that component relating to the first physiological condition and the distribution of values for that component relating to the second physiological condition and discarding those components where the difference between the distribution of values in respect of the first and second physiological conditions is low, to provide a filtered data set [determining a difference between the calculated value and the reference data distribution and comparing the difference against a reference threshold; when the determined difference is greater than the reference threshold, removing the individual … from the training dataset as a pessimistic penalty, pessimistic penalty interpreted as the removal of data from the dataset]”
Smuc does not explicitly teach:
“performing, by the processor and using historical data, the ML model estimation for identifying one or more parameters of the ML model”
(bold only) “when the distribution mismatch is determined to be present: calculating a value for an individual state-action pair in the training dataset and comparing the calculated value against a reference data distribution”
(bold only) “determining a difference between the calculated value and the reference data distribution and comparing the difference against a reference threshold; when the determined difference is greater than the reference threshold, removing the individual state-action pair from the training dataset as a pessimistic penalty”
(bold only) “determining an offset value based on the pessimistic penalty applied to the training dataset”
“generating a modified ML model based on the determined offset value without retraining the ML model”
Harsha teaches:
“performing, by the processor and using historical data, the ML model estimation for identifying one or more parameters of the ML model”: Harsha, paragraph 0018, “In some embodiments, during the stochastic gradient descent, a batch of the training data [historical data, interpreted as pre-existing data] are randomly sampled (e.g., 20 examples/data points from the full training data set). For each sample in the batch, the system determines a set of modified example inputs to include in the loss-e.g., the most violated points that the system can find nearby under the current model parameters [the ML model estimation for identifying one or more parameters of the ML model, interpreted as a method step that connects model parameters to other variables] (neural net weights of the machine learning model). After the batch is processed, the model parameters are updated. Thus, even if the machine learning system encounters the same batch of data in the future, the most-violated points may be different at that time, or a different set of random samples will be used for determining the additional penalty loss.”
“receiving, by a processor, a machine learning (ML) model”: Harsha, paragraph 0017, “In some embodiments, a loss augmentation method is used to provide the set of learning transforms necessary to implement a set of arbitrary constraints (e.g., industry constraints such as the selected domain-specific constraints and general functional relationships) in a DL machine learning model.”
(bold only) “determining an offset value based on the pessimistic penalty applied to the training dataset”: Harsha, paragraph 0017, “In some embodiments, a loss augmentation method is used to provide the set of learning transforms necessary to implement a set of arbitrary constraints (e.g., industry constraints such as the selected domain-specific constraints and general functional relationships) in a DL machine learning model. Loss augmentation is a method that uses modified examples of inputs (e.g., random or selected via optimization) with custom losses that capture the desired functional behaviors [determining an offset value based on the pessimistic penalty applied to the training dataset, interpreted as determining values base on modified input data]. For each data point (or input point) in the training set (or a current batch or subset of the training set in a current model optimization iteration), additional data points are sampled and added to the training set ( or the current batch), as part of the model setup or training procedure. Additional loss function component (s) may be added to the regular loss function components in the objective function of the machine learning model. The additional loss function(s) may be specified to penalize behaviors (so the additional loss function is also referred to as penalty loss function) that do not conform to the desired functional behavior and relationships according to the arbitrary constraints.”
“generating a modified ML model based on the determined offset value without retraining the ML model”: Harsha, paragraph 0017, “In some embodiments, a loss augmentation method is used to provide the set of learning transforms necessary to implement a set of arbitrary constraints (e.g., industry constraints such as the selected domain-specific constraints and general functional relationships) in a DL machine learning model. Loss augmentation is a method that uses modified examples of inputs (e.g., random or selected via optimization) with custom losses that capture the desired functional behaviors. For each data point (or input point) in the training set (or a current batch or subset of the training set in a current model optimization iteration), additional data points are sampled and added to the training set ( or the current batch), as part of the model setup or training procedure. Additional loss function component (s) may be added to the regular loss function components in the objective function of the machine learning model. The additional loss function(s) may be specified to penalize behaviors (so the additional loss function is also referred to as penalty loss function) that do not conform to the desired functional behavior and relationships according to the arbitrary constraints [generating a modified ML model based on the determined offset value without retraining the ML model, interpreted as modifications to a model based on modified input data, the modifications separate to alterations to model weights].”
Harsha and Smuc are analogous arts as they are both related to model optimization. It would have been obvious to a person having ordinary skill in the art prior to the effective filing date of the claimed invention to have combined the penalty loss function of Harsha with the teachings of Smuc to arrive at the present invention, in order to provide model constraints, as stated in Harsha, paragraph 0008, “By modifying the training data set and the objective function of a machine learning model to incorporate penalty data points and penalty loss function, arbitrary constraints (e.g., industry constraints) and known relationships can be incorporated into deep learning. A more automatic, accurate, scalable and efficient predictive modeling system that generalizes correctly to unseen data, to drive downstream industry systems and processes is therefore realized.”
Chakraborty teaches (bold only) “when the distribution mismatch is determined to be present: calculating a value for an individual state-action pair in the training dataset and comparing the calculated value against a reference data distribution” and (bold only) “determining a difference between the calculated value and the reference data distribution and comparing the difference against a reference threshold; when the determined difference is greater than the reference threshold, removing the individual state-action pair from the training dataset as a pessimistic penalty”: Chakraborty, Section 3.3, “For simplicity and analysis in this subsection, let us denote the state-action pair (s, a) → x ∈ X := S × A [state-action pair] and the corresponding next state s′ → y ∈ S.”
Chakraborty and Smuc are analogous arts as they are both related to model training. It would have been obvious to a person having ordinary skill in the art prior to the effective filing date of the claimed invention to have applied the reinforcement learning state-action pairs of Chakraborty with the teachings of Smuc to arrive at the present invention, in order to more accurately direct learning, as stated in Chakraborty, Abstract, “Directed Exploration is a crucial challenge in reinforcement learning (RL), especially when rewards are sparse. Information-directed sampling (IDS), which optimizes the information ratio, seeks to do so by augmenting regret with information gain.”
Regarding claim 2 and analogous claim 12:
Smuc as modified by Harhsa and Chakraborty teaches “[t]he method according to claim 1.”
Harsha further teaches “updating parameters based on the pessimistic penalty applied to the training dataset”: Harsha, paragraph 0017, “In some embodiments, a loss augmentation method is used to provide the set of learning transforms necessary to implement a set of arbitrary constraints (e.g., industry constraints such as the selected domain-specific constraints and general functional relationships) in a DL machine learning model. Loss augmentation is a method that uses modified examples of inputs (e.g., random or selected via optimization) with custom losses that capture the desired functional behaviors. For each data point (or input point) in the training set (or a current batch or subset of the training set in a current model optimization iteration), additional data points are sampled and added to the training set ( or the current batch), as part of the model setup or training procedure. Additional loss function component (s) may be added to the regular loss function components in the objective function of the machine learning model. The additional loss function(s) may be specified to penalize behaviors (so the additional loss function is also referred to as penalty loss function) that do not conform to the desired functional behavior and relationships according to the arbitrary constraints [updating parameters based on the pessimistic penalty applied to the training dataset, interpreted as modifications to a model based on modified input data, the modifications separate to alterations to model weights].”
Harsha and Smuc are combinable for the rationale given under claim 1.
Regarding claim 3 and analogous claim 13:
Smuc as modified by Harhsa and Chakraborty teaches “[t]he method according to claim 1.”
Smuc further teaches:
(bold only) “determining, by the processor, the second distribution for the ML model based on the updated historical data”: Smuc, paragraph 0012, “filter the data set in the second attribute space using a feature selection method to determine which components of the data set are most relevant for determining the different physiological conditions by comparing for each individual component, the distribution of values for that component relating to the first physiological condition and the distribution of values for that component relating to the second physiological condition [the second distribution for the ML model based on the … historical data] and discarding those components where the difference between the distribution of values in respect of the first and second physiological conditions is low, to provide a filtered data set.”
(bold only) “determining whether the distribution mismatch between the first distribution and the second distribution is present or not based on the updated historical data”: Smuc, paragraph 0012, “filter the data set in the second attribute space using a feature selection method to determine which components of the data set are most relevant for determining the different physiological conditions by comparing for each individual component, the distribution of values for that component relating to the first physiological condition and the distribution of values for that component relating to the second physiological condition and discarding those components where the difference between the distribution of values in respect of the first and second physiological conditions is low [determining whether the distribution mismatch between the first distribution and the second distribution is present or not based on the … historical data], to provide a filtered data set.”
Harsha further teaches “receiving an update to the historical data,” (bold only) “determining, by the processor, the second distribution for the ML model based on the updated historical data,” and (bold only) “determining whether the distribution mismatch between the first distribution and the second distribution is present or not based on the updated historical data”: Harsha, paragraph 0017, “In some embodiments, a loss augmentation method is used to provide the set of learning transforms necessary to implement a set of arbitrary constraints (e.g., industry constraints such as the selected domain-specific constraints and general functional relationships) in a DL machine learning model. Loss augmentation is a method that uses modified examples of inputs (e.g., random or selected via optimization) with custom losses that capture the desired functional behaviors. For each data point (or input point) in the training set (or a current batch or subset of the training set in a current model optimization iteration), additional data points are sampled and added to the training set ( or the current batch), as part of the model setup or training procedure [receiving an update to the historical data].”
Harsha and Smuc are combinable for the rationale given under claim 1.
Regarding claim 5 and analogous claim 15:
Smuc as modified by Harhsa and Chakraborty teaches “[t]he method according to claim 1.”
Chakraborty further teaches “wherein the calculated value is a Stein kernel”: Chakraborty, section 1, paragraph 5, “That is, we propose Stein information gain, which is the integral probability metric (IPM) difference between the estimated and true (unknown) transition dynamics (Sriperumbudur et al., 2012), hence inducing directed exploration. Under the assumption that the transition model lies in the Stein class, we employ Stein’s identity (Efron & Morris, 1973; James & Stein, 1992) to evaluate this IPM between the true (unknown) and estimated transitions in closed-form using kernelized Stein discrepancy (KSD) [the calculated value is a Stein kernel] (Gorham & Mackey, 2015; Liu et al., 2016; Hawkins et al.).”
Chakraborty and Smuc are analogous arts as they are both related to model training. It would have been obvious to a person having ordinary skill in the art prior to the effective filing date of the claimed invention to have combined the kernelized Stein discrepancy of Chakraborty with the teachings of Smuc to arrive at the present invention, in order to more accurately direct learning, as stated in Chakraborty, section 1, paragraph , “We emphasize that our notion of KSD-based Stein information gain empowers us to evaluate the distance to the true transition dynamics. Doing so permits us to derive the best-known prior-free information-theoretic Bayesian regret bounds.”
Regarding claim 8 and analogous claim 18:
Smuc as modified by Harhsa and Chakraborty teaches “[t]he method according to claim 1.”
Smuc further teaches (bold only) “wherein the removing the individual state-action pair from the training dataset as the pessimistic penalty is performed during the offline reinforcement learning”: Smuc, paragraph 0012, “filter the data set in the second attribute space using a feature selection method to determine which components of the data set are most relevant for determining the different physiological conditions by comparing for each individual component, the distribution of values for that component relating to the first physiological condition and the distribution of values for that component relating to the second physiological condition and discarding those components where the difference between the distribution of values in respect of the first and second physiological conditions is low, to provide a filtered data set [removing the individual … from the training dataset as the pessimistic penalty, pessimistic penalty interpreted as the removal of data from the dataset]”
Chakraborty further teaches (bold only) “wherein the removing the individual state-action pair from the training dataset as the pessimistic penalty is performed during the offline reinforcement learning”: Chakraborty, Section 3.3, “For simplicity and analysis in this subsection, let us denote the state-action pair (s, a) → x ∈ X := S × A [state-action pair] and the corresponding next state s′ → y ∈ S”; Chakraborty, section E, paragraph 1, “We run a local optimization procedure by dividing the total number of samples H in an episode into batches of size Z with Z' := H / Z batches and select Stein optimal points per batch using an SPMCMC style update [performed during the offline reinforcement learning].”
Chakraborty and Smuc are combinable for the rationale given under claim 1.
Claims 4, 7, and 9 rejected under 35 U.S.C. 103 over Smuc as modified by Harhsa and Chakraborty in view of Liu et al., US Pre-Grant Publication No. 2024/0346244 (hereafter Liu).
Regarding claim 4 and analogous claim 14:
Smuc as modified by Harhsa and Chakraborty teaches “[t]he method according to claim 1.”
Smuc as modified by Harhsa and Chakraborty does not explicitly teach “when the distribution mismatch is determined to be absent, continue utilization of the ML model without modification.”
Liu teaches “when the distribution mismatch is determined to be absent, continue utilization of the ML model without modification”: Liu, paragraph 0155, “At block 318, it may be determined whether a data distribution similarity may be above a threshold. In some embodiments, to determine whether the data distribution similarity corresponding to data in the numerical data subset pair may be above a threshold, a data distribution similarity may be detected and/or determined”; Liu, paragraph 0156, “In some embodiments, in the event the numerical data subset pair includes a data distribution similarity under the threshold, the numerical data subset pair may be discarded or otherwise not considered to generate and/or synthesize new data to add to the dataset. In some embodiments, in the event that the numerical data subset pair includes a data distribution similarity above the threshold, the method may proceed to block 320 [when the distribution mismatch is determined to be absent, continue utilization of the ML model without modification].”
Liu and Smuc are analogous arts as they are both related to data modelling. It would have been obvious to a person having ordinary skill in the art prior to the effective filing date of the claimed invention to have combined the selective model modification of Liu with the teachings of Smuc to arrive at the present invention, in order to assure the quality of model data, as stated in Liu, paragraph 0156, “In some embodiments, using less relevant or irrelevant new data may result in less accurate predictions using the machine learning model than predictions that may be made by the machine learning model that may have been trained using relevant data.”
Regarding claim 7 and analogous claim 17:
Smuc as modified by Harhsa and Chakraborty teaches “[t]he method according to claim 1.”
Chakraborty further teaches (bold only) “wherein, when the determined difference is less than the reference threshold, retaining the individual state-action pair in the training dataset”: Chakraborty, Section 3.3, “For simplicity and analysis in this subsection, let us denote the state-action pair (s, a) → x ∈ X := S × A [state-action pair] and the corresponding next state s′ → y ∈ S.”
Chakraborty and Smuc are combinable for the rationale given under claim 1.
Smuc as modified by Harhsa and Chakraborty does not explicitly teach (bold only) “wherein, when the determined difference is less than the reference threshold, retaining the individual state-action pair in the training dataset.”
Liu teaches (bold only) “wherein, when the determined difference is less than the reference threshold, retaining the individual state-action pair in the training dataset”: Liu, paragraph 0154, “In some embodiments, in the event the numerical data subset pair includes a data range difference over the threshold, the numerical data subset pair may be discarded, filtered out, and/or otherwise not considered to generate and/or synthesize new data to add to the dataset. In some embodiments, in the event that the numerical data subset pair includes a data range difference under the threshold, the method may proceed to block 318 [wherein, when the determined difference is less than the reference threshold, retaining the individual … in the training dataset].”
Liu and Smuc are analogous arts as they are both related to data modelling. It would have been obvious to a person having ordinary skill in the art prior to the effective filing date of the claimed invention to have combined the selective model modification of Liu with the teachings of Smuc to arrive at the present invention, in order to assure the quality of model data, as stated in Liu, paragraph 0156, “In some embodiments, using less relevant or irrelevant new data may result in less accurate predictions using the machine learning model than predictions that may be made by the machine learning model that may have been trained using relevant data.”
Regarding claim 9 and analogous claim 19:
Smuc as modified by Harhsa and Chakraborty teaches “[t]he method according to claim 1.”
Smuc as modified by Harhsa and Chakraborty does not explicitly teach “when the distribution mismatch is determined to be below a reference threshold, continue utilization of the ML model without modification.”
Liu teaches “when the distribution mismatch is determined to be below a reference threshold, continue utilization of the ML model without modification”: Liu, paragraph 0155, “At block 318, it may be determined whether a data distribution similarity may be above a threshold. In some embodiments, to determine whether the data distribution similarity corresponding to data in the numerical data subset pair may be above a threshold, a data distribution similarity may be detected and/or determined”; Liu, paragraph 0156, “In some embodiments, in the event the numerical data subset pair includes a data distribution similarity under the threshold, the numerical data subset pair may be discarded or otherwise not considered to generate and/or synthesize new data to add to the dataset. In some embodiments, in the event that the numerical data subset pair includes a data distribution similarity above the threshold, the method may proceed to block 320 [when the distribution mismatch is determined to be below a reference threshold, continue utilization of the ML model without modification].”
Liu and Smuc are analogous arts as they are both related to data modelling. It would have been obvious to a person having ordinary skill in the art prior to the effective filing date of the claimed invention to have combined the selective model modification of Liu with the teachings of Smuc to arrive at the present invention, in order to assure the quality of model data, as stated in Liu, paragraph 0156, “In some embodiments, using less relevant or irrelevant new data may result in less accurate predictions using the machine learning model than predictions that may be made by the machine learning model that may have been trained using relevant data.”
Claims 6 and 16 rejected under 35 U.S.C. 103 over Smuc as modified by Harhsa and Chakraborty in view of Sprenkle, US Pre-Grant Publication No. 2025/0112026 (hereafter Sprenkle).
Smuc as modified by Harhsa and Chakraborty teaches “[t]he method according to claim 1.”
Smuc as modified by Harhsa and Chakraborty does not explicitly teach “deploying the modified ML model to a production environment; collecting data in the production environment using the modified ML model; and supplementing the historical data with the data collected in the production environment.”
Sprenkle teaches “deploying the modified ML model to a production environment; collecting data in the production environment using the modified ML model; and supplementing the historical data with the data collected in the production environment”: Sprenkle, paragraph 0128, “The model evaluator 920 may be communicatively coupled to a model inference 926. The model inference 926 provides AI/ML model inference output ( e.g., predictions or decisions). Once the ML model is trained and evaluated, it can be deployed in a production environment where it can be used to make predictions on new data [deploying the modified ML model to a production environment]. The model inference 926 receives the evaluated model 922 input 924. The model inference 926 may use the evaluated model 922 as a deployed model 928, which is a final production ML model. The inference output of the deployed model 928 is use case specific. The model inference 926 may also perform model monitoring and maintenance, which involves continuously monitoring performance of the deployed model 928 in the production environment [collecting data in the production environment using the modified ML model] and making any necessary updates or modifications to maintain its accuracy and effectiveness. The model inference 926 may provide feedback 932 to the data collector 906 to train or re-train the ML model. The feedback 932 may include model performance feedback information, which may be used for monitoring and improving performance of the deployed model 928 [supplementing the historical data with the data collected in the production environment].”
Sprenkle and Smuc are analogous arts as they are both related to data modelling. It would have been obvious to a person having ordinary skill in the art prior to the effective filing date of the claimed invention to have combined the production environment updating of Sprenkle with the teachings of Smuc to arrive at the present invention, in order to improve the model with production environment data, as stated in Sprenkle, paragraph 0128, “The model inference 926 may provide feedback 932 to the data collector 906 to train or re-train the ML model. The feedback 932 may include model performance feedback information, which may be used for monitoring and improving performance of the deployed model 928.”
Claim 10 rejected under 35 U.S.C. 103 over Smuc as modified by Harhsa and Chakraborty in view of Aghdasi, US Pre-Grant Publication No. 2022/0253734 (hereafter Aghdasi).
Smuc as modified by Harhsa and Chakraborty teaches “[t]he method according to claim 1.”
Smuc as modified by Harhsa and Chakraborty does not explicitly teach “wherein the historical data is a closed dataset.”
Aghdasi teaches “wherein the historical data is a closed dataset”: Aghdasi, paragraph 0041, “More specifically, concrete ML/optimization system(s) 110 can use proprietary datasets supplemented by partner data, as well as cutting edge AI tools. These can include, inter alia: such as model-based reinforcement learning, Bayesian optimization, model-based multi-armed bandits, convolutional neural networks, generative adversarial networks and other cutting-edge machine learning algorithms to predict the properties of millions of combinations of raw materials as well as organic and chemical admixtures and supplemental cementitious materials used in cement and concrete production, and producing and testing material structures and properties using robotic systems to quickly converge on optimal mixes or to discover new cements or concrete materials. This enables drastic improvements in cost and performance characteristics for each project.”
Aghdasi and Smuc are analogous arts as they are both related to data modelling. It would have been obvious to a person having ordinary skill in the art prior to the effective filing date of the claimed invention to have combined the use of closed datasets from Aghdasi with the teachings of Smuc to arrive at the present invention, in order to produce better models using proprietary data, as stated in Aghdasi, paragraph 0041, “This enables drastic improvements in cost and performance characteristics for each project.”
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Goldsteen et al., US Pre-Grant Publication No. 2022/0309381, discloses a method of computing model similarity by the comparison of distributions of two sets of training samples, in order to verify the removal of a data item.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to VINCENT SPRAUL whose telephone number is (703) 756-1511. The examiner can normally be reached M-F 9:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, MICHAEL HUNTLEY can be reached at (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/VAS/Examiner, Art Unit 2129
/MICHAEL J HUNTLEY/Supervisory Patent Examiner, Art Unit 2129
Prosecution Insights