Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Claims 1-20 are presented in the case.
Allowable Subject Matter
Claims 3, 10 and 17 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-2, 4-9, 11-16 and 18-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The analysis of the claims will follow the 2019 Revised Patent Subject Matter Eligibility Guidance, 84 Fed. Reg. 50 (“2019 PEG”)
Claim 1, 8 and 15 have the following abstract idea analysis.
Step 1: The claim is directed to “a method, system and crm”. The claims are directed to the statutory categories accordingly.
Step 2A Prong 1: claims recite the abstract idea limitations of "identifying a first set of data samples of the one or more data samples that satisfies a threshold criterion;”. These limitations include mental concepts (act of evaluating. Mental processes – concepts performed in the human mind (including an observation, evaluation, judgment, opinion) (see MPEP § 2106.04(a)(2), subsection III). The specification also provides example operations performed such as identifying misclassified data samples. USPGPUB ¶43. Other sections of the claims such as "obtaining, by a processing device, a first output from a machine learning model, wherein the first output comprises one or more data samples; generating, using an explainability tool, a weighted value for each data sample of the first set of data samples; and modifying the machine learning model based at least in part on the weighted value for each data sample of the first set of data samples." are advanced processes, too generic or high level to be listed as a judicial exception given the available descriptions and MPEP comparisons.
Step 2A Prong 2: The judicial exceptions recited in these claims are not integrated into a practical application. Merely invoking "a machine learning model", "a processor" or "memory" does not yield eligibility. Claims are still in line with mental concepts such as claim 1, 8 and 15 are not specific to a practical application. The additional elements as such are processors and instructions which do not include specialized hardware. See MPEP § 2106.05(f).
Claim 1, 8 and 15 do not include a particular field but even doing so may not be sufficient to overcome the abstract idea rejection. Merely applying an model to a field or data without an advancement in the new field or new hardware is ineligible. MPEP § 2106.05(h).
Step 2B: The claims do not contain significantly more than their judicial exceptions. Processors, memory and other hardware are in their standard forms in the field. These additional elements are well-understood, routine, and conventional activity, see MPEP 2106.05(d)(II). Claims lacks any particular "how" or algorithm for a solution in a field in a novel way. Claims require more specificity on processes that would be incapable of simple mathematics, mental processes or use more substantial structure than conventional devices such as non-textbook implementations.
Regarding claims 2, 4-7, 9, 11-14, 16 and 18-20 merely narrow the previously recited abstract idea limitations with more abstract concepts and/or routine fundamental processes. For the reasons described above with respect to claim 1 and 9 this judicial exception is not meaningfully integrated into a practical application, or significantly more than the abstract idea. Abstract idea steps 1, 2A prong 1 and 2 remain the same as independent analysis above. See specification for more practical application concepts as none are seen in claims 2-8 and 10-15.
With respect to step 2B These claims disclose similar limitations described for the independent claims above and do not provide anything significantly more than mathematical or mental concepts. Claims 2, 4-7, 9, 11-14, 16 and 18-20 recite the additional elements of "wherein identifying the first set of data samples that satisfies the threshold criterion comprises identifying one or more misclassified data samples of the one or more data samples. validating the modified machine learning model using one or more new data samples. modifying the first set of data samples by multiplying the weighted value to each data sample of the first set of data samples; and retraining the machine learning model using the modified first set of data samples." These elements are more abstract concepts, generic applications to a field of use or well-understood, routine, conventional activity (see MPEP § 2106.05(d) and can't be simply appended to qualify as significantly more or being a practical application. What type of application, or structure of components beyond generic machine learning is still unknown for these claims. Therefore claims 2, 4-7, 9, 11-14, 16 and 18-20 also recite abstract ideas that do not integrate into a practical application or amount to significantly more than the judicial exception, and are rejected under U.S.C. 101.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 8 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over WENCHEL et al. (US 20210174258 A1 hereinafter Wenchel) in view of Vijaykeerthy et al. (US 11636331 B2 hereinafter Vijaykeerthy)
As to independent claim 1, Wenchel teaches a method comprising:
obtaining, by a processing device, a first output from a machine learning model, wherein the first output comprises one or more data samples; [provides model outputs Fig. 1 114 that include features and metrics such as risk scores ¶50, ¶44 " ML model inferences and provide model outputs 114 (e.g., to one or more users, in a manner that is interpretable by the one or more users). The model outputs 114 can include explanations of the ML model inferences. The explanations can be based on the metrics at any phase of the system (pre, during, and/or post model inference). The explanations can be stored with such metrics, for example in common records of a database or other storage medium. Such explanations can include indications of data features and their associated feature importance."]
identifying a first set of data samples of the one or more data samples that satisfies a threshold criterion; [thresholds based on a risk metric ¶68 "a modal alert to the user based on a threshold (e.g., a maximum or a minimum) single risk score;"]
generating, using an explainability tool, a weighted value for each data sample of the first set of data samples; and [LIME tool based values ¶47 " Examples of single metrics 112 (e.g., based on individual data or data sets from the received data stream—also referred to herein as “non-aggregated data metrics”) can include, but are not limited to: minimum Local Interpretable Model-Agnostic Explanations (“LIME”) value, maximum LIME value, variance of LIME value, and gradient values"; ¶62 "weights for features or sets of features representing a significance"]
modifying the machine learning model [retraining model ¶47 "adaptation of training data, resampling of data from the data stream 102, retraining of the ML model 110A"]
Wenchel does not specifically teach modifying the machine learning model based at least in part on the weighted value for each data sample of the first set of data samples.
However, Vijaykeerthy teaches modifying the machine learning model based at least in part on the weighted value for each data sample of the first set of data samples. [retrain based on explanation data (weight result from explanations data) Col. 6 ln. 23-67 "The model may then be retrained using the trainable explanation-guided loss, which guides the model to focus on the actual important regions (indicated by the annotated explanation) while learning."]
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the learning model disclosed by Wenchel by incorporating the modifying the machine learning model based at least in part on the weighted value for each data sample of the first set of data samples disclosed by Vijaykeerthy because both techniques address the same field of machine learning and by incorporating Vijaykeerthy into Wenchel improves balance and reduces bias of models for better performance [Vijaykeerthy Col. 2 ln. 45-58]
As to independent claim 8, Wenchel teaches a system comprising: [ML monitoring system ¶44]
a memory; and [memories ¶92]
a processing device operatively coupled to the memory, wherein the processing device is to: [processors ¶92]
obtain a first output from a machine learning model, wherein the first output comprises one or more data samples; [provides model outputs Fig. 1 114 that include features and metrics such as risk scores ¶50, ¶44 " ML model inferences and provide model outputs 114 (e.g., to one or more users, in a manner that is interpretable by the one or more users). The model outputs 114 can include explanations of the ML model inferences. The explanations can be based on the metrics at any phase of the system (pre, during, and/or post model inference). The explanations can be stored with such metrics, for example in common records of a database or other storage medium. Such explanations can include indications of data features and their associated feature importance."]
identify a first set of data samples of the one or more data samples that satisfies a threshold criterion; [thresholds based on a risk metric ¶68 "a modal alert to the user based on a threshold (e.g., a maximum or a minimum) single risk score;"]
generate, using an explainability tool, a weighted value for each data sample of the first set of data samples; and [LIME tool based values ¶47 " Examples of single metrics 112 (e.g., based on individual data or data sets from the received data stream—also referred to herein as “non-aggregated data metrics”) can include, but are not limited to: minimum Local Interpretable Model-Agnostic Explanations (“LIME”) value, maximum LIME value, variance of LIME value, and gradient values"; ¶62 "weights for features or sets of features representing a significance"]
modify the machine learning model [retraining model ¶47 "adaptation of training data, resampling of data from the data stream 102, retraining of the ML model 110A"]
Wenchel does not specifically teach modify the machine learning model based at least in part on the weighted value for each data sample of the first set of data samples.
However, Vijaykeerthy teaches modify the machine learning model based at least in part on the weighted value for each data sample of the first set of data samples. [retrain based on explanation data (weight result from explanations data) Col. 6 ln. 23-67 "The model may then be retrained using the trainable explanation-guided loss, which guides the model to focus on the actual important regions (indicated by the annotated explanation) while learning."]
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the learning model disclosed by Wenchel by incorporating the modifying the machine learning model based at least in part on the weighted value for each data sample of the first set of data samples disclosed by Vijaykeerthy because both techniques address the same field of machine learning and by incorporating Vijaykeerthy into Wenchel improves balance and reduces bias of models for better performance [Vijaykeerthy Col. 2 ln. 45-58]
As to independent claim 15, Wenchel teaches a non-transitory computer-readable storage medium comprising instructions that, when executed by a processing device, cause the processing device to perform operations comprising: [memories, processor and instructions ¶92]
obtaining, by a processing device, a first output from a machine learning model, wherein the first output comprises one or more data samples; [provides model outputs Fig. 1 114 that include features and metrics such as risk scores ¶50, ¶44 " ML model inferences and provide model outputs 114 (e.g., to one or more users, in a manner that is interpretable by the one or more users). The model outputs 114 can include explanations of the ML model inferences. The explanations can be based on the metrics at any phase of the system (pre, during, and/or post model inference). The explanations can be stored with such metrics, for example in common records of a database or other storage medium. Such explanations can include indications of data features and their associated feature importance."]
identifying a first set of data samples of the one or more data samples that satisfies a threshold criterion; [thresholds based on a risk metric ¶68 "a modal alert to the user based on a threshold (e.g., a maximum or a minimum) single risk score;"]
generating, using an explainability tool, a weighted value for each data sample of the first set of data samples; and [LIME tool based values ¶47 " Examples of single metrics 112 (e.g., based on individual data or data sets from the received data stream—also referred to herein as “non-aggregated data metrics”) can include, but are not limited to: minimum Local Interpretable Model-Agnostic Explanations (“LIME”) value, maximum LIME value, variance of LIME value, and gradient values"; ¶62 "weights for features or sets of features representing a significance"]
modifying the machine learning model [retraining model ¶47 "adaptation of training data, resampling of data from the data stream 102, retraining of the ML model 110A"]
Wenchel does not specifically teach modifying the machine learning model based at least in part on the weighted value for each data sample of the first set of data samples.
However, Vijaykeerthy teaches modifying the machine learning model based at least in part on the weighted value for each data sample of the first set of data samples. [retrain based on explanation data (weight result from explanations data) Col. 6 ln. 23-67 "The model may then be retrained using the trainable explanation-guided loss, which guides the model to focus on the actual important regions (indicated by the annotated explanation) while learning."]
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the learning model disclosed by Wenchel by incorporating the modifying the machine learning model based at least in part on the weighted value for each data sample of the first set of data samples disclosed by Vijaykeerthy because both techniques address the same field of machine learning and by incorporating Vijaykeerthy into Wenchel improves balance and reduces bias of models for better performance [Vijaykeerthy Col. 2 ln. 45-58]
Claims 2, 9 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Wenchel and Vijaykeerthy, as applied in the rejection of claim 1, 8 and 15 above, and further in view of Chitiveli et al. (US 8352386 B2 hereinafter Chitiveli)
As to dependent claim 2, Wenchel and Vijaykeerthy teach the method of claim 1 above that is incorporated,
Wenchel and Vijaykeerthy do not specifically teach wherein identifying the first set of data samples that satisfies the threshold criterion comprises identifying one or more misclassified data samples of the one or more data samples.
However, Chitiveli teaches wherein identifying the first set of data samples that satisfies the threshold criterion comprises identifying one or more misclassified data samples of the one or more data samples. [Chitiveli finds misclassified for retraining Col. 10 ln. 49-67 "documents misclassified by the classifier may be used as additional training examples"]
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the modeling disclosed by Wenchel and Vijaykeerthy by incorporating the wherein identifying the first set of data samples that satisfies the threshold criterion comprises identifying one or more misclassified data samples of the one or more data samples disclosed by Chitiveli because all techniques address the same field of machine learning and by incorporating Chitiveli into Wenchel and Vijaykeerthy provides more accurate and reliable categorization of unstructured data [Chitiveli Col. 1 ln. 11-38]
As to dependent claim 9, Wenchel and Vijaykeerthy teach the method of claim 8 above that is incorporated,
Wenchel and Vijaykeerthy do not specifically teach wherein identifying the first set of data samples that satisfies the threshold criterion comprises identifying one or more misclassified data samples of the one or more data samples.
However, Chitiveli teaches wherein identifying the first set of data samples that satisfies the threshold criterion comprises identifying one or more misclassified data samples of the one or more data samples. [Chitiveli finds misclassified for retraining Col. 10 ln. 49-67 "documents misclassified by the classifier may be used as additional training examples"]
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the modeling disclosed by Wenchel and Vijaykeerthy by incorporating the wherein identifying the first set of data samples that satisfies the threshold criterion comprises identifying one or more misclassified data samples of the one or more data samples disclosed by Chitiveli because all techniques address the same field of machine learning and by incorporating Chitiveli into Wenchel and Vijaykeerthy provides more accurate and reliable categorization of unstructured data [Chitiveli Col. 1 ln. 11-38]
As to dependent claim 16, Wenchel and Vijaykeerthy teach the method of claim 15 above that is incorporated,
Wenchel and Vijaykeerthy do not specifically teach wherein identifying the first set of data samples that satisfies the threshold criterion comprises identifying one or more misclassified data samples of the one or more data samples.
However, Chitiveli teaches wherein identifying the first set of data samples that satisfies the threshold criterion comprises identifying one or more misclassified data samples of the one or more data samples. [Chitiveli finds misclassified for retraining Col. 10 ln. 49-67 "documents misclassified by the classifier may be used as additional training examples"]
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the modeling disclosed by Wenchel and Vijaykeerthy by incorporating the wherein identifying the first set of data samples that satisfies the threshold criterion comprises identifying one or more misclassified data samples of the one or more data samples disclosed by Chitiveli because all techniques address the same field of machine learning and by incorporating Chitiveli into Wenchel and Vijaykeerthy provides more accurate and reliable categorization of unstructured data [Chitiveli Col. 1 ln. 11-38]
Claims 4, 11 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Wenchel and Vijaykeerthy, as applied in the rejection of claim 1, 8 and 15 above, and further in view of Ravishankar et al. (US 20230238134 A1 Ravishankar)
As to dependent claim 4, Wenchel and Vijaykeerthy teach the method of claim 1 above that is incorporated,
Wenchel and Vijaykeerthy do not specifically teach validating the modified machine learning model using one or more new data samples.
However, Ravishankar teaches validating the modified machine learning model using one or more new data samples. [validation dataset for validating model with different data ¶55 "In some embodiments, the loss may be determined using a validation dataset, wherein the validation dataset is distinct from the training dataset and comprises data not seen by the tri-net deep learning model during training. In this way, the method 400 enables the tri-net deep neural network to learn features of ECG data and vital sign data that enable cardiac arrhythmias to be predicted before they occur."]
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the modeling disclosed by Wenchel and Vijaykeerthy by incorporating the validating the modified machine learning model using one or more new data samples disclosed by Ravishankar because all techniques address the same field of machine learning and by incorporating Ravishankar into Wenchel and Vijaykeerthy enables improved models with more accurate predictions that are validated [Ravishankar ¶28, ¶15]
As to dependent claim 11, Wenchel and Vijaykeerthy teach the method of claim 8 above that is incorporated,
Wenchel and Vijaykeerthy do not specifically teach validate the modified machine learning model using one or more new data samples.
However, Ravishankar teaches validate the modified machine learning model using one or more new data samples. [validation dataset for validating model with different data ¶55 "In some embodiments, the loss may be determined using a validation dataset, wherein the validation dataset is distinct from the training dataset and comprises data not seen by the tri-net deep learning model during training. In this way, the method 400 enables the tri-net deep neural network to learn features of ECG data and vital sign data that enable cardiac arrhythmias to be predicted before they occur."]
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the modeling disclosed by Wenchel and Vijaykeerthy by incorporating the validate the modified machine learning model using one or more new data samples disclosed by Ravishankar because all techniques address the same field of machine learning and by incorporating Ravishankar into Wenchel and Vijaykeerthy enables improved models with more accurate predictions that are validated [Ravishankar ¶28, ¶15]
As to dependent claim 18, Wenchel and Vijaykeerthy teach the method of claim 15 above that is incorporated,
Wenchel and Vijaykeerthy do not specifically teach validate the modified machine learning model using one or more new data samples.
However, Ravishankar teaches validate the modified machine learning model using one or more new data samples. [validation dataset for validating model with different data ¶55 "In some embodiments, the loss may be determined using a validation dataset, wherein the validation dataset is distinct from the training dataset and comprises data not seen by the tri-net deep learning model during training. In this way, the method 400 enables the tri-net deep neural network to learn features of ECG data and vital sign data that enable cardiac arrhythmias to be predicted before they occur."]
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the modeling disclosed by Wenchel and Vijaykeerthy by incorporating the validate the modified machine learning model using one or more new data samples disclosed by Ravishankar because all techniques address the same field of machine learning and by incorporating Ravishankar into Wenchel and Vijaykeerthy enables improved models with more accurate predictions that are validated [Ravishankar ¶28, ¶15]
Claims 5, 12 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Wenchel and Vijaykeerthy, as applied in the rejection of claim 1, 8 and 15 above, and further in view of Ribera et al. (US 20210073675 A1 hereinafter Ribera)
As to dependent claim 5, Wenchel and Vijaykeerthy teach the method of claim 1 above that is incorporated,
Wenchel and Vijaykeerthy do not specifically teach modifying the first set of data samples by multiplying the weighted value to each data sample of the first set of data samples; and retraining the machine learning model using the modified first set of data samples.
However, Ribera teaches modifying the first set of data samples by multiplying the weighted value to each data sample of the first set of data samples; and [modifies the model loss function with multiplying creating a weighted loss function ¶10 "Weighting the loss function custom-character(x, {circumflex over (x)}) may include multiplying the loss function custom-character(x, {circumflex over (x)}) by the weight function w(x) to compute the weighted loss function custom-character.sub.w(x, {circumflex over (x)}"]
retraining the machine learning model using the modified first set of data samples. [continuously trains (retrains) using weighted loss function ¶11-12 "train a continuous machine learning model in accordance with the training data set and the weighted loss function"]
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the modeling disclosed by Wenchel and Vijaykeerthy by incorporating the modifying the first set of data samples by multiplying the weighted value to each data sample of the first set of data samples; and retraining the machine learning model using the modified first set of data samples disclosed by Ribera because all techniques address the same field of machine learning and by incorporating Ribera into Wenchel and Vijaykeerthy improves the accuracy of models through parameter adjustments [Ribera ¶2-3]
As to dependent claim 12, Wenchel and Vijaykeerthy teach the method of claim 8 above that is incorporated,
Wenchel and Vijaykeerthy do not specifically teach modifying the first set of data samples by multiplying the weighted value to each data sample of the first set of data samples; and retraining the machine learning model using the modified first set of data samples.
However, Ribera teaches modifying the first set of data samples by multiplying the weighted value to each data sample of the first set of data samples; and [modifies the model loss function with multiplying creating a weighted loss function ¶10 "Weighting the loss function custom-character(x, {circumflex over (x)}) may include multiplying the loss function custom-character(x, {circumflex over (x)}) by the weight function w(x) to compute the weighted loss function custom-character.sub.w(x, {circumflex over (x)}"]
retraining the machine learning model using the modified first set of data samples. [continuously trains (retrains) using weighted loss function ¶11-12 "train a continuous machine learning model in accordance with the training data set and the weighted loss function"]
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the modeling disclosed by Wenchel and Vijaykeerthy by incorporating the modifying the first set of data samples by multiplying the weighted value to each data sample of the first set of data samples; and retraining the machine learning model using the modified first set of data samples disclosed by Ribera because all techniques address the same field of machine learning and by incorporating Ribera into Wenchel and Vijaykeerthy improves the accuracy of models through parameter adjustments [Ribera ¶2-3]
As to dependent claim 19, Wenchel and Vijaykeerthy teach the method of claim 15 above that is incorporated,
Wenchel and Vijaykeerthy do not specifically teach modifying the first set of data samples by multiplying the weighted value to each data sample of the first set of data samples; and retraining the machine learning model using the modified first set of data samples.
However, Ribera teaches modifying the first set of data samples by multiplying the weighted value to each data sample of the first set of data samples; and [modifies the model loss function with multiplying creating a weighted loss function ¶10 "Weighting the loss function custom-character(x, {circumflex over (x)}) may include multiplying the loss function custom-character(x, {circumflex over (x)}) by the weight function w(x) to compute the weighted loss function custom-character.sub.w(x, {circumflex over (x)}"]
retraining the machine learning model using the modified first set of data samples. [continuously trains (retrains) using weighted loss function ¶11-12 "train a continuous machine learning model in accordance with the training data set and the weighted loss function"]
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the modeling disclosed by Wenchel and Vijaykeerthy by incorporating the modifying the first set of data samples by multiplying the weighted value to each data sample of the first set of data samples; and retraining the machine learning model using the modified first set of data samples disclosed by Ribera because all techniques address the same field of machine learning and by incorporating Ribera into Wenchel and Vijaykeerthy improves the accuracy of models through parameter adjustments [Ribera ¶2-3]
Claims 6-7, 13-14 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Wenchel and Vijaykeerthy, as applied in the rejection of claim 1, 8 and 15 above, and further in view of COX et al. (US 20180012106 A1 hereinafter Cox)
As to dependent claim 6, Wenchel and Vijaykeerthy teach the method of claim 1 above that is incorporated,
Wenchel and Vijaykeerthy do not specifically teach modifying the first set of data samples by additively applying the weighted value to each data sample of the first set of data samples; and retraining the machine learning model using the modified first set of data samples.
However, Cox teaches modifying the first set of data samples by additively applying the weighted value to each data sample of the first set of data samples; and [modifies models with penalties of additional weightings ¶6 "For example, greater penalties may be introduced for misclassification, on the part of the machine-learning system, of examples that are easily classified by humans. Similarly, lesser penalties may be introduced for misclassification of examples that are relatively more difficult for humans to classify. By imposing such additional weightings in the objective function of the machine-learning algorithm"]
retraining the machine learning model using the modified first set of data samples. [penalized training data used for learning and adjusting model loss function (retraining) ¶67-68 "human-weighted loss and margins that are penalized for being inconsistent with those established by human annotators"]
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the modeling disclosed by Wenchel and Vijaykeerthy by incorporating the modifying the first set of data samples by additively applying the weighted value to each data sample of the first set of data samples; and retraining the machine learning model using the modified first set of data samples disclosed by Cox because all techniques address the same field of machine learning and by incorporating Cox into Wenchel and Vijaykeerthy enables models to perform better against overfitting by utilizing regularization [Cox ¶4-5]
As to dependent claim 7, Wenchel and Vijaykeerthy teach the method of claim 1 above that is incorporated,
Wenchel and Vijaykeerthy do not specifically teach wherein modifying the machine learning model is further based on a regularization parameter.
However, Cox teaches wherein modifying the machine learning model is further based on a regularization parameter. [regularization penalties (parameter) ¶4 "Machine-learning systems typically combat the effects of overfitting by a process called “regularization,” in which penalties are placed on solutions that are thought to be more likely to be the result of overfitting, typically because they are more complex or because they exhibit less stable behavior under injected noise."
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the modeling disclosed by Wenchel and Vijaykeerthy by incorporating the wherein modifying the machine learning model is further based on a regularization parameter disclosed by Cox because all techniques address the same field of machine learning and by incorporating Cox into Wenchel and Vijaykeerthy enables models to perform better against overfitting by utilizing regularization [Cox ¶4-5]
As to dependent claim 13, Wenchel and Vijaykeerthy teach the method of claim 8 above that is incorporated,
Wenchel and Vijaykeerthy do not specifically teach modifying the first set of data samples by additively applying the weighted value to each data sample of the first set of data samples; and retraining the machine learning model using the modified first set of data samples.
However, Cox teaches modifying the first set of data samples by additively applying the weighted value to each data sample of the first set of data samples; and [modifies models with penalties of additional weightings ¶6 "For example, greater penalties may be introduced for misclassification, on the part of the machine-learning system, of examples that are easily classified by humans. Similarly, lesser penalties may be introduced for misclassification of examples that are relatively more difficult for humans to classify. By imposing such additional weightings in the objective function of the machine-learning algorithm"]
retraining the machine learning model using the modified first set of data samples. [penalized training data used for learning and adjusting model loss function (retraining) ¶67-68 "human-weighted loss and margins that are penalized for being inconsistent with those established by human annotators"]
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the modeling disclosed by Wenchel and Vijaykeerthy by incorporating the modifying the first set of data samples by additively applying the weighted value to each data sample of the first set of data samples; and retraining the machine learning model using the modified first set of data samples disclosed by Cox because all techniques address the same field of machine learning and by incorporating Cox into Wenchel and Vijaykeerthy enables models to perform better against overfitting by utilizing regularization [Cox ¶4-5]
As to dependent claim 14, Wenchel and Vijaykeerthy teach the method of claim 8 above that is incorporated,
Wenchel and Vijaykeerthy do not specifically teach wherein modifying the machine learning model is further based on a regularization parameter.
However, Cox teaches wherein modifying the machine learning model is further based on a regularization parameter. [regularization penalties (parameter) ¶4 "Machine-learning systems typically combat the effects of overfitting by a process called “regularization,” in which penalties are placed on solutions that are thought to be more likely to be the result of overfitting, typically because they are more complex or because they exhibit less stable behavior under injected noise."
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the modeling disclosed by Wenchel and Vijaykeerthy by incorporating the wherein modifying the machine learning model is further based on a regularization parameter disclosed by Cox because all techniques address the same field of machine learning and by incorporating Cox into Wenchel and Vijaykeerthy enables models to perform better against overfitting by utilizing regularization [Cox ¶4-5]
As to dependent claim 20, Wenchel and Vijaykeerthy teach the method of claim 1 above that is incorporated,
Wenchel and Vijaykeerthy do not specifically teach wherein modifying the machine learning model is further based on a regularization parameter.
However, Cox teaches wherein modifying the machine learning model is further based on a regularization parameter. [regularization penalties (parameter) ¶4 "Machine-learning systems typically combat the effects of overfitting by a process called “regularization,” in which penalties are placed on solutions that are thought to be more likely to be the result of overfitting, typically because they are more complex or because they exhibit less stable behavior under injected noise."
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to modify the modeling disclosed by Wenchel and Vijaykeerthy by incorporating the wherein modifying the machine learning model is further based on a regularization parameter disclosed by Cox because all techniques address the same field of machine learning and by incorporating Cox into Wenchel and Vijaykeerthy enables models to perform better against overfitting by utilizing regularization [Cox ¶4-5]
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Applicant is required under 37 C.F.R. § 1.111(c) to consider these references fully when responding to this action.
Brown et al. (US 20230011777 A1 hereinafter Brown) teaches explainability data and scoring modules for better understanding (see ¶127)
It is noted that any citation to specific pages, columns, lines, or figures in the prior art references and any interpretation of the references should not be considered to be limiting in any way. A reference is relevant for all it contains and may be relied upon for all that it would have reasonably suggested to one having ordinary skill in the art. In re Heck, 699 F.2d 1331, 1332-33, 216 U.S.P.Q. 1038, 1039 (Fed. Cir. 1983) (quoting In re Lemelson, 397 F.2d 1006, 1009, 158 U.S.P.Q. 275, 277 (C.C.P.A. 1968)).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Beau Spratt whose telephone number is 571 272 9919. The examiner can normally be reached 8:30am to 5:00pm (PST).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Welch can be reached at 571 272 7212. The fax phone number for the organization where this application or proceeding is assigned is 571 483 7388.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866 217 9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800 786 9199 (IN USA OR CANADA) or 571 272 1000.
/BEAU D SPRATT/Primary Examiner, Art Unit 2143