DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Title
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed. Examiner believes that the title of the invention is imprecise. A descriptive title indicative of the invention will help in proper indexing, classifying, searching, etc. See MPEP 606.01. However, the title of the invention should be limited to 500 characters. Examiner suggests including the aspect(s) of the claims which Applicant believes to be novel or nonobvious over the prior art.
Claim Interpretation
For the sake of clarity of record, claim 1’s limitation of determine a size of a second training data set includes the interpretation of the number being zero. In this case, there would be nothing to display in the subsequent limitation.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-26 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim 1 is a system claim. Claim 16 is a system claim. Claim 24 is a method claim. Therefore, claims 1, 16, and 24 are directed to either a process, machine, manufacture or composition of matter.
With respect to Claim 1:
Step 2A Prong 1:
compute, based at least on re-training a machine learning model over a plurality of iterations using a regression data set, at least one validation score for one or more iterations of the plurality of iterations, the regression data set being sampled from the first training data set, the machine learning model having a validation performance below a target validation score after being trained using the first number of training samples (mental process – user can manually compute at least one validation score for one or more iterations of the plurality of iterations, the regression data set being sampled from the first training data set, the machine learning model having a validation performance below a target validation score after being trained using the first number of training samples)
determine a size of a second training data set comprising a second number of training samples for re-training the machine learning model based at least on the at least one validation score and the target validation score (mental process – user can manually determine a size of a second training data set comprising a second number of training samples for re-training the machine learning model based at least on the at least one validation score and the target validation score)
Step 2A Prong 2: This judicial exception is not integrated into a practical application. Additional elements:
One or more processors comprising: one or more processing units (mere instructions to apply the exception using a generic computer component)
receive a first training data set comprising a first number of training samples (Adding insignificant extra-solution activity to the judicial exception - see MPEP 2106.05(g))
compute, based at least on re-training a machine learning model over a plurality of iterations using a regression data set, at least one validation score for one or more iterations of the plurality of iterations, the regression data set being sampled from the first training data set, the machine learning model having a validation performance below a target validation score after being trained using the first number of training samples (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s Note: high level recitation of training a machine learning model with previously determined data)
cause a display to present a value indicating the size of the second training data set (Adding insignificant extra-solution activity to the judicial exception - see MPEP 2106.05(g))
Step 2B: The claim does not include additional elements considered individually and in combination that are sufficient to amount to significantly more than the judicial exception. Additional elements:
One or more processors comprising: one or more processing units (mere instructions to apply the exception using a generic computer component)
receive a first training data set comprising a first number of training samples (MPEP 2106.05(d)(II) indicate that merely “storing and retrieving information in memory” or “receiving or transmitting data over a network” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed step is well-understood, routine, conventional activity is supported under Berkheimer)
compute, based at least on re-training a machine learning model over a plurality of iterations using a regression data set, at least one validation score for one or more iterations of the plurality of iterations, the regression data set being sampled from the first training data set, the machine learning model having a validation performance below a target validation score after being trained using the first number of training samples (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s Note: high level recitation of training a machine learning model with previously determined data)
cause a display to present a value indicating the size of the second training data set (MPEP 2106.05(d)(II) indicate that merely “storing and retrieving information in memory” or “receiving or transmitting data over a network” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed step is well-understood, routine, conventional activity is supported under Berkheimer)
Conclusion: The claim is not patent eligible.
Regarding Claim 2: The limitation(s), as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, other than the additional elements, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) encompasses the user manually wherein the one or more processing units compute the at least one validation score by deriving at least one validation score estimation function.
The limitation(s) includes the additional elements of wherein the one or more processing units compute the at least one validation score by deriving at least one validation score estimation function.
These judicial exceptions are not integrated into a practical application. The additional element(s) of wherein the one or more processing units compute the at least one validation score by are recited at a high-level of generality such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element(s) of wherein the one or more processing units compute the at least one validation score by amount to no more than mere instructions to apply the exception using a generic computer component or operation. Mere instructions to apply an exception using a generic computer component or operation cannot provide an inventive concept. Accordingly, the claims are not patent eligible.
Regarding Claim 3: The limitation(s), as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, other than the additional elements, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) encompasses the user manually wherein the one or more processing units compute the at least one validation score by deriving at least one validation score estimation function, wherein one or more parameters of the at least one validation score estimation function are determined by curve fitting the at least one validation score corresponding to one or more iterations of the plurality of iterations.
The limitation(s) includes the additional elements of wherein the one or more processing units compute the at least one validation score by deriving at least one validation score estimation function, wherein one or more parameters of the at least one validation score estimation function are determined by curve fitting the at least one validation score corresponding to one or more iterations of the plurality of iterations.
These judicial exceptions are not integrated into a practical application. The additional element(s) of wherein the one or more processing units compute the at least one validation score by are recited at a high-level of generality such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element(s) of wherein the one or more processing units compute the at least one validation score by amount to no more than mere instructions to apply the exception using a generic computer component or operation. Mere instructions to apply an exception using a generic computer component or operation cannot provide an inventive concept. Accordingly, the claims are not patent eligible.
Regarding Claim 4: The limitation(s), as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, other than the additional elements, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) encompasses the user manually wherein the one or more processing units compute the at least one validation score by deriving at least one validation score estimation function, further wherein the at least one validation score estimation function is solved by, at least in part, minimizing the size of the training data set subject to the at least one validation score estimation function having a value greater than the target validation score.
The limitation(s) includes the additional elements of wherein the one or more processing units compute the at least one validation score by deriving at least one validation score estimation function, further wherein the at least one validation score estimation function is solved by, at least in part, minimizing the size of the training data set subject to the at least one validation score estimation function having a value greater than the target validation score.
These judicial exceptions are not integrated into a practical application. The additional element(s) of wherein the one or more processing units compute the at least one validation score by are recited at a high-level of generality such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element(s) of wherein the one or more processing units compute the at least one validation score by amount to no more than mere instructions to apply the exception using a generic computer component or operation. Mere instructions to apply an exception using a generic computer component or operation cannot provide an inventive concept. Accordingly, the claims are not patent eligible.
Regarding Claim 5: The limitation(s), as drafted, are a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) encompasses the user manually wherein the regression data set comprises a plurality of subsets of training data generated from the first training data set and the at least one validation score is associated with a respective subset of training data of the plurality of subsets of training data.
These judicial exceptions are not integrated into a practical application. In particular, the claims do not recite any additional elements. Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, no additional elements are cited. Accordingly, the claim is not patent eligible.
Regarding Claim 6: The limitation(s), as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, other than the additional elements, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) encompasses the user manually wherein the one or more processing units are further to:
determine a correction factor; and
determine the size of the second training data set based at least on a sum of the target validation score and the correction factor.
The limitation(s) includes the additional elements of wherein the one or more processing units are further to.
These judicial exceptions are not integrated into a practical application. The additional element(s) of wherein the one or more processing units are further to are recited at a high-level of generality such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element(s) of wherein the one or more processing units are further to amount to no more than mere instructions to apply the exception using a generic computer component or operation. Mere instructions to apply an exception using a generic computer component or operation cannot provide an inventive concept. Accordingly, the claims are not patent eligible.
Regarding Claim 7: The limitation(s), as drafted, are a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) encompasses the user manually wherein the correction factor is computed from a second training data set used to train a second machine learning model.
These judicial exceptions are not integrated into a practical application. In particular, the claims do not recite any additional elements. Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, no additional elements are cited. Accordingly, the claim is not patent eligible.
Regarding Claim 8: The limitation(s), as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, other than the additional elements, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) encompasses the user manually wherein the one or more processing units are further to:
determine a linear score function of ground truth based at least on training the machine learning model using the first training data set.
The limitation(s) includes the additional elements of wherein the one or more processing units are further to:
determine a linear score function of ground truth based at least on training the machine learning model using the first training data set.
These judicial exceptions are not integrated into a practical application. The additional element(s) of wherein the one or more processing units are further to are recited at a high-level of generality such that it amounts no more than mere instructions to apply the exception using a generic computer component. The additional element(s) of based at least on training the machine learning model using the first training data set recite merely adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f). Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element(s) of wherein the one or more processing units are further to amount to no more than mere instructions to apply the exception using a generic computer component or operation. Mere instructions to apply an exception using a generic computer component or operation cannot provide an inventive concept. The additional element(s) of based at least on training the machine learning model using the first training data set recite adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f). Accordingly, the claims are not patent eligible.
Regarding Claim 9: The limitation(s), as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, other than the additional elements, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) encompasses the user manually compute a data collection ratio based at least in part on the linear score function of ground truth, the second number of training samples, and the first number of training samples; and
compute a correction factor, the correction factor computed to generate a ratio greater than one for the data collection ratio.
The limitation(s) includes the additional elements of wherein the one or more processing units are further to.
These judicial exceptions are not integrated into a practical application. The additional element(s) of wherein the one or more processing units are further to are recited at a high-level of generality such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element(s) of wherein the one or more processing units are further to amount to no more than mere instructions to apply the exception using a generic computer component or operation. Mere instructions to apply an exception using a generic computer component or operation cannot provide an inventive concept. Accordingly, the claims are not patent eligible.
Regarding Claim 10: The limitation(s), as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, other than the additional elements, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) encompasses the user manually deriving at least one validation score estimation function, the at least one validation score estimation function comprising a concave monotonic increasing regression function.
The limitation(s) includes the additional elements of wherein the one or more processing units compute the at least one validation score by.
These judicial exceptions are not integrated into a practical application. The additional element(s) of wherein the one or more processing units compute the at least one validation score by are recited at a high-level of generality such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element(s) of wherein the one or more processing units compute the at least one validation score by amount to no more than mere instructions to apply the exception using a generic computer component or operation. Mere instructions to apply an exception using a generic computer component or operation cannot provide an inventive concept. Accordingly, the claims are not patent eligible.
Regarding Claim 11: The limitation(s), as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, other than the additional elements, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) encompasses the user manually determine an error in the size of the second training data set based at least on the linear score function of ground truth.
The limitation(s) includes the additional elements of wherein the one or more processing units are further to:
determine an error in the second number of training samples based at least on the linear score function of ground truth; and
cause the display of the error in the size of the second training data set.
These judicial exceptions are not integrated into a practical application. The additional element(s) of wherein the one or more processing units are further to are recited at a high-level of generality such that it amounts no more than mere instructions to apply the exception using a generic computer component. The additional element(s) of cause the display of the error in the size of the second training data set recite adding insignificant extra-solution activity to the judicial exception - see MPEP 2106.05(g). Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element(s) of wherein the one or more processing units are further to amount to no more than mere instructions to apply the exception using a generic computer component or operation. Mere instructions to apply an exception using a generic computer component or operation cannot provide an inventive concept. The additional element(s) of cause the display of the error in the size of the second training data set recite merely “storing and retrieving information in memory” or “receiving or transmitting data over a network” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim) (MPEP 2106.05(d)(II)). Thereby, a conclusion that the claimed storing step is well-understood, routine, conventional activity is supported under Berkheimer. Accordingly, the claims are not patent eligible.
Regarding Claim 12: The limitation(s), as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, other than the additional elements, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) encompasses the user manually determine a linear score function of ground truth based at least on training the machine learning model using the first training data set;
compute a data collection ratio based at least on part on the linear score function of ground truth, the second number of training samples, and the first number of training samples.
The limitation(s) includes the additional elements of wherein the one or more processing units are further to:
determine a linear score function of ground truth based at least on training the machine learning model using the first training data set
compute a data collection ratio based at least on part on the linear score function of ground truth, the second number of training samples, and the first number of training samples; and
cause the display to indicate at least one of an optimism indication or a pessimism indication for the at least one validation score estimation function based at least on the data collection ratio.
These judicial exceptions are not integrated into a practical application. The additional element(s) of wherein the one or more processing units are further to are recited at a high-level of generality such that it amounts no more than mere instructions to apply the exception using a generic computer component. The additional element(s) of based at least on training the machine learning model using the first training data set recite merely adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f). The additional element(s) of cause the display to indicate at least one of an optimism indication or a pessimism indication for the at least one validation score estimation function based at least on the data collection ratio recite adding insignificant extra-solution activity to the judicial exception - see MPEP 2106.05(g). Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element(s) of wherein the one or more processing units are further to amount to no more than mere instructions to apply the exception using a generic computer component or operation. Mere instructions to apply an exception using a generic computer component or operation cannot provide an inventive concept. The additional element(s) of based at least on training the machine learning model using the first training data set recite adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f). The additional element(s) of cause the display to indicate at least one of an optimism indication or a pessimism indication for the at least one validation score estimation function based at least on the data collection ratio recite merely “storing and retrieving information in memory” or “receiving or transmitting data over a network” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim) (MPEP 2106.05(d)(II)). Thereby, a conclusion that the claimed storing step is well-understood, routine, conventional activity is supported under Berkheimer. Accordingly, the claims are not patent eligible.
Regarding Claim 13: The limitation(s), as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, other than the additional elements, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) encompasses the user manually wherein the one or more processing units compute the at least one validation score by deriving a plurality of validation score estimation functions, and the one or more processing units are further to:
solve each of the plurality of validation score estimation functions to determine, based at least on the target validation score, a respective size of the second training data set comprising a respective second number of training samples.
The limitation(s) includes the additional elements of wherein the one or more processing units compute the at least one validation score by deriving a plurality of validation score estimation functions, and the one or more processing units are further to:
solve each of the plurality of validation score estimation functions to determine, based at least on the target validation score, a respective size of the second training data set comprising a respective second number of training samples; and
cause the display to present each of the respective size of the second training data set comprising a respective second number of training samples.
These judicial exceptions are not integrated into a practical application. The additional element(s) of wherein the one or more processing units compute the at least one validation score by deriving a plurality of validation score estimation functions, and the one or more processing units are further to are recited at a high-level of generality such that it amounts no more than mere instructions to apply the exception using a generic computer component. The additional element(s) of cause the display to present each of the respective size of the second training data set comprising a respective second number of training samples recite adding insignificant extra-solution activity to the judicial exception - see MPEP 2106.05(g). Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element(s) of wherein the one or more processing units compute the at least one validation score by deriving a plurality of validation score estimation functions, and the one or more processing units are further to amount to no more than mere instructions to apply the exception using a generic computer component or operation. Mere instructions to apply an exception using a generic computer component or operation cannot provide an inventive concept. The additional element(s) of cause the display to present each of the respective size of the second training data set comprising a respective second number of training samples recite merely “storing and retrieving information in memory” or “receiving or transmitting data over a network” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim) (MPEP 2106.05(d)(II)). Thereby, a conclusion that the claimed storing step is well-understood, routine, conventional activity is supported under Berkheimer. Accordingly, the claims are not patent eligible.
Regarding Claim 14: The limitation(s), as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, other than the additional elements, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) includes the additional elements of wherein the one or more processing units are further to receive an input indicating the target validation score for training the machine learning model.
These judicial exceptions are not integrated into a practical application. The additional element(s) of wherein the one or more processing units are further to are recited at a high-level of generality such that it amounts no more than mere instructions to apply the exception using a generic computer component. The additional element(s) of receive an input indicating the target validation score for training the machine learning model recite adding insignificant extra-solution activity to the judicial exception - see MPEP 2106.05(g). Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element(s) of wherein the one or more processing units are further to amount to no more than mere instructions to apply the exception using a generic computer component or operation. Mere instructions to apply an exception using a generic computer component or operation cannot provide an inventive concept. The additional element(s) of receive an input indicating the target validation score for training the machine learning model recite merely “storing and retrieving information in memory” or “receiving or transmitting data over a network” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim) (MPEP 2106.05(d)(II)). Thereby, a conclusion that the claimed storing step is well-understood, routine, conventional activity is supported under Berkheimer. Accordingly, the claims are not patent eligible.
Regarding Claim 15: The limitation(s), as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, other than the additional elements, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) includes the additional elements of wherein the one or more processors are comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational Al operations; a system for generating synthetic data; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources.
These judicial exceptions are not integrated into a practical application. The additional element(s) of wherein the one or more processors are comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational Al operations; a system for generating synthetic data; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources are recited at a high-level of generality such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element(s) of wherein the one or more processors are comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational Al operations; a system for generating synthetic data; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources amount to no more than mere instructions to apply the exception using a generic computer component or operation. Mere instructions to apply an exception using a generic computer component or operation cannot provide an inventive concept. Accordingly, the claims are not patent eligible.
With respect to Claim 16:
Step 2A Prong 1:
generate a regression data set using the first training data set, the regression data set comprising a plurality of subsets of training data generated using the first training data set (mental process – user can manually generate a regression data set using the first training data set, the regression data set comprising a plurality of subsets of training data generated using the first training data set)
compute a plurality of validation scores using at least a first validation score estimation function, the plurality of validation scores comprising a respective validation score for at least one iteration of the plurality of iterations (mental process – user can manually compute a plurality of validation scores using at least a first validation score estimation function, the plurality of validation scores comprising a respective validation score for at least one iteration of the plurality of iterations)
determine, based at least on a target validation score and using the at least a first validation score estimation function, a size of a second training data set comprising an additional number of training samples for re-training the machine learning model (mental process – user can manually determine, based at least on a target validation score and using the at least a first validation score estimation function, a size of a second training data set comprising an additional number of training samples for re-training the machine learning model)
perform one or more operations to indicate the determination of the size of the second training data set (mental process – user can manually perform one or more operations to indicate the determination of the size of the second training data set)
Step 2A Prong 2: This judicial exception is not integrated into a practical application. Additional elements:
one or more processing units to (mere instructions to apply the exception using a generic computer component)
access a data store comprising a first training data set including a number of training samples (Adding insignificant extra-solution activity to the judicial exception - see MPEP 2106.05(g))
re-train the machine learning model over a plurality of iterations using the regression data set , the machine learning model having a validation performance below a target validation score after being trained using the number of training samples (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s Note: high level recitation of training a machine learning model with previously determined data)
Step 2B: The claim does not include additional elements considered individually and in combination that are sufficient to amount to significantly more than the judicial exception. Additional elements:
one or more processing units to (mere instructions to apply the exception using a generic computer component)
access a data store comprising a first training data set including a number of training samples (MPEP 2106.05(d)(II) indicate that merely “storing and retrieving information in memory” or “receiving or transmitting data over a network” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed step is well-understood, routine, conventional activity is supported under Berkheimer)
re-train the machine learning model over a plurality of iterations using the regression data set , the machine learning model having a validation performance below a target validation score after being trained using the number of training samples (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s Note: high level recitation of training a machine learning model with previously determined data)
Conclusion: The claim is not patent eligible.
Regarding Claim 17: The limitation(s), as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, other than the additional elements, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) includes the additional elements of wherein the one or more processing units are further to cause display of a training data collection recommendation based at least on the size of the second training data set.
These judicial exceptions are not integrated into a practical application. The additional element(s) of wherein the one or more processing units are further to are recited at a high-level of generality such that it amounts no more than mere instructions to apply the exception using a generic computer component. The additional element(s) of cause display of a training data collection recommendation based at least on the size of the second training data set recite adding insignificant extra-solution activity to the judicial exception - see MPEP 2106.05(g). Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element(s) of wherein the one or more processing units are further to amount to no more than mere instructions to apply the exception using a generic computer component or operation. Mere instructions to apply an exception using a generic computer component or operation cannot provide an inventive concept. The additional element(s) of cause display of a training data collection recommendation based at least on the size of the second training data set recite merely “storing and retrieving information in memory” or “receiving or transmitting data over a network” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim) (MPEP 2106.05(d)(II)). Thereby, a conclusion that the claimed storing step is well-understood, routine, conventional activity is supported under Berkheimer. Accordingly, the claims are not patent eligible.
Regarding Claim 18: The limitation(s), as drafted, are a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) encompasses the user manually wherein the at least one validation score estimation function comprises a concave monotonic increasing regression function.
These judicial exceptions are not integrated into a practical application. In particular, the claims do not recite any additional elements. Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, no additional elements are cited. Accordingly, the claim is not patent eligible.
Regarding Claim 19: The limitation(s), as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, other than the additional elements, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) encompasses the user manually curve fit the plurality of validation scores to compute one or more parameters of the at least one validation score estimation function.
The limitation(s) includes the additional elements of wherein the one or more processing units are further to.
These judicial exceptions are not integrated into a practical application. The additional element(s) of wherein the one or more processing units are further to are recited at a high-level of generality such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element(s) of wherein the one or more processing units are further to amount to no more than mere instructions to apply the exception using a generic computer component or operation. Mere instructions to apply an exception using a generic computer component or operation cannot provide an inventive concept. Accordingly, the claims are not patent eligible.
Regarding Claim 20: The limitation(s), as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, other than the additional elements, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) encompasses the user manually solve, based at least on a sum of the target validation score and a correction factor, the at least one validation score estimation function to determine a second size of the second training data set comprising a second additional number of training samples.
The limitation(s) includes the additional elements of wherein the one or more processing units are further to.
These judicial exceptions are not integrated into a practical application. The additional element(s) of wherein the one or more processing units are further to are recited at a high-level of generality such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element(s) of wherein the one or more processing units are further to amount to no more than mere instructions to apply the exception using a generic computer component or operation. Mere instructions to apply an exception using a generic computer component or operation cannot provide an inventive concept. Accordingly, the claims are not patent eligible.
Regarding Claim 21: The limitation(s), as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, other than the additional elements, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) encompasses the user manually compute a data collection ratio for the at least one validation score estimation function based at least on a linear score function of ground truth computed at least in part by training the machine learning model using the first training data set, the additional number of training samples, and the second additional number of training samples; and
wherein the correction factor is computed to generate a ratio greater than one for the data collection ratio.
The limitation(s) includes the additional elements of wherein the one or more processing units are further to:
compute a data collection ratio for the at least one validation score estimation function based at least on a linear score function of ground truth computed at least in part by training the machine learning model using the first training data set, the additional number of training samples, and the number of training samples.
These judicial exceptions are not integrated into a practical application. The additional element(s) of wherein the one or more processing units are further to are recited at a high-level of generality such that it amounts no more than mere instructions to apply the exception using a generic computer component. The additional element(s) of training the machine learning model using the first training data set recite merely adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f). Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element(s) of wherein the one or more processing units are further to amount to no more than mere instructions to apply the exception using a generic computer component or operation. Mere instructions to apply an exception using a generic computer component or operation cannot provide an inventive concept. The additional element(s) of training the machine learning model using the first training data set recite adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f). Accordingly, the claims are not patent eligible.
Regarding Claim 22: The limitation(s), as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, other than the additional elements, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) encompasses the user manually derive another validation score estimation function based at least on iteratively re- training the machine learning model using the regression data set to compute a second respective plurality of validation scores, the second respective plurality of validation scores comprising a respective validation score for each iteration of a plurality of iterations, wherein the another validation score estimation function comprises a concave monotonic increasing regression function different from the at least one validation score estimation function;
solve, based at least on the target validation score, the additional validation score estimation function to determine a second size of the second training data set comprising a second additional number of training samples.
The limitation(s) includes the additional elements of wherein the one or more processing units are further to:
derive another validation score estimation function based at least on iteratively re- training the machine learning model using the regression data set to compute a second respective plurality of validation scores, the second respective plurality of validation scores comprising a respective validation score for each iteration of a plurality of iterations, wherein the another validation score estimation function comprises a concave monotonic increasing regression function different from the at least one validation score estimation function;
solve, based at least on the target validation score, the additional validation score estimation function to determine another additional number of training samples; and
cause a display to present the size of the second training data set and the second size of the second training data set.
These judicial exceptions are not integrated into a practical application. The additional element(s) of wherein the one or more processing units are further to are recited at a high-level of generality such that it amounts no more than mere instructions to apply the exception using a generic computer component. The additional element(s) of based at least on iteratively re- training the machine learning model using the regression data set to compute a second respective plurality of validation scores recite merely adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f). The additional element(s) of cause a display to present the size of the second training data set and the second size of the second training data set recite adding insignificant extra-solution activity to the judicial exception - see MPEP 2106.05(g). Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element(s) of wherein the one or more processing units are further to amount to no more than mere instructions to apply the exception using a generic computer component or operation. Mere instructions to apply an exception using a generic computer component or operation cannot provide an inventive concept. The additional element(s) of based at least on iteratively re- training the machine learning model using the regression data set to compute a second respective plurality of validation scores recite adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f). The additional element(s) of cause a display to present the size of the second training data set and the second size of the second training data set recite merely “storing and retrieving information in memory” or “receiving or transmitting data over a network” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim) (MPEP 2106.05(d)(II)). Thereby, a conclusion that the claimed storing step is well-understood, routine, conventional activity is supported under Berkheimer. Accordingly, the claims are not patent eligible.
Regarding Claim 23: The limitation(s), as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, other than the additional elements, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) includes the additional elements of wherein the system is comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational Al operations; a system for generating synthetic data; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources.
These judicial exceptions are not integrated into a practical application. The additional element(s) of wherein the system is comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational Al operations; a system for generating synthetic data; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources are recited at a high-level of generality such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element(s) of wherein the system is comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational Al operations; a system for generating synthetic data; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources amount to no more than mere instructions to apply the exception using a generic computer component or operation. Mere instructions to apply an exception using a generic computer component or operation cannot provide an inventive concept. Accordingly, the claims are not patent eligible.
With respect to Claim 24:
Step 2A Prong 1:
determining a size of a second training data set comprising a number of additional training data samples for re-training a machine learning model by, at least in part, solving at least one validation score estimation function based at least one a target validation score, wherein one or more parameters of the validation score estimation function is determined based at least on iteratively re-training the machine learning model using a regression data set sampled from a first training data set, the machine learning model having a validation performance below a target validation score after being trained using the first training data set; (mental process – user can manually determine a number of additional training data samples for re-training a machine learning model by, at least in part, solving at least one validation score estimation function wherein one or more parameters of the validation score estimation function is determined)
performing one or more operations to indicate the determination of the size of the second training data set (mental process – user can manually perform one or more operations to indicate the determination of the size of the second training data set)
Step 2A Prong 2: This judicial exception is not integrated into a practical application. Additional elements:
determining a number of additional training data samples for re-training a machine learning model by, at least in part, solving at least one validation score estimation function wherein one or more parameters of the validation score estimation function is determined based at least on iteratively re-training the machine learning model using a regression data set sampled from a first training data set, the machine learning model having a validation performance below a target validation score after being trained using the first training data set (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s Note: high level recitation of training a machine learning model with previously determined data)
Step 2B: The claim does not include additional elements considered individually and in combination that are sufficient to amount to significantly more than the judicial exception. Additional elements:
determining a number of additional training data samples for re-training a machine learning model by, at least in part, solving at least one validation score estimation function wherein one or more parameters of the validation score estimation function is determined based at least on iteratively re-training the machine learning model using a regression data set sampled from a first training data set, the machine learning model having a validation performance below a target validation score after being trained using the first training data set (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s Note: high level recitation of training a machine learning model with previously determined data)
Conclusion: The claim is not patent eligible.
Regarding Claim 25: The limitation(s), as drafted, are a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) encompasses the user manually generating the regression data set comprising a plurality of subsets of data generated from a training data set, wherein one or more parameters of the at least one validation score estimation function are computed during the iteratively re-training the machine learning model using the regression data set.
These judicial exceptions are not integrated into a practical application. In particular, the claims do not recite any additional elements. Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, no additional elements are cited. Accordingly, the claim is not patent eligible.
Regarding Claim 26: The limitation(s), as drafted, are a process that, under its broadest reasonable interpretation, covers performance of the limitation(s) in the mind. That is, nothing in the claim limitation(s) precludes the step from practically being performed in the mind.
The limitation(s) encompasses the user manually solving the at least one validation score estimation function to determine the size of the second training data set to meet or exceed a target validation score.
These judicial exceptions are not integrated into a practical application. In particular, the claims do not recite any additional elements. Accordingly, this does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, no additional elements are cited. Accordingly, the claim is not patent eligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 5, 13-17, 23-26 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bangera et al. (hereinafter Bangera) A Progressive Sampling based Approach to Reduce Sampling Time, in view of Wang et al. (hereinafter Wang), U.S. Patent Application Publication 2023/0214582.
Regarding Claim 1, Bangera discloses one or more processors comprising:
one or more processing units to:
receive a first training data set comprising a first number of training samples [“the full dataset” §IV.B ¶1; “the concept of training data and test data is used wherein we train the data to learn the characteristics of the output and test the data with the learned data” §II.B ¶1];
compute, based at least on re-training a machine learning model over a plurality of iterations using a regression data set, at least one validation score for one or more iterations of the plurality of iterations, the regression data set being sampled from the first training data set [“Progressive Sampling (PS) starts with a small data sample from the full dataset and use progressively larger samples until the model accuracy cannot increase substantially.” §IV.B ¶1; Fig. 1; Algorithm 2], the machine learning model having a validation performance below a target validation score after being trained using the first number of training samples [“with desirable error rate” §II.B ¶1; Fig. 1; Algorithm 2];
determine a size of a second training data set comprising a second number of training samples for re-training the machine learning model based at least on the at least one validation score and the target validation score [“Progressive Sampling (PS) starts with a small data sample from the full dataset and use progressively larger samples until the model accuracy cannot increase substantially.” §VI.B ¶1; Fig. 1; Algorithm 2]; and
cause a display to present a value indicating the size of the second training data set [Fig. 1].
However, Bangera fails to explicitly disclose one or more processors comprising one or more processing units
using a regression data set
cause a display to present.
Wang discloses one or more processors comprising one or more processing units [“one or more processors” §70; Fig. 1]
using a regression data set [“regression analysis” ¶38]
cause a display to present [“a data presentation routine 138C for generating or presenting received data to the user via the display” ¶28].
It would have been obvious to one having ordinary skill in the art, having the teachings of Bangera and Wang before him before the effective filing date of the claimed invention, to modify the progressive sampling for training a model of Bangera to incorporate the hardware and regressive data set of Wang.
Given the advantage of implementing the progressive sampling on hardware for faster computations and display as well as using known regression type data, one having ordinary skill in the art would have been motivated to make this obvious modification.
Regarding Claim 5, Bangera and Wang disclose the one or more processors of claim 1. Bangera further discloses wherein the regression data set comprises a plurality of subsets of training data generated from the first training data set [“Progressive Sampling (PS) starts with a small data sample from the full dataset and use progressively larger samples until the model accuracy cannot increase substantially.” §IV.B ¶1; Fig. 1; Algorithm 2] and the at least one validation score is associated with a respective subset of training data of the plurality of subsets of training data [“Progressive Sampling (PS) starts with a small data sample from the full dataset and use progressively larger samples until the model accuracy cannot increase substantially.” §IV.B ¶1; Fig. 1; Algorithm 2].
Regarding Claim 13, Bangera and Wang disclose the one or more processors of claim 1. Bangera further discloses wherein the one or more processing units compute the at least one validation score by deriving a plurality of validation score estimation functions [“Progressive Sampling (PS) starts with a small data sample from the full dataset and use progressively larger samples until the model accuracy cannot increase substantially.” §IV.B ¶1; Fig. 1; Algorithm 2], and the one or more processing units are further to:
solve each of the plurality of validation score estimation functions [“Progressive Sampling (PS) starts with a small data sample from the full dataset and use progressively larger samples until the model accuracy cannot increase substantially.” §IV.B ¶1; Fig. 1; Algorithm 2] to determine, based at least on the target validation score, a respective size of the second training data set comprising a respective second number of training samples [“Progressive Sampling (PS) starts with a small data sample from the full dataset and use progressively larger samples until the model accuracy cannot increase substantially.” §IV.B ¶1; Fig. 1; “new sample n` ← n + schedule” Algorithm 2]; and
cause the display to present each of the respective sizes of the second training data set comprising the respective second number of training samples [Fig. 1; Algorithm 2].
However, Bangera fails to explicitly disclose cause the display to present.
Wang discloses cause the display to present [“a data presentation routine 138C for generating or presenting received data to the user via the display” ¶28].
It would have been obvious to one having ordinary skill in the art, having the teachings of Bangera and Wang before him before the effective filing date of the claimed invention, to modify the combination to incorporate the displaying of Wang.
Given the advantage of providing useful information to the user, one having ordinary skill in the art would have been motivated to make this obvious modification.
Regarding Claim 14, Bangera and Wang disclose the one or more processors of claim 1.
However, Bangera fails to explicitly disclose wherein the one or more processing units are further to receive an input indicating the target validation score for training the machine learning model.
Wang discloses wherein the one or more processing units are further to receive an input indicating the target validation score for training the machine learning model [“the client computing device 110 may include one or more inputs 114 to receive instructions, selections, or other information from a user of the client computing device 110.” ¶18; “The user feedback may include behavior such as at least one of: cursor movement, clicking, highlighting and/or copying text, scrolling behavior, zooming behavior, or other behavior of the user on the electronic listing. The behavior may be used to determine if the scoring algorithm is accurate in its scoring.” ¶39].
It would have been obvious to one having ordinary skill in the art, having the teachings of Bangera and Wang before him before the effective filing date of the claimed invention, to modify the combination to incorporate user input for the validation score of Wang.
Given the advantage of selecting a desired accuracy score for a model, one having ordinary skill in the art would have been motivated to make this obvious modification.
Regarding Claim 15, Bangera and Wang disclose the one or more processors of claim 1. Bangera further discloses wherein the one or more processors are comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations [“used for application reliability of communication network in the simulation environment” §III ¶1]; a system for performing digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational Al operations; a system for generating synthetic data; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources [“Internet & mobile computing like web, social media networking, blogs, forums and sensors” §I ¶1]
Regarding Claim 16, Bangera discloses a system comprising:
one or more processing units to:
access a data store comprising a first training data set including a number of training samples [“the full dataset” §IV.B ¶1; “the concept of training data and test data is used wherein we train the data to learn the characteristics of the output and test the data with the learned data” §II.B ¶1];
generate a regression data set using the first training data set, the regression data set comprising a plurality of subsets of training data generated using the first training data set [“Progressive Sampling (PS) starts with a small data sample from the full dataset and use progressively larger samples until the model accuracy cannot increase substantially.” §IV.B ¶1; Fig. 1; Algorithm 2];
re-train a machine learning model over a plurality of iterations using the regression data set, the machine learning model having a validation performance below a target validation score after being trained using the number of training samples [“Progressive Sampling (PS) starts with a small data sample from the full dataset and use progressively larger samples until the model accuracy cannot increase substantially.” §IV.B ¶1; Fig. 1; Algorithm 2];
compute a plurality of validation scores using at least a first validation score estimation function, the plurality of validation scores comprising a respective validation score for at least one iteration of the plurality of iterations [“Progressive Sampling (PS) starts with a small data sample from the full dataset and use progressively larger samples until the model accuracy cannot increase substantially.” §IV.B ¶1; Fig. 1; Algorithm 2];
determine, based at least on a target validation score and using the at least a first validation score estimation function, a size of a second training data set comprising an additional number of training samples for re-training the machine learning model [“Progressive Sampling (PS) starts with a small data sample from the full dataset and use progressively larger samples until the model accuracy cannot increase substantially.” §IV.B ¶1; Fig. 1; “new sample n` ← n + schedule” Algorithm 2]; and
perform one or more operations to indicate the determination of the size of the second training data set [“Progressive Sampling (PS) starts with a small data sample from the full dataset and use progressively larger samples until the model accuracy cannot increase substantially.” §IV.B ¶1; Fig. 1; Algorithm 2].
However, Bangera fails to explicitly disclose one or more processing units, and
regression data.
Wang discloses one or more processing units [“one or more processors” §70; Fig. 1], and
regression data [“regression analysis” ¶38].
It would have been obvious to one having ordinary skill in the art, having the teachings of Bangera and Wang before him before the effective filing date of the claimed invention, to modify the progressive sampling for training a model of Bangera to incorporate the hardware and regressive data set of Wang.
Given the advantage of implementing the progressive sampling on hardware for faster computations as well as using known regression type data, one having ordinary skill in the art would have been motivated to make this obvious modification.
Regarding Claim 17, Bangera and Wang disclose the system of claim 16. Bangera further discloses wherein the one or more processing units are further to cause display of a training data collection recommendation based at least on the additional number of training samples [Fig. 1; Algorithm 2].
However, Wang fails to explicitly disclose wherein the one or more processing units are further to cause display of a training data collection recommendation based at least on the additional number of training samples.
Wang discloses wherein the one or more processing units are further to cause display [“a data presentation routine 138C for generating or presenting received data to the user via the display” ¶28] of a training data collection recommendation based at least on the additional number of training samples.
It would have been obvious to one having ordinary skill in the art, having the teachings of Wang and Figueroa before him before the effective filing date of the claimed invention, to modify the combination to incorporate displaying information of Wang.
Given the advantage of providing useful information to the user, one having ordinary skill in the art would have been motivated to make this obvious modification.
Claim 23 is rejected on the same grounds as claim 15.
Regarding claim 24, Bangera discloses a method comprising:
determining a size of a second training data set comprising a number of additional training data samples for re-training a machine learning model by, at least in part, solving at least one validation score estimation function based at least on a target validation score, wherein one or more parameters of the at least one validation score estimation function is determined based at least on iteratively re-training the machine learning model using a regression data set sampled from a first training data set, the machine learning model having a validation performance below a target validation score after being trained using the first training data set [“Progressive Sampling (PS) starts with a small data sample from the full dataset and use progressively larger samples until the model accuracy cannot increase substantially.” §IV.B ¶1; Fig. 1; Algorithm 2]; and
performing one or more operations to indicate the determination of the size of the second training data set [“Progressive Sampling (PS) starts with a small data sample from the full dataset and use progressively larger samples until the model accuracy cannot increase substantially.” §IV.B ¶1; Fig. 1; Algorithm 2].
However, Bangera fails to explicitly disclose regression data.
Wang discloses regression data [“regression analysis” ¶38].
It would have been obvious to one having ordinary skill in the art, having the teachings of Bangera and Wang before him before the effective filing date of the claimed invention, to modify the progressive sampling for training a model of Bangera to incorporate the regressive data set of Wang.
Given the advantage of using known regression type data to solve a regression problem, one having ordinary skill in the art would have been motivated to make this obvious modification.
Regarding Claim 25, Bangera and Wang disclose the method of claim 24. Bangera further discloses further comprising:
generating the regression data set comprising a plurality of subsets of data generated from a training data set, wherein one or more parameters of the at least one validation score estimation function are computed during the iteratively re-training the machine learning model using the regression data set [“Progressive Sampling (PS) starts with a small data sample from the full dataset and use progressively larger samples until the model accuracy cannot increase substantially.” §IV.B ¶1; Fig. 1; Algorithm 2].
However, Bangera fails to explicitly disclose regression data.
Wang discloses regression data [“regression analysis” ¶38].
It would have been obvious to one having ordinary skill in the art, having the teachings of Bangera and Wang before him before the effective filing date of the claimed invention, to modify the combination to incorporate the regressive data set of Wang.
Given the advantage of using known regression type data to solve a regression problem, one having ordinary skill in the art would have been motivated to make this obvious modification.
Regarding Claim 26, Bangera and Wang disclose the method of claim 25. Bangera further discloses further comprising:
solving the at least one validation score estimation function to determine the size of the second training data set to meet or exceed a target validation score [“Progressive Sampling (PS) starts with a small data sample from the full dataset and use progressively larger samples until the model accuracy cannot increase substantially.” §IV.B ¶1; Fig. 1; Algorithm 2].
Claim(s) 2-4, 8, 10-12, 18-19, 22 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bangera and Wang in view of Figueroa et al. (hereinafter Figueroa), Predicting sample size required for classification performance.
Regarding Claim 2, Bangera and Wang disclose the one or more processors of claim 1.
However, Bangera fails to explicitly disclose wherein the one or more processing units compute the at least one validation score by deriving at least one validation score estimation function.
Figueroa discloses wherein the one or more processing units compute the at least one validation score by deriving at least one validation score estimation function [“a new fitting algorithm to predict classifier performance based on a learning curve. This algorithm fits an inverse power law model to a small set of initial points of a learning curve with the purpose of predicting a classifier’s performance at larger sample sizes.” pg. 3 §Methods ¶1].
It would have been obvious to one having ordinary skill in the art, having the teachings of Bangera, Wang, and Figueroa before him before the effective filing date of the claimed invention, to modify the combination to incorporate the power law model of Figueroa.
Given the advantage of determining a sample size required for achieve sufficient statistical power, one having ordinary skill in the art would have been motivated to make this obvious modification.
Regarding Claim 3, Bangera and Wang disclose the one or more processors of claim 1.
However, Bangera fails to explicitly disclose wherein the one or more processing units compute the at least one validation score by deriving at least one validation score estimation function, wherein one or more parameters of the at least one validation score estimation function are determined by curve fitting the at least one validation score corresponding to one or more iterations of the plurality of iterations.
Figueroa discloses wherein the one or more processing units compute the at least one validation score by deriving at least one validation score estimation function [“a new fitting algorithm to predict classifier performance based on a learning curve. This algorithm fits an inverse power law model to a small set of initial points of a learning curve with the purpose of predicting a classifier’s performance at larger sample sizes.” pg. 3 §Methods ¶1], wherein one or more parameters of the at least one validation score estimation function are determined by curve fitting the at least one validation score corresponding to one or more iterations of the plurality of iterations [“A learning curve is a collection of data points (xj, yj) that in this case describe how the performance of a classifier (yj) is related to training sample sizes (xj), where j = 1 to m, m being the total number of instances.” pg. 2 §Learning curve fitting ¶1; Fig. 1].
It would have been obvious to one having ordinary skill in the art, having the teachings of Bangera, Wang, and Figueroa before him before the effective filing date of the claimed invention, to modify the combination to incorporate the power law model of Figueroa.
Given the advantage of determining a sample size required for achieve sufficient statistical power, one having ordinary skill in the art would have been motivated to make this obvious modification.
Regarding Claim 4, Bangera and Wang disclose the one or more processors of claim 1.
However, Bangera fails to explicitly disclose wherein the one or more processing units compute the at least one validation score by deriving at least one validation score estimation function, further wherein the at least one validation score estimation function is solved by, at least in part, minimizing the size of the second training data set subject to the at least one validation score estimation function having a value greater than the target validation score.
Figueroa discloses wherein the one or more processing units compute the at least one validation score by deriving at least one validation score estimation function [“a new fitting algorithm to predict classifier performance based on a learning curve. This algorithm fits an inverse power law model to a small set of initial points of a learning curve with the purpose of predicting a classifier’s performance at larger sample sizes.” pg. 3 §Methods ¶1], further wherein the at least one validation score estimation function is solved by, at least in part, minimizing the size of the second training data set [“Active learning is a sampling technique that aims to minimize the size of the training set for classification” pg. 1 §Problem formulation ¶1] subject to the at least one validation score estimation function having a value greater than the target validation score [“Although termination criteria is an issue for both passive and active learning, identifying an optimal termination point and training sample size maybe more important in active learning.” pg. 2, col. 1, ¶1].
It would have been obvious to one having ordinary skill in the art, having the teachings of Bangera, Wang, and Figueroa before him before the effective filing date of the claimed invention, to modify the combination to incorporate the power law model of Figueroa.
Given the advantage of determining a sample size required for achieve sufficient statistical power, one having ordinary skill in the art would have been motivated to make this obvious modification.
Regarding Claim 8, Bangera and Wang disclose the one or more processors of claim 1. Bangera further discloses wherein the one or more processing units are further to:
determine a linear score function of ground truth based at least on training the machine learning model using the first training data set [“Progressive Sampling (PS) starts with a small data sample from the full dataset and use progressively larger samples until the model accuracy cannot increase substantially.” §IV.B ¶1; Fig. 1; Algorithm 2; “Prediction model which uses the concept of training data and test data” §II.B ¶1].
However, Wang fails to explicitly disclose determine a linear score function of ground truth based at least in part on training the machine learning model using the first training data set.
Figueroa discloses determine a linear score function of ground truth [“selected a convergence method that used linear regression with local sampling (LRLS)” pg. 3, col. 1, ¶1] based at least in part on training the machine learning model using the first training data set.
It would have been obvious to one having ordinary skill in the art, having the teachings of Bangera, Wang, and Figueroa before him before the effective filing date of the claimed invention, to modify the combination to incorporate the score function of Figueroa.
Given the advantage of determining validation of a model, one having ordinary skill in the art would have been motivated to make this obvious modification.
Regarding Claim 10, Bangera and Wang disclose the one or more processors of claim 1.
However, Bangera fails to explicitly disclose wherein the one or more processing units compute the at least one validation score by deriving at least one validation score estimation function, the at least one validation score estimation function comprising a concave monotonic increasing regression function.
Figueroa discloses wherein the one or more processing units compute the at least one validation score by deriving at least one validation score estimation function [“a new fitting algorithm to predict classifier performance based on a learning curve. This algorithm fits an inverse power law model to a small set of initial points of a learning curve with the purpose of predicting a classifier’s performance at larger sample sizes.” pg. 3 §Methods ¶1], the at least one validation score estimation function comprising a concave monotonic increasing regression function [Fig. 1; Examiner’s Note: A concave monotonic increasing regression function describes a relationship where the output rises as the input increases (monotonic increasing), but the rate of increase slows down (concave), exhibiting diminishing returns, as shown in the graph].
It would have been obvious to one having ordinary skill in the art, having the teachings of Bangera, Wang, and Figueroa before him before the effective filing date of the claimed invention, to modify the combination to incorporate the progressive sampling approach of Figueroa.
Given the advantage of determining a sample size required for achieve sufficient statistical power, one having ordinary skill in the art would have been motivated to make this obvious modification.
Regarding Claim 11, Bangera, Wang, and Figueroa disclose the processor of claim 10.
However, Bangera fails to explicitly disclose wherein the one or more processing units are further to:
determine an error in the size of the second training data set based at least on the linear score function of ground truth; and
cause the display of the error in the size of the second training data set.
Wang discloses wherein the one or more processing units are further to:
determine an error in the size of the second training data set based at least on the linear score function of ground truth [When a scoring algorithm has not achieved sufficient performance, additional training may be performed, which may include refinement of the scoring algorithm or retraining on a different first subset of the training data set, after which the new scoring algorithm may again be validated and assessed.” ¶39]; and
cause the display of the error in the size of the second training data set [a data presentation routine 138C for generating or presenting received data to the user via the display” ¶28].
It would have been obvious to one having ordinary skill in the art, having the teachings of Bangera, Wang, and Figueroa before him before the effective filing date of the claimed invention, to modify the combination to incorporate the error determination of Wang.
Given the advantage of determining error in order to determine the accuracy of the model to see if more training is required, one having ordinary skill in the art would have been motivated to make this obvious modification.
Regarding Claim 12, Bangera, Wang, and Figueroa disclose the one or more processors of claim 10. Bangera further discloses wherein the one or more processing units are further to:
determine a linear score function of ground truth [“selected a convergence method that used linear regression with local sampling (LRLS)” pg. 3, col. 1, ¶1] based at least on training the machine learning model using the first training data set [“Prediction model which uses the concept of training data and test data” §II.B ¶1; “Progressive Sampling (PS) starts with a small data sample from the full dataset and use progressively larger samples until the model accuracy cannot increase substantially.” §IV.B ¶1; Fig. 1; Algorithm 2];
compute a data collection ratio based at least in part on the linear score function of ground truth, the second number of training samples, and the first number of training samples [“Progressive Sampling (PS) starts with a small data sample from the full dataset and use progressively larger samples until the model accuracy cannot increase substantially.” §IV.B ¶1; Fig. 1; Algorithm 2];
cause the display to indicate at least one of an optimism indication or a pessimism indication for the at least one validation score estimation function based at least on the data collection ratio.
However, Bangera fails to explicitly disclose cause the display to indicate at least one of an optimism indication or a pessimism indication for the at least one validation score estimation function based at least on the data collection ratio.
Wang discloses cause the display to indicate [a data presentation routine 138C for generating or presenting received data to the user via the display” ¶28] at least one of an optimism indication or a pessimism indication for the at least one validation score estimation function based at least on the data collection ratio.
It would have been obvious to one having ordinary skill in the art, having the teachings of Bangera, Wang, and Figueroa before him before the effective filing date of the claimed invention, to modify the combination to incorporate displaying information of Wang.
Given the advantage of providing useful information to the user, one having ordinary skill in the art would have been motivated to make this obvious modification.
However, Bangera fails to explicitly disclose determine a linear score function of ground truth based at least on training the machine learning model using the first training data set;
compute a data collection ratio based at least in part on the linear score function of ground truth, the second number of training samples, and the first number of training samples;
cause the display to indicate at least one of an optimism indication or a pessimism indication for the at least one validation score estimation function based at least on the data collection ratio.
Figueroa discloses determine a linear score function of ground truth [“selected a convergence method that used linear regression with local sampling (LRLS)” pg. 3, col. 1, ¶1] based at least on training the machine learning model using the first training data set;
compute a data collection ratio based at least in part on the linear score function of ground truth [“differences between the predicted and actual classification errors were found to be in the range of 1%-7%” pg. 2, col. 2 ¶3], the second number of training samples, and the first number of training samples;
cause the display to indicate at least one of an optimism indication or a pessimism indication for the at least one validation score estimation function based at least on the data collection ratio [“start with a very small batch of instances and progressively increase the training data size until a termination criteria is met” pg. 2 §Progressive sampling ¶1; Examiner Note: if more data is required this is a pessimistic indication, but if less data is required this is optimistic indication].
It would have been obvious to one having ordinary skill in the art, having the teachings of Bangera, Wang, and Figueroa before him before the effective filing date of the claimed invention, to modify the combination to incorporate the data ratio of Figueroa.
Given the advantage of calculating between actual and predicted values to determine validation, one having ordinary skill in the art would have been motivated to make this obvious modification.
Regarding Claim 18, Bangera and Wang disclose the system of claim 16.
However, Bangera fails to explicitly disclose wherein the at least one validation score estimation function comprises a concave monotonic increasing regression function.
Figueroa discloses wherein the at least one validation score estimation function comprises a concave monotonic increasing regression function [Fig. 1; Examiner’s Note: A concave monotonic increasing regression function describes a relationship where the output rises as the input increases (monotonic increasing), but the rate of increase slows down (concave), exhibiting diminishing returns, as shown in the graph].
It would have been obvious to one having ordinary skill in the art, having the teachings of Bangera, Wang, and Figueroa before him before the effective filing date of the claimed invention, to modify the combination to incorporate the progressive sampling approach of Figueroa.
Given the advantage of determining a sample size required for achieve sufficient statistical power, one having ordinary skill in the art would have been motivated to make this obvious modification.
Regarding Claim 19, Bangera and Wang discloses the system of claim 16.
However, Bangera fails to explicitly disclose wherein the one or more processing units are further to:
curve fit the plurality of validation scores to compute one or more parameters of the at least one validation score estimation function.
Figueroa discloses wherein the one or more processing units are further to:
curve fit the plurality of validation scores to compute one or more parameters of the at least one validation score estimation function [“A learning curve is a collection of data points (xj, yj) that in this case describe how the performance of a classifier (yj) is related to training sample sizes (xj), where j = 1 to m, m being the total number of instances.” pg. 2 §Learning curve fitting ¶1; Fig. 1].
It would have been obvious to one having ordinary skill in the art, having the teachings of Bangera, Wang, and Figueroa before him before the effective filing date of the claimed invention, to modify the combination to incorporate the power law model of Figueroa.
Given the advantage of determining a sample size required for achieve sufficient statistical power, one having ordinary skill in the art would have been motivated to make this obvious modification.
Regarding Claim 22, Bangera and Wang disclose the system of claim 16. Bangera further discloses wherein the one or more processing units are further to:
derive another validation score estimation function based at least on iteratively re- training the machine learning model using the regression data set to compute a second respective plurality of validation scores, the second respective plurality of validation scores comprising a respective validation score for each iteration of a plurality of iterations [“Prediction model which uses the concept of training data and test data” §II.B ¶1; “Progressive Sampling (PS) starts with a small data sample from the full dataset and use progressively larger samples until the model accuracy cannot increase substantially.” §IV.B ¶1; Fig. 1; Algorithm 2], wherein the another validation score estimation function comprises a concave monotonic increasing regression function different from the at least one validation score estimation function;
solve, based at least on the target validation score, the additional validation score estimation function to determine a second size of the second training data set comprising a second additional number of training samples [“Prediction model which uses the concept of training data and test data” §II.B ¶1; “Progressive Sampling (PS) starts with a small data sample from the full dataset and use progressively larger samples until the model accuracy cannot increase substantially.” §IV.B ¶1; Fig. 1; Algorithm 2]; and
cause a display to present the size of the second training data set and the second size of the second training data set [“Prediction model which uses the concept of training data and test data” §II.B ¶1; “Progressive Sampling (PS) starts with a small data sample from the full dataset and use progressively larger samples until the model accuracy cannot increase substantially.” §IV.B ¶1; Fig. 1; Algorithm 2].
However, Bangera fails to explicitly disclose cause a display to present.
Wang discloses cause a display to present [“a data presentation routine 138C for generating or presenting received data to the user via the display” ¶28].
It would have been obvious to one having ordinary skill in the art, having the teachings of Bangera and Wang before him before the effective filing date of the claimed invention, to modify the combination to incorporate displaying information of Wang.
Given the advantage of providing useful information to the user, one having ordinary skill in the art would have been motivated to make this obvious modification.
However, Bangera fails to explicitly disclose wherein the another validation score estimation function comprises a concave monotonic increasing regression function different from the at least one validation score estimation function.
Figueroa discloses wherein the another validation score estimation function comprises a concave monotonic increasing regression function different from the at least one validation score estimation function [“methods for progressive sampling and selected a convergence method that used linear regression with local sampling (LRLS)” pg. 3, col. 1 ¶1; Fig. 1; Examiner’s Note: A concave monotonic increasing regression function describes a relationship where the output rises as the input increases (monotonic increasing), but the rate of increase slows down (concave), exhibiting diminishing returns, as shown in the graph].
It would have been obvious to one having ordinary skill in the art, having the teachings of Bangera, Wang, and Figueroa before him before the effective filing date of the claimed invention, to modify the combination to incorporate the progressive sampling approach of Figueroa.
Given the advantage of determining a sample size required for achieve sufficient statistical power, one having ordinary skill in the art would have been motivated to make this obvious modification.
Claim(s) 6, 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bangera and Wang in view of Wang et al. (hereinafter QiWang), A Comprehensive Survey of Loss Functions in Machine Learning.
Regarding Claim 6, Bangera and Wang disclose the one or more processors of claim 1.
However, Bangera fails to explicitly disclose wherein the one or more processing units are further to:
determine a correction factor; and
determine the size of the second training data set based at least on a sum of the target validation score and the correction factor.
QiWang discloses wherein the one or more processing units are further to:
determine a correction factor [“the second term R(f) is the regularization item representing the model complexity” §1 ¶1; Equation 1]; and
determine the second number of training samples based at least on a sum of the target validation score and the correction factor [“L(·) is the loss function, 0 is the parameter vector, the second term R(f) is the regularization item representing the model complexity” §1 ¶1; Equation 1; Examiner Note: Equation 1:
PNG
media_image1.png
114
354
media_image1.png
Greyscale
shows the validation score (i.e., the loss function) being added to the correction factor (i.e., regularization term)].
It would have been obvious to one having ordinary skill in the art, having the teachings of Bangera, Wang, and QiWang before him before the effective filing date of the claimed invention, to modify the combination to incorporate the regularization term of QiWang.
Given the advantage of preventing overfitting and penalizing complexity, one having ordinary skill in the art would have been motivated to make this obvious modification.
Regarding Claim 20, Bangera and Wang disclose the system of claim 16.
However, Bangera fails to explicitly disclose wherein the one or more processing units are further to:
solve, based at least on a sum of the target validation score and a correction factor, the at least one validation score estimation function to determine another additional number of training samples.
QiWang discloses wherein the one or more processing units are further to:
solve, based at least on a sum of the target validation score and a correction factor [“the second term R(f) is the regularization item representing the model complexity” §1 ¶1; Equation 1], the at least one validation score estimation function to determine another additional number of training samples [“L(·) is the loss function, 0 is the parameter vector, the second term R(f) is the regularization item representing the model complexity” §1 ¶1; Equation 1; Examiner Note: Equation 1:
PNG
media_image1.png
114
354
media_image1.png
Greyscale
shows the validation score (i.e., the loss function) being added to the correction factor (i.e., regularization term) to solve the optimization problem].
It would have been obvious to one having ordinary skill in the art, having the teachings of Bangera, Wang, and QiWang before him before the effective filing date of the claimed invention, to modify the combination to incorporate the regularization term of QiWang.
Given the advantage of preventing overfitting and penalizing complexity, one having ordinary skill in the art would have been motivated to make this obvious modification.
Claim(s) 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bangera, Wang and QiWang, in view of He, Learn Biologically Meaningful Representation with Transfer Learning.
Regarding Claim 7, Bangera, Wang and QiWang disclose the one or more processors of claim 6.
However, Bangera fails to explicitly disclose wherein the correction factor is computed from a second training data set used to train a second machine learning model.
He discloses wherein the correction factor is computed from a second training data set used to train a second machine learning model [“The feature encoder networks for different domains could share weights, share similar regularization or nonrelated” pg. 26 ¶2].
It would have been obvious to one having ordinary skill in the art, having the teachings of Bangera, Wang, QiWang, and He before him before the effective filing date of the claimed invention, to modify the combination to incorporate receiving a correction factor from another model.
Given the advantage of reusing correction data to save resources on recalculating it for each model, one having ordinary skill in the art would have been motivated to make this obvious modification.
Claim(s) 9, 21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bangera, Wang, Figueroa, and QiWang.
Regarding Claim 9, Bangera, Wang and Figueroa disclose the one or more processors of claim 8. Bangera further discloses wherein the one or more processing units are further to:
compute a data collection ratio based at least on the linear score function of ground truth, the second number of training samples, and the first number of training samples [“Progressive Sampling (PS) starts with a small data sample from the full dataset and use progressively larger samples until the model accuracy cannot increase substantially.” §IV.B ¶1; Fig. 1; Algorithm 2].
However, Wang fails to explicitly disclose compute a data collection ratio based at least in part on the linear score function of ground truth, the second number of training samples, and the first number of training samples.
Figueroa discloses compute a data collection ratio based at least on the linear score function of ground truth [“differences between the predicted and actual classification errors were found to be in the range of 1%-7%” pg. 2, col. 2 ¶3], the second number of training samples, and the first number of training samples.
It would have been obvious to one having ordinary skill in the art, having the teachings of Bangera, Wang and Figueroa before him before the effective filing date of the claimed invention, to modify the combination to incorporate the data ratio of Figueroa.
Given the advantage of calculating between actual and predicted values to determine validation, one having ordinary skill in the art would have been motivated to make this obvious modification.
However, Bangera fails to explicitly disclose compute a correction factor, the correction factor computed to generate a ratio greater than one for the data collection ratio.
QiWang discloses compute a correction factor, the correction factor computed to generate a ratio greater than one for the data collection ratio [“ λ ≥ 0 is the tradeoff to balance the empirical risk and the model” §1 ¶1; Examiner Note: in the equation, λ and the regularization term can be the correction factor].
It would have been obvious to one having ordinary skill in the art, having the teachings of Bangera, Wang, Figueroa, and QiWang before him before the effective filing date of the claimed invention, to modify the combination to incorporate a ratio greater than a data ratio.
Given the advantage of validating a model for accuracy and avoiding overfitting, one having ordinary skill in the art would have been motivated to make this obvious modification.
Claim 21 is rejected on the same grounds as claim 9.
Examiner’s Note
The Examiner respectfully requests of the Applicant in preparing responses, to fully consider the entirety of the reference(s) as potentially teaching all or part of the claimed invention. It is noted, REFERENCES ARE RELEVANT AS PRIOR ART FOR ALL THEY CONTAIN. “The use of patents as references is not limited to what the patentees describe as their own inventions or to the problems with which they are concerned. They are part of the literature of the art, relevant for all they contain.” In re Heck, 699 F.2d 1331, 1332-33, 216 USPQ 1038, 1039 (Fed. Cir. 1983) (quoting In re Lemelson, 397 F.2d 1006, 1009, 158 USPQ 275, 277 (CCPA 1968)). A reference may be relied upon for all that it would have reasonably suggested to one having ordinary skill in the art, including non-preferred embodiments (see MPEP 2123). The Examiner has cited particular locations in the reference(s) as applied to the claim(s) above for the convenience of the Applicant. Although the specified citations are representative of the teachings of the art and are applied to the specific limitations within the individual claim(s), typically other passages and figures will apply as well.
Additionally, any claim amendments for any reason should include remarks indicating clear support in the originally filed specification.
Response to Arguments
Regarding the title, Applicant indicates the “title is descriptive in view of the amendments submitted herewith” however no amendment to the title was submitted. Accordingly, the objection to the title is maintained.
Regarding the §101 rejections, Applicant's arguments have been fully considered but have been found unpersuasive. Applicant argues that 1) claim 1 does not recite a mental process, and 2) claim 1 integrates any judicial exception into a practical application. Examiner disagrees for at least the following reasons.
First, the claim recites a mental process. Specifically, the computation of a score is a mathematical concept which can be done mentally, and the determination of a size of a data set is observation and judgement, which can both be done mentally. While the processors and training the machine learning model are additional elements, the other limitations can be practically performed in the mind. For example, a person can compute a validation score based on the model’s outputs, or a person can determine a size of the data set (i.e., increase or decrease it) based on an actual score and a target score. Therefore, the claim does recite a mental process.
Second, the additional elements do not integrate the judicial exception into a practical application. There is no improvement to a technical field that is realized in the claim language. Rather, any alleged improvement is to the abstract idea. The additional element of processors is merely to apply the judicial exception to a generic computing environment, the receiving and displaying are merely extra solution activities, and the use of a machine learning model is merely applying the judicial exception to use a computer as a tool. Accordingly, the additional elements do not integrate the judicial exception into a practical application.
For at least these reasons, the rejections are maintained.
Regarding the prior art rejections, Applicant's arguments with respect to the claims have been considered but are moot because the arguments do not apply to the references being used in the current rejection of the limitations.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
Elomaa et al. (Progressive Rademacher Sampling) discloses progressive sampling. The idea in it is to increase the sample size according to some schedule until accuracy close to that which would be obtained using all of the data is reached.
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ROBERT H BEJCEK II whose telephone number is (571)270-3610. The examiner can normally be reached Monday - Friday: 9:00am - 5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michelle T. Bechtold can be reached at (571) 431-0762. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/R.B./ Examiner, Art Unit 2148
/MICHELLE T BECHTOLD/ Supervisory Patent Examiner, Art Unit 2148
