DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 08/19/2026 has been entered.
Response to Amendment
The amendment filed 08/19/2026 has been entered. Claims 18-37 remain pending in the application.
Response to Arguments
Applicant’s arguments, filed 08/19/2026, with respect to the rejections of claims 18 and 28 under 103 have been fully considered and are persuasive because of the amendments. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Li et al. (US Pub. 2016/0078339) in view of Rosswog et al. (US Patent 9,514,414) and further in view of Forman et al. (US Pub. 2006/0224538).
Applicant argues (pages 11-13) that “The Rejection Lacks a Reasoned Motivation With Rational Underpinning” because Li, Rosswog, and Paquet address different problems in different technical settings …
In response
Independents claims 18 and 28 are rejected under 103 by a new ground of rejection (Li et al. in view of Rosswog et al. and further in view of Forman et al.) because of the amendments, therefore, argument related to Paquet reference is moot.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 18-19 and 28-29 are rejected under 35 U.S.C. 103 as being unpatentable over Li et al. (US Pub. 2016/0078339) in view of Rosswog et al. (US Patent 9,514,414) and further in view of Forman et al. (US Pub. 2006/0224538).
As per claim 18, Li teaches a machine learning (ML) computer system comprising [abstract, “Systems and methods are provided for generating a DNN classifier by "learning" a "student" DNN model from a larger more accurate "teacher" DNN model”]:
a student ML system that is iteratively trained through machine learning on training data to perform a machine learning task [Fig. 3, abstract, “an iterative process is applied to train the student DNN by minimize the divergence of the output distributions from the teacher and student DNN models”; paragraph 0005, “To learn a DNN with a smaller number of hidden nodes, a larger size (more accurate) "teacher" DNN is used to train the smaller "student" DNN … The student DNN may be trained from un-la be led (or un-transcribed) data … The student DNN may be iteratively optimized until its output converges with the output of the teacher DNN”];
a reference system for generating the training data for the student ML system [Fig. 3, paragraphs 0042-0043, “system 300 for learning a smaller student DNN from a larger teacher DNN … teacher DNN 302 comprises a trained DNN model … Initially, student DNN 301 is untrained or may be pre-trained, but has not yet been trained by the teacher DNN … for each iteration, a small piece of unlabeled (or un-transcribed) data 310 is provided to both student DNN 301 and teacher DNN 302. Using forward propagation, the posterior distribution (output distribution 351 and 352) is determined. An error signal 360 is then determined from the distribution 351 and 352. The error signal may be calculated by determining the KL divergence between distributions 351 and 352 … If the output distribution 351 of student DNN 301 has converged with the output distribution 352 of teacher DNN 302, then the student DNN is deemed to be trained. However, if the output has not converged, and in some embodiments the output still appears to be converging, then the student DNN 301 is trained based on the error”; paragraphs 0026-0027, “As shown in FIG. 1, storage 106 includes DNN models 107 and 109. DNN model 107 represents a teacher DNN model, and DNN model 109 represents a student DNN model having a smaller size than teacher DNN model 107 … DNN models (or DNN classifiers) … The DNN model generator 120, in general, is responsible for generating DNN models, such as the CD-DNN-HMM classifiers”; The examiner interprets the teacher DNN (classifier) as a reference system. It can be seen that Fig. 3 and the above cited paragraphs disclose a process of training a student DNN using the teacher DNN. At each iteration, the teacher DNN (reference system) process input data to generate an output which is used to train the student DNN. The examiner interprets the output generated by the teacher DNN (reference system) at each iteration as the training data for the student DNN], wherein the reference system is for generating the training data for the ML system based on classifications that the reference system produces for corresponding inputs [Fig. 3, paragraphs 0042-0043, “system 300 for learning a smaller student DNN from a larger teacher DNN … teacher DNN 302 comprises a trained DNN model … Initially, student DNN 301 is untrained or may be pre-trained, but has not yet been trained by the teacher DNN … for each iteration, a small piece of unlabeled (or un-transcribed) data 310 is provided to both student DNN 301 and teacher DNN 302. Using forward propagation, the posterior distribution (output distribution 351 and 352) is determined. An error signal 360 is then determined from the distribution 351 and 352. The error signal may be calculated by determining the KL divergence between distributions 351 and 352 … If the output distribution 351 of student DNN 301 has converged with the output distribution 352 of teacher DNN 302, then the student DNN is deemed to be trained. However, if the output has not converged, and in some embodiments the output still appears to be converging, then the student DNN 301 is trained based on the error”; As explained above, Fig. 3 and the above cited paragraphs disclose a process of training a student DNN using the teacher DNN. At each iteration, the teacher DNN (reference system) process input data to generate an output which is used to train the student DNN, and since the teacher DNN (reference system) is a classifier, it generates a classification output for corresponding input]; and
a learning experimentation system [Fig. 1 shows the evaluating component 128], wherein:
the learning experimentation system comprises a computer [Fig. 7, paragraph 0021, “The components shown in FIG. 1 may be implemented on or using one or more computing devices, such as computing device 700 described in connection to FIG. 7”];
the learning experimentation system receives observations from the student ML system during training of the student ML system [paragraphs 0036-0037, “evaluating component 128 evaluates the output distributions of the student and teacher DNNs, determines the difference (which may be determined as an error signal) between the outputs and also determines whether the student is continuing to improve or whether the student is no longer improving … evaluating component 128 determine whether to complete another iteration … evaluating component 128 determines whether to continue iterating based on whether the student is continuing to show improvement … evaluating component 128 apply a threshold to determine convergence of the teacher DNN and student DNN output distributions. Where the threshold is not satisfied, iteration may continue, thereby further training the student to approximate the teacher. Where the threshold is satisfied, then convergence is determined (indicating the student output distribution is sufficiently close enough to the teacher DNN' s output distribution) and the student DNN may be considered trained … evaluating component 128 evaluates the student DNN according to the methods 500 and 600”];
the reference system generates a set of additional training data for the student ML system [paragraph 0005, “The student DNN may be iteratively optimized until its output converges with the output of the teacher DNN”; Fig. 3, paragraphs 0034 and 0043, disclose a process of training the student DNN, where, at each iteration, unlabeled input data is provided to both student DNN and teacher DNN (reference system), the outputs from the DNNs are compared to determine the error, if the output of the student DNN has not converged with the output of the teacher DNN, then the student is trained again (complete another iteration) using the output generated by the teacher DNN (reference system) which examiner interprets as the additional training data for the student DNN]; and
the set of additional training data is input to the student ML system such that the set of additional training data trains the student ML system to imitate classifications by the reference system [Fig. 3, paragraphs 0034 and 0043, disclose a process of training the student DNN, where, at each iteration, unlabeled input data is provided to both student DNN and teacher DNN (reference system), the outputs from the DNNs are compared to determine the error, if the output of the student DNN has not converged with the output of the teacher DNN, then the student is trained again (for another iteration) using the output generated by the teacher DNN (reference system) as a training input; paragraph 0005, “The student DNN may be iteratively optimized until its output converges with the output of the teacher DNN. In this way, the student DNN approaches the behavior of the teacher, so that whatever the output of the teacher, the student will approximate”; It can be seen that the student DNN is iteratively trained using the outputs generated by the teacher DNN (additional training data) as input until its output converges with the output of the teacher DNN (reference system); paragraph 0027, “training an initialized "student" DNN model to approximate a trained teacher DNN model having a larger model size (e.g. number of parameters) than the student”].
Li does not teach
wherein the reference system comprises two or more classifiers each configured to produce a respective classification for a common input;
based on the observations, the learning experimentation system transmits a control parameter to the reference system;
based on the control parameter, the reference system generates a set of additional training data for the student ML system, wherein the control parameter sets a probability value used to randomly select, for a common input to the two or more classifiers, one of the respective classifications produced by the two or more classifiers for inclusion in the set of additional training data for the student ML system;
Rosswog teaches
based on the observations, the learning experimentation system transmits a control parameter to the reference system [Fig. 2, Col. 12, lines 37-66, “document categorizer 250 may include a performance tracker 254 that tracks one or more metrics associated with the performance of document categorizer 250's categorizations. The metrics may include the number of electronic documents categorized in each category (e.g., relevant and not relevant), the confidence modifiers of all the categorized electronic documents … document categorizer 250 may send an indication to seed set generator 230 (included in an admin subsystem 111) that a second or subsequent seed set of electronic document classifications is needed (control parameter- increasing the training data) to retrain machine learning algorithm 252 to improve its categorization performance”; Examiner interprets an admin subsystem 111 which comprising a seed set generator 230 as a reference system that generating the training data];
based on the control parameter, the reference system generates a set of additional training data for the student ML system [Col. 12, line 37 – Col. 13, lines 1-2, “document categorizer 250 may include a performance tracker 254 that tracks one or more metrics associated with the performance of document categorizer 250's categorizations … document categorizer 250 may send an indication to seed set generator 230 (included in an admin subsystem 111) that a second or subsequent seed set of electronic document classifications is needed (control parameter- increasing the training data) to retrain machine learning algorithm 252 to improve its categorization performance … seed set generator 230 may generate additional seed sets based on the metrics tracked by performance tracker 254”];
Since Li teaches a process of iteratively training a student DNN from trained teacher DNN, at each iteration, the student DNN is trained using the output from the teacher DNN, the outputs from the student DNN and the teacher DNN is compared by the evaluating component to determine convergence, if the output has not converged, iteration may continue to further training the student to approximate the teacher, and the teacher DNN will process the input to generate another output (additional training data) that is used to train/retrain the student DNN.
Li, however, is silent of the teacher DNN (reference system) receiving a signal (control parameter) from the evaluating component (the learning experimentation system) to complete another iteration to generate another output (additional training data) for training the student DNN.
While Rosswog teaches a seed set generator 230 (reference system) receives a control parameter (to increase the training data) from the learning experimentation system to generate additional training data to retrain machine learning algorithm 252 to improve its categorization performance.
Therefore, the combination of Li and Rosswog teaches the above claim limitation.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the method for generating a DNN classifier by learning a student DNN model from a larger teacher DNN model of Li to include the reference system to receive from the learning experimentation system, a control parameter for generating a set of additional training data of Rosswog. Doing so would help training or retraining a network with the additional training data generated by the reference system to improve the performance of the model (Rosswog, Col. 5, lines 49-53).
Li and Rosswog do not teach
the reference system comprises two or more classifiers each configured to produce a respective classification for a common input;
the control parameter sets a probability value used to randomly select, for a common input to the two or more classifiers, one of the respective classifications produced by the two or more classifiers for inclusion in the set of additional training data for the student ML system;
Forman teaches
the reference system comprises two or more classifiers each configured to produce a respective classification for a common input [Fig. 2, paragraph 0014, disclose classifier system 128 comprises two set of classifiers, the first set includes classifiers 130a – 130d, each receives the same training data 122 as input to generate an output (a respective classification such as health, financial, sports and soccer), and those outputs can be used as the training data to train other set of classifiers];
the control parameter [paragraph 0019, “generate the classifiers 130e-130h in the second set may include parameters that take maximum score into consideration”] sets a probability value used to randomly select, for a common input to the two or more classifiers, one of the respective classifications produced by the two or more classifiers for inclusion in the set of additional training data for the student ML system [paragraphs 0014-0019, “the second set receives as input the predictions from the first set of classifiers … The classifiers 130e-130h shown in FIG. 2 are a second set of classifiers that correspond to the first set of classifiers including the classifiers 130a-130d. In particular, the classifier 130e corresponds to the classifier 130a because both classifiers are predicting for the same category, such as the health category … Each of the classifiers 130e-130h is trained with the same training set as the classifiers 130a-d, as shown with the input 210. However, in addition to the input 210, the classifiers 130e-130h input the predictions of non-corresponding classifiers from the first set for classifying cases and for training … For example, the classifier 130e receives the predictions from the non-corresponding classifiers 130b-130d in the first set of classifiers … The classifier 130a predicts that there is a 55% probability that a document in the training set 122 should be classified under the health category. The classifier 130b predicts that there is a 95% probability that the document should be classified under label 2, which is the financial category. The classifier 130e receives the prediction of the classifier 130b and predicts that the document should not be classified under the health category, which is label 1 … a maximum score of non-corresponding classifiers in the first set may be used as an additional input attribute for the second set of classifiers or the maximum score may be determined by each of the second set of classifiers based on the predictions of respective non-corresponding classifiers … For example, if the classifier 130e receives predictions of 85%, 35%, and 10% from classifiers 130b-d respectively for classifying the same document, the maximum score is 85% for the category for label 2. This maximum score is used as an additional input attribute by the classifier 130e to make its prediction … maximum score is a probability or another parameter representing a confidence or degree of confidence that a case belongs to a category”; It can be seen that a probability value is set for each prediction (classification) produces by classifiers 130a-130d which are trained with the same input training set, and the prediction (classification) with highest probability value is selected for inclusion in the set of additional training data to train other classifiers];
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the method for generating a DNN classifier by learning a student DNN model from a larger teacher DNN model of Li to include the reference system comprises two or more classifiers each configured to produce a respective classification for a common input, and the control parameter sets a probability value used to randomly select one of the respective classifications produced by the two or more classifiers for inclusion in the set of additional training data for other model of Forman. Doing so would help improving prediction accuracy of the predictions generated by the classifiers (Forman, 0014).
As per claim 19, Li, Rosswog and Forman teach the ML computer system of claim 18.
Li further teaches
the observations are a learning behavior and performance of the student ML system [paragraphs 0036-0037, “evaluating component 128 evaluates the output distributions of the student and teacher DNNs, determines the difference (which may be determined as an error signal) between the outputs and also determines whether the student is continuing to improve or whether the student is no longer improving … evaluating component 128 determine whether to complete another iteration … evaluating component 128 determines whether to continue iterating based on whether the student is continuing to show improvement … evaluating component 128 apply a threshold to determine convergence of the teacher DNN and student DNN output distributions. Where the threshold is not satisfied, iteration may continue, thereby further training the student to approximate the teacher. Where the threshold is satisfied, then convergence is determined (indicating the student output distribution is sufficiently close enough to the teacher DNN' s output distribution) and the student DNN may be considered trained … evaluating component 128 evaluates the student DNN according to the methods 500 and 600”; paragraph 0005, “The student DNN may be iteratively optimized until its output converges with the output of the teacher DNN. In this way, the student DNN approaches the behavior of the teacher, so that whatever the output of the teacher, the student will approximate”].
As per claim 28, Li teaches a computerized method of improving operation of a machine learning (ML) system, the method comprising [abstract, “Systems and methods are provided for generating a DNN classifier by "learning" a "student" DNN model from a larger more accurate "teacher" DNN model”]:
initial training, iteratively through machine learning, by a computer system, a student ML system on training data to perform a machine learning task [Fig. 3, abstract, “an iterative process is applied to train the student DNN by minimize the divergence of the output distributions from the teacher and student DNN models”; paragraph 0005, “To learn a DNN with a smaller number of hidden nodes, a larger size (more accurate) "teacher" DNN is used to train the smaller "student" DNN … The student DNN may be trained from un-la be led (or un-transcribed) data … The student DNN may be iteratively optimized until its output converges with the output of the teacher DNN”],
generating, by a reference system of the computer system, the training data for the student ML system [Fig. 3, paragraphs 0042-0043, “system 300 for learning a smaller student DNN from a larger teacher DNN … teacher DNN 302 comprises a trained DNN model … Initially, student DNN 301 is untrained or may be pre-trained, but has not yet been trained by the teacher DNN … for each iteration, a small piece of unlabeled (or un-transcribed) data 310 is provided to both student DNN 301 and teacher DNN 302. Using forward propagation, the posterior distribution (output distribution 351 and 352) is determined. An error signal 360 is then determined from the distribution 351 and 352. The error signal may be calculated by determining the KL divergence between distributions 351 and 352 … If the output distribution 351 of student DNN 301 has converged with the output distribution 352 of teacher DNN 302, then the student DNN is deemed to be trained. However, if the output has not converged, and in some embodiments the output still appears to be converging, then the student DNN 301 is trained based on the error”; paragraphs 0026-0027, “As shown in FIG. 1, storage 106 includes DNN models 107 and 109. DNN model 107 represents a teacher DNN model, and DNN model 109 represents a student DNN model having a smaller size than teacher DNN model 107 … DNN models (or DNN classifiers) … The DNN model generator 120, in general, is responsible for generating DNN models, such as the CD-DNN-HMM classifiers”; The examiner interprets the teacher DNN (classifier) as a reference system. It can be seen that Fig. 3 and the above cited paragraphs disclose a process of training a student DNN using the teacher DNN. At each iteration, the teacher DNN (reference system) process input data to generate an output which is used to train the student DNN. The examiner interprets the output generated by the teacher DNN (reference system) at each iteration as the training data for the student DNN], wherein the training data comprises classifications produced by the reference system for corresponding inputs [Fig. 3, paragraphs 0042-0043, “system 300 for learning a smaller student DNN from a larger teacher DNN … teacher DNN 302 comprises a trained DNN model … Initially, student DNN 301 is untrained or may be pre-trained, but has not yet been trained by the teacher DNN … for each iteration, a small piece of unlabeled (or un-transcribed) data 310 is provided to both student DNN 301 and teacher DNN 302. Using forward propagation, the posterior distribution (output distribution 351 and 352) is determined. An error signal 360 is then determined from the distribution 351 and 352. The error signal may be calculated by determining the KL divergence between distributions 351 and 352 … If the output distribution 351 of student DNN 301 has converged with the output distribution 352 of teacher DNN 302, then the student DNN is deemed to be trained. However, if the output has not converged, and in some embodiments the output still appears to be converging, then the student DNN 301 is trained based on the error”; As explained above, Fig. 3 and the above cited paragraphs disclose a process of training a student DNN using the teacher DNN. At each iteration, the teacher DNN (reference system) process input data to generate an output which is used to train the student DNN, and since the teacher DNN (reference system) is a classifier, it generates a classification output for corresponding input]; and
following the initial training of the student ML system:
receiving, by a learning experimentation system of the computer system, observations from the student ML system from training of the student ML system [paragraphs 0036-0037, “evaluating component 128 evaluates the output distributions of the student and teacher DNNs, determines the difference (which may be determined as an error signal) between the outputs and also determines whether the student is continuing to improve or whether the student is no longer improving … evaluating component 128 determine whether to complete another iteration … evaluating component 128 determines whether to continue iterating based on whether the student is continuing to show improvement … evaluating component 128 apply a threshold to determine convergence of the teacher DNN and student DNN output distributions. Where the threshold is not satisfied, iteration may continue, thereby further training the student to approximate the teacher. Where the threshold is satisfied, then convergence is determined (indicating the student output distribution is sufficiently close enough to the teacher DNN' s output distribution) and the student DNN may be considered trained … evaluating component 128 evaluates the student DNN according to the methods 500 and 600”];
generating, by the reference system, a set of additional training data for the student ML system [paragraph 0005, “The student DNN may be iteratively optimized until its output converges with the output of the teacher DNN”; Fig. 3, paragraphs 0034 and 0043, disclose a process of training the student DNN, where, at each iteration, unlabeled input data is provided to both student DNN and teacher DNN (reference system), the outputs from the DNNs are compared to determine the error, if the output of the student DNN has not converged with the output of the teacher DNN, then the student is trained again (complete another iteration) using the output generated by the teacher DNN (reference system) which examiner interprets as the additional training data for the student DNN];
inputting the set of additional training data to the student ML system; and training the student ML system with the additional training data to imitate classifications by the reference system [Fig. 3, paragraphs 0034 and 0043, disclose a process of training the student DNN, where, at each iteration, unlabeled input data is provided to both student DNN and teacher DNN (reference system), the outputs from the DNNs are compared to determine the error, if the output of the student DNN has not converged with the output of the teacher DNN, then the student is trained again (for another iteration) using the output generated by the teacher DNN (reference system) as a training input; paragraph 0005, “The student DNN may be iteratively optimized until its output converges with the output of the teacher DNN. In this way, the student DNN approaches the behavior of the teacher, so that whatever the output of the teacher, the student will approximate”; It can be seen that the student DNN is iteratively trained using the outputs generated by the teacher DNN (additional training data) as input until its output converges with the output of the teacher DNN (reference system); paragraph 0027, “training an initialized "student" DNN model to approximate a trained teacher DNN model having a larger model size (e.g. number of parameters) than the student”].
Li does not teach
wherein the reference system comprises two or more classifiers each configured to produce a respective classification for a common input;
transmitting, by the learning experimentation system, a control parameter based on the observation to the reference system;
generating, by the reference system, a set of additional training data for the student ML system based on the control parameter, wherein the control parameter sets a probability value used to randomly select, for a common input to the two or more classifiers, one of the respective classifications produced by the two or more classifier for inclusion in the set of additional training data for the student ML system;
Rosswog teaches
transmitting, by the learning experimentation system, a control parameter based on the observation to the reference system [Fig. 2, Col. 12, lines 37-66, “document categorizer 250 may include a performance tracker 254 that tracks one or more metrics associated with the performance of document categorizer 250's categorizations. The metrics may include the number of electronic documents categorized in each category (e.g., relevant and not relevant), the confidence modifiers of all the categorized electronic documents … document categorizer 250 may send an indication to seed set generator 230 (included in an admin subsystem 111) that a second or subsequent seed set of electronic document classifications is needed (control parameter- increasing the training data) to retrain machine learning algorithm 252 to improve its categorization performance”; Examiner interprets an admin subsystem 111 which comprising a seed set generator 230 as a reference system that generating the training data];
generating, by the reference system, a set of additional training data for the student ML system based on the control parameter [Col. 12, line 37 – Col. 13, lines 1-2, “document categorizer 250 may include a performance tracker 254 that tracks one or more metrics associated with the performance of document categorizer 250's categorizations … document categorizer 250 may send an indication to seed set generator 230 (included in an admin subsystem 111) that a second or subsequent seed set of electronic document classifications is needed (control parameter- increasing the training data) to retrain machine learning algorithm 252 to improve its categorization performance … seed set generator 230 may generate additional seed sets based on the metrics tracked by performance tracker 254”];
Since Li teaches a process of iteratively training a student DNN from trained teacher DNN, at each iteration, the student DNN is trained using the output from the teacher DNN, the outputs from the student DNN and the teacher DNN is compared by the evaluating component to determine convergence, if the output has not converged, iteration may continue to further training the student to approximate the teacher, and the teacher DNN will process the input to generate another output (additional training data) that is used to train/retrain the student DNN.
Li, however, is silent of the teacher DNN (reference system) receiving a signal (control parameter) from the evaluating component (the learning experimentation system) to complete another iteration to generate another output (additional training data) for training the student DNN.
While Rosswog teaches a seed set generator 230 (reference system) receives a control parameter (to increase the training data) from the learning experimentation system to generate additional training data to retrain machine learning algorithm 252 to improve its categorization performance.
Therefore, the combination of Li and Rosswog teaches the above claim limitation.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the method for generating a DNN classifier by learning a student DNN model from a larger teacher DNN model of Li to include the reference system to receive from the learning experimentation system, a control parameter for generating a set of additional training data of Rosswog. Doing so would help training or retraining a network with the additional training data generated by the reference system to improve the performance of the model (Rosswog, Col. 5, lines 49-53).
Li and Rosswog do not teach
wherein the reference system comprises two or more classifiers each configured to produce a respective classification for a common input;
the control parameter sets a probability value used to randomly select, for a common input to the two or more classifiers, one of the respective classifications produced by the two or more classifier
Forman teaches
the reference system comprises two or more classifiers each configured to produce a respective classification for a common input [Fig. 2, paragraph 0014, disclose classifier system 128 comprises two set of classifiers, the first set includes classifiers 130a – 130d, each receives the same training data 122 as input to generate an output (a respective classification such as health, financial, sports and soccer), and those outputs can be used as the training data to train other set of classifiers];
the control parameter [paragraph 0019, “generate the classifiers 130e-130h in the second set may include parameters that take maximum score into consideration”] sets a probability value used to randomly select, for a common input to the two or more classifiers, one of the respective classifications produced by the two or more classifiers for inclusion in the set of additional training data for the student ML system [paragraphs 0014-0019, “the second set receives as input the predictions from the first set of classifiers … The classifiers 130e-130h shown in FIG. 2 are a second set of classifiers that correspond to the first set of classifiers including the classifiers 130a-130d. In particular, the classifier 130e corresponds to the classifier 130a because both classifiers are predicting for the same category, such as the health category … Each of the classifiers 130e-130h is trained with the same training set as the classifiers 130a-d, as shown with the input 210. However, in addition to the input 210, the classifiers 130e-130h input the predictions of non-corresponding classifiers from the first set for classifying cases and for training … For example, the classifier 130e receives the predictions from the non-corresponding classifiers 130b-130d in the first set of classifiers … The classifier 130a predicts that there is a 55% probability that a document in the training set 122 should be classified under the health category. The classifier 130b predicts that there is a 95% probability that the document should be classified under label 2, which is the financial category. The classifier 130e receives the prediction of the classifier 130b and predicts that the document should not be classified under the health category, which is label 1 … a maximum score of non-corresponding classifiers in the first set may be used as an additional input attribute for the second set of classifiers or the maximum score may be determined by each of the second set of classifiers based on the predictions of respective non-corresponding classifiers … For example, if the classifier 130e receives predictions of 85%, 35%, and 10% from classifiers 130b-d respectively for classifying the same document, the maximum score is 85% for the category for label 2. This maximum score is used as an additional input attribute by the classifier 130e to make its prediction … maximum score is a probability or another parameter representing a confidence or degree of confidence that a case belongs to a category”; It can be seen that a probability value is set for each prediction (classification) produces by classifiers 130a-130d which are trained with the same input training set, and the prediction (classification) with highest probability value is selected for inclusion in the set of additional training data to train other classifiers];
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the method for generating a DNN classifier by learning a student DNN model from a larger teacher DNN model of Li to include the reference system comprises two or more classifiers each configured to produce a respective classification for a common input, and the control parameter sets a probability value used to randomly select one of the respective classifications produced by the two or more classifiers for inclusion in the set of additional training data for other model of Forman. Doing so would help improving prediction accuracy of the predictions generated by the classifiers (Forman, 0014).
As per claim 29, Li, Rosswog and Forman teach the method of claim 28.
Li further teaches
the observations are a learning behavior and performance of the student ML system [paragraphs 0036-0037, “evaluating component 128 evaluates the output distributions of the student and teacher DNNs, determines the difference (which may be determined as an error signal) between the outputs and also determines whether the student is continuing to improve or whether the student is no longer improving … evaluating component 128 determine whether to complete another iteration … evaluating component 128 determines whether to continue iterating based on whether the student is continuing to show improvement … evaluating component 128 apply a threshold to determine convergence of the teacher DNN and student DNN output distributions. Where the threshold is not satisfied, iteration may continue, thereby further training the student to approximate the teacher. Where the threshold is satisfied, then convergence is determined (indicating the student output distribution is sufficiently close enough to the teacher DNN' s output distribution) and the student DNN may be considered trained … evaluating component 128 evaluates the student DNN according to the methods 500 and 600”; paragraph 0005, “The student DNN may be iteratively optimized until its output converges with the output of the teacher DNN. In this way, the student DNN approaches the behavior of the teacher, so that whatever the output of the teacher, the student will approximate”].
Claims 20, 27, 30 and 37 are rejected under 35 U.S.C. 103 as being unpatentable over Li et al. in view of Rosswog et al. in view of Forman et al. and further in view of Paquet et al. (US Pub. 2012/0158620).
As per claim 20, Li, Rosswog and Forman teach the ML computer system of claim 18.
Li, Rosswog and Forman do not teach
an architecture of the reference system is the same as an architecture of the student ML system.
Paquet teaches
an architecture of the reference system is the same as an architecture of the student ML system [paragraph 0023, “an automated classifier 32, such as an artificial neural network (student ML system) … the automated classifier 32 is developed using a training set 34 … (wherein) The training set 34 may be generated … by utilizing a sample content set 12 prepared by another automated classifier 32”; Examiner interprets “the automated classifier 32 is developed using a training set 34” as a student ML system, and the “another automated classifier 32” that generated the training set is the classifier that included in the reference system, and the reference system (comprising another automated classifier 32) has an architecture that is the same as an architecture of the student ML system “an automated classifier 32”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the method for generating a DNN classifier by learning a student DNN model from a larger teacher DNN model of Li to include an architecture of the reference system is the same as an architecture of the student ML system of Paquet. Doing so would help determining the categories of the content items of the training data within an acceptable range (Paquet, 0023).
As per claim 27, Li, Rosswog and Forman teach the ML computer system of claim 18.
Forman teaches
the reference system randomly selects the classifications from the two or more classifiers as the set of additional training data for the student ML system [paragraphs 0014-0019, “the second set receives as input the predictions from the first set of classifiers … the classifiers 130e-130h input the predictions of non-corresponding classifiers from the first set for classifying cases and for training … a maximum score of non-corresponding classifiers in the first set may be used as an additional input attribute for the second set of classifiers … For example, if the classifier 130e receives predictions of 85%, 35%, and 10% from classifiers 130b-d respectively for classifying the same document, the maximum score is 85% for the category for label 2. This maximum score is used as an additional input attribute by the classifier 130e to make its prediction]; and
the probability controlled by the control parameter governs the random selection of the classifications from each of the two or more classifiers [paragraphs 0014-0019, “generate the classifiers 130e-130h in the second set may include parameters that take maximum score into consideration … the classifiers 130e-130h input the predictions of non-corresponding classifiers from the first set for classifying cases and for training … a maximum score of non-corresponding classifiers in the first set may be used as an additional input attribute for the second set of classifiers … For example, if the classifier 130e receives predictions of 85%, 35%, and 10% from classifiers 130b-d respectively for classifying the same document, the maximum score is 85% for the category for label 2. This maximum score is used as an additional input attribute by the classifier 130e to make its prediction; It can be seen that the control parameter which considering the scores of the prediction from the first set of classifiers controls the selection of the predictions (classifications) by determining the maximum score from the set of scores].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the method for generating a DNN classifier by learning a student DNN model from a larger teacher DNN model of Li to include the reference system selecting the classifications from the two or more classifiers as the set of additional training data, and the probability controlled by the control parameter governs the random selection of the classifications from each of the two or more classifiers of Forman. Doing so would help improving prediction accuracy of the predictions generated by the classifiers (Forman, 0014).
Li, Rosswog and Forman do not teach
the control parameter is tunable.
Paquet teaches
wherein the control parameter is tunable [paragraph 0032, “a classification confidence threshold 54 defined by the device 82”; paragraph 0030, “The automated classifier 32 may be invoked to perform a classification 18 of content items 14 of a content set 12 ( such as the three content items 14 identified in this exemplary scenario 50 as "A", "B", and "C"), and each classification 18 may result in an identified association with one or more categories 16, and also a classification confidence 52 (e.g., computed as a probability between 0.00, indicating no confidence, and 1.00, indicating absolute confidence). An embodiment of these techniques may compare the classification confidence 52 of each classification 18 with a classification confidence threshold 54 (e.g., a 0.50 probability); Examiner interprets the classification confidence threshold 54 as a control parameter, and it can be seen that the control parameter can be adjusted; since Li (as modified) teaches a set of classifiers process the same input training set to generate outputs, each comprises a confidence score indicating the processed input belongs to a classification, Li (as modified) also teaches a control parameter which take the classification scores into consideration (Forman, paragraphs 0014-0019), while Paquet teaches the classification scores from the classifier are evaluated using a classification confidence threshold (which examiner interprets as a control parameter), and it can be seen that the classification confidence threshold can be adjusted, therefore, the combination of Li (as modified) and Paquet teaches the above claim limitation].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the method for generating a DNN classifier by learning a student DNN model from a larger teacher DNN model of Li (as modified) to include the control parameter is tunable of Paquet. Doing so would help classifying content items with an acceptable classification confidence and accuracy (Paquet, 0029).
Claim 30 is substantially similar to claim 20 and thus rejected for similar reasons as claim 20.
Claim 37 is substantially similar to claim 27 and thus rejected for similar reasons as claim 27.
Claims 21, 26, 31 and 36 are rejected under 35 U.S.C. 103 as being unpatentable over Li et al. in view of Rosswog et al. in view of Forman et al. and further in view of Aslan et al. (US Pub. 2017/0132528).
As per claim 21, Li, Rosswog and Forman teach the ML computer system of claim 18.
Li, Rosswog and Forman do not teach
the student ML system performs a first set of computations; the reference system performs a second set of computations; and the first set of computations is a superset of the second set of computations.
Aslan teaches
the student ML system performs a first set of computations; the reference system performs a second set of computations; and the first set of computations is a superset of the second set of computations.
[paragraph 0055, “FIG. 5 is a schematic diagram of another example technique for joint training of multiple machine learning models. In the example of FIG. 5, an ensemble of Q teacher models 500, represented in FIG. 5 as models 500(1), 500(2), .... 500(Q) can be trained in parallel with a student model 502 … each of the Q teacher models 500 is shown as receiving a respective portion 504.1, 504.2, ... 504.Q of a large set of training data 504 … In this example, the training data 504 can be too large for any one machine learning model 500 to handle because the training data 504 can be too large (in terms of storage footprint) to store on any single computing device on which the machine learning models are executed. Accordingly, each of the teacher models 500 in the set of Q teacher models can run on a computing device with respective portion 504.1-504.Q of the training data 504 that can be maintained on the computing device. In this manner, the multiple teacher models 504 can enable a student model 502 to learn from a relatively large set of training data 504 indirectly through the passing of information between the student model 502 and each of the teacher models 500”; The above reciting discloses each teacher model processes a small portion of training data during training, while the student model processes a larger portion of training data based on the passing information between the models to influence the training of the teacher models, and it can be seen that processing more data requires performing more computations, therefore, the set of computations performed by the student model is a superset of the set of computations performed by the teacher model; Since Surazhsky in Figs. 1-2 teaches the machine learning system comprising a student machine learning system (neural network engine 140) and other systems including the reference system, while Aslan teaches the student ML system performing a first set of computations that is a superset of a second set of computations performed by other systems, thus, the combination of Surazhsky and Aslan teaches the above claim limitation].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the method for generating a DNN classifier by learning a student DNN model from a larger teacher DNN model of Li to include the student ML system performs a first set of computations, the reference system performs a second set of computations, and the first set of computations is a superset of the second set of computations of Aslan. Doing so would help training the machine learning models such that one of the machine learning models influences the training of the other machine learning model (Aslan, 0005).
As per claim 26, Li, Rosswog and Forman teach the ML computer system of claim 18.
Li, Rosswog and Forman do not teach
the student ML system imitates the reference system to transfer learning from the reference system to the student ML system.
Aslan teaches
the student ML system imitates the reference system to transfer learning from the reference system to the student ML system [paragraph 0044, “the first (teacher) model 100 of FIG. 1 can comprise a large, complex ensemble of machine learning models … the second (student) model 102 can comprise a much smaller machine learning model … the second model 102 can be trained to mimic the much larger first model 100 (through learning how to approximate the function learned by the first model 100) without significant loss in accuracy of the second model's 102 output”; abstract, “During the training of the first machine learning model, information can be passed between the first machine learning model and the second machine learning model. Such passing of information (or "transfer of knowledge") between the machine learning models can be accomplished via the formulation, and optimization, of an objective function”; paragraph 0006, “During the training of the first machine learning model, information can be passed between the first machine learning model and the second machine learning model. Such passing of information (or "transfer of knowledge") between the machine learning models allows for one machine learning model to influence the other while the multiple machine learning models are trained in parallel”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the method for generating a DNN classifier by learning a student DNN model from a larger teacher DNN model of Li to include the student ML system imitates the reference system to transfer learning from the reference system to the student ML system of Aslan. Doing so would allow one machine learning model to influence the other while the multiple machine learning models are trained (Aslan, 0051).
Claim 31 is substantially similar to claim 21 and thus rejected for similar reasons as claim 21.
Claim 36 is substantially similar to claim 26 and thus rejected for similar reasons as claim 26.
Claims 22, 24, 32 and 34 are rejected under 35 U.S.C. 103 as being unpatentable over Li et al. in view of Rosswog et al. in view of Forman et al. in view of Bazrafkan et al. (US Pub. 2018/0211164) and further in view of Chaudhari et al. (US Pub. 2018/0005111).
As per claim 22, Li, Rosswog and Forman teach the ML computer system of claim 18.
Li further teaches
the student ML system comprises a first neural network, such that the student ML system comprises an input layer, an output layer, and at least a first inner layer between the input and output layers [abstract, “a student DNN”; paragraph 0030, “Like the teacher DNN model, the untrained student DNN model includes a number of hidden layers that may be equal to the number of layers of the teacher”; It can be seen that a deep neural network comprises multiple layers (input, hidden and output layers], wherein:
each layer comprises at least one node, such that the first inner layer comprises at least a first node [paragraph 0030, “the student DNN model size, including the number or nodes or parameters for each layer”];
Li, Rosswog and Forman do not teach
the first node of the first inner layer outputs an activation value for each set of input values to the first node of the first inner layer;
the activation value is determined based on an activation function for the first node of the first inner layer and based on learned parameters for the first node of the first inner layer; and
following iterations of the training of the student ML system, the learned parameters for the first node of the first inner layer are updated.
Bazrafkan teaches
the first node of the first inner layer outputs an activation value [paragraph 0020, “training a target network B”; Fig. 4 shows the target network B includes the input, hidden and output layers, Fig. 4 also shows the first node of the inner layer outputs the values indicating a likelihood of the input image being male or female] for each set of input values to the first node of the first inner layer [Fig. 4, paragraph 0051, one output would represent a likelihood of the input image being male, with the other representing a likelihood of the input image being female. In this case, the targets for these outputs could be 1 and 0 with 1 for male, so causing one of the network B output neurons to fire, and with 0 for female, so causing the other neuron to fire; Fig. 1, paragraph 0029, “Network B is designed to perform gender classification and so it produces a single output (LB) indicating the likelihood of an input image containing for example, a male image”; paragraph 0034, “The loss function LB for network B can for example be a categorical cross-entropy”];
the activation value is determined based on learned parameters for the first node of the first inner layer [paragraph 0038, “In the training process, the loss function error back propagates from network B to network A. This tunes network A to generate the best augmentations for network B that can be produced by network A”; Fig. 1, paragraph 0040 discloses the augmented sample data/training data is fed to network B in the training process; paragraph 0029, “Network B is designed to perform gender classification and so it produces a single output (LB) indicating the likelihood of an input image containing for example, a male image”; where the augmented sample data is one of the learned parameters]; and
following iterations of the training of the student ML system, the learned parameters for the first node of the first inner layer are updated [paragraph 0038, “In the training process, the loss function error back propagates from network B to network A. This tunes network A to generate the best augmentations for network B that can be produced by network A”; paragraphs 0053-0054, “when training a neural network, a batch of data X(T) is given to the network and these are used to train instances of network A in parallel and to subsequently train instances of network B in parallel, the instances of network B being fed (at least partially) with augmented samples generated by the instances of network A from processing the samples of batch X(T) … the total loss … is fed back to network A for the subsequent batch … the network parameters are updated based on the loss function(s) … The parameters for network B (including the learned parameters for the first node, weight parameter for example) are updated based on the loss function for network B”];
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the method for generating a DNN classifier by learning a student DNN model from a larger teacher DNN model of Li to include the student ML system comprises multiple layers, the first node of the first inner layer outputs an activation value, the activation value is determined based on learned parameters for the first node of the first inner layer, and following iterations of the training of the student ML system, the learned parameters for the first node of the first inner layer are updated of Bazrafkan. Doing so would help training the neural network based on the updated parameters (Bazrafkan, abstract)
Li, Rosswog, Forman and Bazrafkan do not teach
the activation value is determined based on an activation function for the first node of the first inner layer;
Chaudhari teaches
the activation value is determined based on an activation function for the first node of the first inner layer [paragraph 0014, “Each neuron node in the first hidden layer may be associated with a corresponding activation function … The activation function corresponding to each neuron node in the first hidden layer may receive as inputs the initial scale parameter from the initial input layer, the bias parameter, the set of training inputs, and a set of weights … The activation function may be executed on the second linear combination and the initial scale parameter to generate an activation result … a respective activation result may be determined for each neuron node in the first hidden layer”; Since Bazrafkan in Fig. 4, paragraph 0029 discloses the first node of the inner layer outputs the values indicating a likelihood of the input image being male or female “Network B is designed to perform gender classification and so it produces a single output (LB) indicating the likelihood of an input image containing for example, a male image”. However, Bazrafkan teaches the node outputs an activation value but does not explicitly teach the output is generated using an activation function. Combining Chaudhari is to add into Bazrafkan the missing elements which is “the activation value is determined based on an activation function for the first node”, therefore, the combination of Bazrafkan and Chaudhari read on the claim limitation];
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the method for generating a DNN classifier by learning a student DNN model from a larger teacher DNN model of Li (as modified) to include the process of determining the activation value based on an activation function for the first node of the first inner layer of Chaudhari. Doing so would help generating a set of classifier outputs of the classifier using the activation function and the scale parameter or providing the activation results as input to a next iteration of the training (Chaudhari, 0003).
As per claim 24, Li, Rosswog, Forman, Bazrafkan and Chaudhari teach the ML computer system of claim 22.
Li further teaches
the observations are a learning behavior and performance of the student ML system [paragraphs 0036-0037, “evaluating component 128 evaluates the output distributions of the student and teacher DNNs, determines the difference (which may be determined as an error signal) between the outputs and also determines whether the student is continuing to improve or whether the student is no longer improving … evaluating component 128 determine whether to complete another iteration … evaluating component 128 determines whether to continue iterating based on whether the student is continuing to show improvement … evaluating component 128 apply a threshold to determine convergence of the teacher DNN and student DNN output distributions. Where the threshold is not satisfied, iteration may continue, thereby further training the student to approximate the teacher. Where the threshold is satisfied, then convergence is determined (indicating the student output distribution is sufficiently close enough to the teacher DNN' s output distribution) and the student DNN may be considered trained … evaluating component 128 evaluates the student DNN according to the methods 500 and 600”; paragraph 0005, “The student DNN may be iteratively optimized until its output converges with the output of the teacher DNN. In this way, the student DNN approaches the behavior of the teacher, so that whatever the output of the teacher, the student will approximate”].
Claim 32 is substantially similar to claim 22 and thus rejected for similar reasons as claim 22.
Claim 34 is substantially similar to claim 24 and thus rejected for similar reasons as claim 24.
Claims 23 and 33 are rejected under 35 U.S.C. 103 as being unpatentable over Li et al. in view of Rosswog et al. in view of Forman et al. in view of Bazrafkan et al. in view of Chaudhari et al. and further in view of Paquet et al. (US Pub. 2012/0158620).
As per claim 23, Li, Rosswog, Forman, Bazrafkan and Chaudhari teach the ML computer system of claim 22.
Rosswog, Forman, Bazrafkan and Chaudhari do not teach
the reference system comprises a second neural network such that the first and second neural networks have the same architecture.
Paquet teaches
the reference system comprises a second neural network such that the first and second neural networks have the same architecture [paragraph 0023, “an automated classifier 32, such as an artificial neural network (student ML system) … the automated classifier 32 is developed using a training set 34 … (wherein) The training set 34 may be generated … by utilizing a sample content set 12 prepared by another automated classifier 32”; Examiner interprets “the automated classifier 32 is developed using a training set 34” as a student ML system, and the “another automated classifier 32” that generated the training set is the classifier that included in the reference system, and the reference system (comprising another automated classifier 32) has an architecture that is the same as an architecture of the student ML system “an automated classifier 32”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the method for generating a DNN classifier by learning a student DNN model from a larger teacher DNN model of Li to include the reference system comprises a second neural network such that the first and second neural networks have the same architecture of Paquet. Doing so would help determining the categories of the content items of the training data within an acceptable range (Paquet, 0023).
Claim 33 is substantially similar to claim 23 and thus rejected for similar reasons as claim 23.
Claims 25 and 35 are rejected under 35 U.S.C. 103 as being unpatentable over Li et al. in view of Rosswog et al. in view of Forman et al. in view of Bazrafkan et al. in view of Chaudhari et al. in view of Zoph et al. (US Pub. 2019/0251439) and further in view of Li et al. (US Pub. 2018/0240257).
As per claim 25, Li, Rosswog, Forman, Bazrafkan and Chaudhari teach the ML computer system of claim 22.
Li, Rosswog, Forman, Bazrafkan and Chaudhari do not teach
a learning coach ML system that is in communication with the student ML system, wherein:
the learning coach ML system has been trained through machine learning to determine one or more revised hyperparameter values for the student ML system;
each of the one or more revised hyperparameter values controls an aspect of the machine learning by the student ML system;
the learning coach ML system comprises an input layer and an output layer;
the learning coach ML system determines the one or more revised hyperparameter values for the student ML system based on an input to the learning coach ML system from the student ML system;
the input to the learning coach ML system from the student ML system is input to the input layer of the learning coach ML system;
the input from the student ML system comprises an internal state observation of the student ML system during training of the student ML system;
the internal state observation of the student ML system comprise values related to the learned parameters for the first node on the first inner layer of the student ML system and the activation value for the first node on the first inner layer of the student ML system; and
the learning coach ML system determines the one or more revised hyperparameter values for the student ML system based on the observations about the first node of the first inner layer of the student ML system from training on the set of additional training data generated by the reference system to improve operation of the student ML system.
Zoph teaches
a learning coach ML system that is in communication with the student ML system [abstract, “generating, using a controller neural network, a batch of output sequences, each output sequence in the batch defining a respective architecture of a child neural network that is configured to perform a particular neural network task; for each output sequence in the batch: training a respective instance of the child neural network having the architecture defined by the output sequence”; Examiner interprets the controller neural network as a learning coach ML system, and interprets a child neural network as a student ML system], wherein:
the learning coach ML system has been trained through machine learning to determine one or more revised hyperparameter values for the student ML system [paragraphs 0027-0029, “The controller neural network 110 is a neural network that has parameters, referred to in this specification as "controller parameters," and that is configured to generate output sequences in accordance with the controller parameters. Each output sequence generated by the controller neural network 110 defines a respective possible architecture for the child neural network … each output sequence includes a respective output at each of multiple time steps and each time step in the output sequence corresponds to a different hyperparameter of the architecture of the child neural network. Thus, each output sequence includes, at each time step, a respective value of the corresponding hyperparameter … determines the architecture for the child neural network by training the controller neural network 110 to adjust the values of the controller parameters”];
each of the one or more revised hyperparameter values controls an aspect of the machine learning by the student ML system [paragraphs 0027-0029, “Each output sequence generated by the controller neural network 110 defines a respective possible architecture for the child neural network … each output sequence includes a respective output at each of multiple time steps and each time step in the output sequence corresponds to a different hyperparameter of the architecture of the child neural network. Thus, each output sequence includes, at each time step, a respective value of the corresponding hyperparameter … determines the architecture for the child neural network by training the controller neural network 110 to adjust the values of the controller parameters”];
the learning coach ML system comprises an input layer and an output layer [paragraphs 0038-0039, “The controller neural network 110 is a recurrent neural network that includes one or more recurrent neural network layers, e.g., layers 220 and 230, that are configured to, for each time step, receive as input the value of the hyperparameter corresponding to the preceding time step … The controller neural network 110 also includes a respective output layer for each time step in the output sequence, e.g., output layers 242-254”];
the learning coach ML system determines the one or more revised hyperparameter values for the student ML system based on an input to the learning coach ML system from the student ML system [abstract, “for each output sequence in the batch: training a respective instance of the child neural network having the architecture defined by the output sequence; evaluating a performance of the trained instance of the child neural network on the particular neural network task to determine a performance metric for the trained instance of the child neural network on the particular neural network task; and using the performance metrics for the trained instances of the child neural network to adjust the current values of the controller parameters of the controller neural network”; paragraph 0030, “For each output sequence in the batch 112, the training engine 120 trains an instance of the child neural network that has the architecture defined by the output sequence on the training data 102 and evaluates the performance of the trained instance on the validation set 104. The controller parameter updating engine 130 then uses the results of the evaluations for the output sequences in the batch 112 to update the current values of the controller parameters to improve the expected performance of the architectures defined by the output sequences generated by the controller neural network 110 on the task”];
the input from the student ML system comprises an internal state observation of the student ML system during training of the student ML system [abstract, “for each output sequence in the batch: training a respective instance of the child neural network having the architecture defined by the output sequence; evaluating a performance of the trained instance of the child neural network on the particular neural network task to determine a performance metric for the trained instance of the child neural network on the particular neural network task; and using the performance metrics for the trained instances of the child neural network to adjust the current values of the controller parameters of the controller neural network”];
the internal state observation of the student ML system comprise values related to the learned parameters for the first node on the first inner layer of the student ML system and the activation value for the first node on the first inner layer of the student ML system [paragraphs 0069-0071, “For each output sequence in the batch, the system evaluates the performance of the corresponding trained instance of the child neural network on the particular neural network task to determine a performance metric for the trained instance on the particular neural network task (step 306). For example, the performance metric can be an accuracy of the trained instance on the validation set as measured by an appropriate accuracy measure. For example, the accuracy can be a classification error rate when the task is a classification task. As another example, the performance metric can be an average or a maximum of the accuracies of the instance the instance for each of the last two, five, or ten epochs of the training of the instance … The system uses the performance metrics for the trained instances to adjust the current values of the controller parameters … the system adjusts the current values by training the controller neural network to generate output sequences that result in child neural networks having increased performance metrics using a reinforcement learning technique. More specifically, the system trains the controller neural network to generate output sequences that maximize a received reward that is determined based on the performance metrics of the trained instances. In particular, the reward for a given output sequence is a function of the performance metric for the trained instance”]; and
the learning coach ML system determines the one or more revised hyperparameter values for the student ML system based on the observations about the first node of the first inner layer of the student ML system from training on the set of additional training data generated by the reference system to improve operation of the student ML system [abstract, “for each output sequence in the batch: training a respective instance of the child neural network having the architecture defined by the output sequence; evaluating a performance of the trained instance of the child neural network on the particular neural network task to determine a performance metric for the trained instance of the child neural network on the particular neural network task; and using the performance metrics for the trained instances of the child neural network to adjust the current values of the controller parameters of the controller neural network”; paragraphs 0028-0031, “For each output sequence in the batch 112, the training engine 120 trains an instance of the child neural network that has the architecture defined by the output sequence on the training data 102 and evaluates the performance of the trained instance on the validation set 104. The controller parameter updating engine 130 then uses the results of the evaluations for the output sequences in the batch 112 to update the current values of the controller parameters to improve the expected performance of the architectures defined by the output sequences generated by the controller neural network 110 on the task … each output sequence includes a respective output at each of multiple time steps and each time step in the output sequence corresponds to a different hyperparameter of the architecture of the child neural network. Thus, each output sequence includes, at each time step, a respective value of the corresponding hyperparameter … determines the architecture for the child neural network by training the controller neural network 110 to adjust the values of the controller parameters … repeatedly updating the values of the controller parameters in this manner, the system 100 can train the controller neural network 110 to generate output sequences that result in child neural networks that have increased performance on the particular task”; Since Surazhsky in paragraph 0041 teaches “the training controller 212 may conduct additional training of the convolutional neural network based at least on the performance of the convolutional neural network in processing a mixed training set … The training controller 212 may train the convolutional neural network using additional training data that have been generated by the synthetic image generator 210 and/or the training set generator 216 (reference system) … The training controller 212 may continue to train the convolutional neural network with additional training data until the performance of the convolutional neural network meets a certain threshold value”, while Zoph teaches the controller neural network (the learning coach ML system), based on the performance metrics for the trained instances of the child neural network, adjusts the current values of the controller parameters of the controller neural network to improve the expected performance of the architectures defined by the output sequences (corresponding to a different hyperparameter of the architecture of the child neural network) generated by the controller neural network 110 on the task, and repeatedly updating the values of the controller parameters to increase performance of child neural networks on the particular task, therefore, the combination of Surazhsky and Zoph teaches the claim limitation “the learning coach ML system determines the one or more revised hyperparameter values for the student ML system based on the observations about the first node of the first inner layer of the student ML system from training on the set of additional training data generated by the reference system to improve operation of the student ML system”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the method for generating a DNN classifier by learning a student DNN model from a larger teacher DNN model of Li to include a learning coach ML system determines the one or more revised hyperparameter values for the student ML system based on the observations about the performance of the student ML system of Zoph. Doing so would help increasing the performance metric of the student ML system using the updated hyperparameter values (Zoph, 0071).
Li, Rosswog, Forman, Bazrafkan and Chaudhari and Zoph do not teach
the input to the learning coach ML system from the student ML system is input to the input layer of the learning coach ML system;
Li (2018/0240257) teaches
the input to the learning coach ML system [Fig. 2, the loss neural network] from the student ML system [Fig. 2, the generator neural network] is input to the input layer of the learning coach ML system [Fig. 2 shows output 212 (from the generator neural network/student ML system) which is the input to the loss neural network/learning coach ML is input to the input layer of the loss neural network/learning coach ML; paragraph 0047, “the generator neural network 210 outputs an image, referred to herein as an intermediate output 212, based on the input noise vector. The intermediate output 212 then becomes an input to the loss neural network 240. In turn, the loss neural network 240 outputs style features of the intermediate output 212, referred to herein as intermediate style features 242”; Since Bazrafkan teaches in Fig. 1 that the network A/ learning coach ML system for training a target network B/student ML system, and in paragraph 0036 that output from the network B is used to train the network A, while Li teaches the input from the generator neural network/student ML system to the loss neural network/learning coach ML is input to the input layer of the loss neural network/learning coach ML, therefore, the combination of Bazrafkan and Li read on the claim limitations];
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the method for generating a DNN classifier by learning a student DNN model from a larger teacher DNN model of Li (as modified) to include the input to the learning coach ML system from the student ML system is input to the input layer of the learning coach ML system of Li (2018/0240257). Doing so would help iteratively updating the parameters of the generator neural network to minimize the losses (Li, 0004).
As per claim 35, Li, Rosswog, Forman, Bazrafkan and Chaudhari teach the method of claim 32.
Li, Rosswog, Forman, Bazrafkan and Chaudhari do not teach
following the initial training of the student ML system further comprises:
receiving, by a learning coach ML system of the computer system, from the student ML system, internal state observations of the student ML system as the student ML system is being trained on the set of training data generated by the reference system, wherein:
the learning coach ML system has been trained through machine learning to determine one or more revised hyperparameter values for the student ML system, wherein each of the one or more revised hyperparameter values controls an aspect of the machine learning by the student ML system;
the learning coach ML system comprises an input layer and an output layer;
the learning coach ML system determines the one or more revised hyperparameter values for the student ML system based on input to the learning coach ML system from the student ML system;
the input to the learning coach ML system from the student ML system is input to the input layer of the learning coach ML system;
the input from the student ML system to the learning coach ML system comprises the internal state observations of the student ML system; and
the observations about the internal state of the student ML system comprise values related to the learned parameters and the activation values of the first node of the first inner layer of the student ML system; and
determining by the learning coach ML system, the one or more revised hyperparameter values for the student ML system based on the observations about the first node of the first inner layer of the student ML system from training on the set of additional training data generated by the reference system to improve operation of the student ML system.
Zoph teaches
following the initial training of the student ML system further comprises:
receiving, by a learning coach ML system of the computer system, from the student ML system, internal state observations of the student ML system as the student ML system is being trained on the set of training data generated by the reference system [abstract, “for each output sequence in the batch: training a respective instance of the child neural network having the architecture defined by the output sequence; evaluating a performance of the trained instance of the child neural network on the particular neural network task to determine a performance metric for the trained instance of the child neural network on the particular neural network task; and using the performance metrics for the trained instances of the child neural network to adjust the current values of the controller parameters of the controller neural network”; paragraphs 0028-0031, “For each output sequence in the batch 112, the training engine 120 trains an instance of the child neural network that has the architecture defined by the output sequence on the training data 102 and evaluates the performance of the trained instance on the validation set 104. The controller parameter updating engine 130 then uses the results of the evaluations for the output sequences in the batch 112 to update the current values of the controller parameters to improve the expected performance of the architectures defined by the output sequences generated by the controller neural network 110 on the task; Since Surazhsky in paragraph 0041 teaches the student ML system is being trained on the training data generated by the reference system “the training controller 212 may conduct additional training of the convolutional neural network based at least on the performance of the convolutional neural network in processing a mixed training set … The training controller 212 may train the convolutional neural network using additional training data that have been generated by the synthetic image generator 210 and/or the training set generator 216 (reference system) … The training controller 212 may continue to train the convolutional neural network with additional training data until the performance of the convolutional neural network meets a certain threshold value”, while Zoph teaches the controller neural network (the learning coach ML system), based on the performance metrics for the trained instances of the child neural network, adjusts the current values of the controller parameters of the controller neural network to improve the expected performance of the architectures defined by the output sequences generated by the controller neural network 110 on the task, and repeatedly updating the values of the controller parameters to increase performance of child neural networks on the particular task, therefore, the combination of Surazhsky and Zoph teaches the claim limitation “receiving … internal state observations of the student ML system as the student ML system is being trained on the set of training data generated by the reference system”], wherein:
the learning coach ML system has been trained through machine learning to determine one or more revised hyperparameter values for the student ML system [paragraphs 0027-0029, “The controller neural network 110 is a neural network that has parameters, referred to in this specification as "controller parameters," and that is configured to generate output sequences in accordance with the controller parameters. Each output sequence generated by the controller neural network 110 defines a respective possible architecture for the child neural network … each output sequence includes a respective output at each of multiple time steps and each time step in the output sequence corresponds to a different hyperparameter of the architecture of the child neural network. Thus, each output sequence includes, at each time step, a respective value of the corresponding hyperparameter … determines the architecture for the child neural network by training the controller neural network 110 to adjust the values of the controller parameters”], wherein each of the one or more revised hyperparameter values controls an aspect of the machine learning by the student ML system [paragraphs 0027-0029, “Each output sequence generated by the controller neural network 110 defines a respective possible architecture for the child neural network … each output sequence includes a respective output at each of multiple time steps and each time step in the output sequence corresponds to a different hyperparameter of the architecture of the child neural network. Thus, each output sequence includes, at each time step, a respective value of the corresponding hyperparameter … determines the architecture for the child neural network by training the controller neural network 110 to adjust the values of the controller parameters”];
the learning coach ML system comprises an input layer and an output layer [paragraphs 0038-0039, “The controller neural network 110 is a recurrent neural network that includes one or more recurrent neural network layers, e.g., layers 220 and 230, that are configured to, for each time step, receive as input the value of the hyperparameter corresponding to the preceding time step … The controller neural network 110 also includes a respective output layer for each time step in the output sequence, e.g., output layers 242-254”];
the learning coach ML system determines the one or more revised hyperparameter values for the student ML system based on input to the learning coach ML system from the student ML system [abstract, “for each output sequence in the batch: training a respective instance of the child neural network having the architecture defined by the output sequence; evaluating a performance of the trained instance of the child neural network on the particular neural network task to determine a performance metric for the trained instance of the child neural network on the particular neural network task; and using the performance metrics for the trained instances of the child neural network to adjust the current values of the controller parameters of the controller neural network”; paragraph 0030, “For each output sequence in the batch 112, the training engine 120 trains an instance of the child neural network that has the architecture defined by the output sequence on the training data 102 and evaluates the performance of the trained instance on the validation set 104. The controller parameter updating engine 130 then uses the results of the evaluations for the output sequences in the batch 112 to update the current values of the controller parameters to improve the expected performance of the architectures defined by the output sequences generated by the controller neural network 110 on the task”];
the input from the student ML system to the learning coach ML system comprises the internal state observations of the student ML system [abstract, “for each output sequence in the batch: training a respective instance of the child neural network having the architecture defined by the output sequence; evaluating a performance of the trained instance of the child neural network on the particular neural network task to determine a performance metric for the trained instance of the child neural network on the particular neural network task; and using the performance metrics for the trained instances of the child neural network to adjust the current values of the controller parameters of the controller neural network”]; and
the observations about the internal state of the student ML system comprise values related to the learned parameters and the activation values of the first node of the first inner layer of the student ML system [paragraphs 0069-0071, “For each output sequence in the batch, the system evaluates the performance of the corresponding trained instance of the child neural network on the particular neural network task to determine a performance metric for the trained instance on the particular neural network task (step 306). For example, the performance metric can be an accuracy of the trained instance on the validation set as measured by an appropriate accuracy measure. For example, the accuracy can be a classification error rate when the task is a classification task. As another example, the performance metric can be an average or a maximum of the accuracies of the instance the instance for each of the last two, five, or ten epochs of the training of the instance … The system uses the performance metrics for the trained instances to adjust the current values of the controller parameters … the system adjusts the current values by training the controller neural network to generate output sequences that result in child neural networks having increased performance metrics using a reinforcement learning technique. More specifically, the system trains the controller neural network to generate output sequences that maximize a received reward that is determined based on the performance metrics of the trained instances. In particular, the reward for a given output sequence is a function of the performance metric for the trained instance”]; and
determining by the learning coach ML system, the one or more revised hyperparameter values for the student ML system based on the observations about the first node of the first inner layer of the student ML system from training on the set of additional training data generated by the reference system to improve operation of the student ML system [abstract, “for each output sequence in the batch: training a respective instance of the child neural network having the architecture defined by the output sequence; evaluating a performance of the trained instance of the child neural network on the particular neural network task to determine a performance metric for the trained instance of the child neural network on the particular neural network task; and using the performance metrics for the trained instances of the child neural network to adjust the current values of the controller parameters of the controller neural network”; paragraphs 0028-0031, “For each output sequence in the batch 112, the training engine 120 trains an instance of the child neural network that has the architecture defined by the output sequence on the training data 102 and evaluates the performance of the trained instance on the validation set 104. The controller parameter updating engine 130 then uses the results of the evaluations for the output sequences in the batch 112 to update the current values of the controller parameters to improve the expected performance of the architectures defined by the output sequences generated by the controller neural network 110 on the task … each output sequence includes a respective output at each of multiple time steps and each time step in the output sequence corresponds to a different hyperparameter of the architecture of the child neural network. Thus, each output sequence includes, at each time step, a respective value of the corresponding hyperparameter … determines the architecture for the child neural network by training the controller neural network 110 to adjust the values of the controller parameters … repeatedly updating the values of the controller parameters in this manner, the system 100 can train the controller neural network 110 to generate output sequences that result in child neural networks that have increased performance on the particular task”; Since Surazhsky in paragraph 0041 teaches “the training controller 212 may conduct additional training of the convolutional neural network based at least on the performance of the convolutional neural network in processing a mixed training set … The training controller 212 may train the convolutional neural network using additional training data that have been generated by the synthetic image generator 210 and/or the training set generator 216 (reference system) … The training controller 212 may continue to train the convolutional neural network with additional training data until the performance of the convolutional neural network meets a certain threshold value”, while Zoph teaches the controller neural network (the learning coach ML system), based on the performance metrics for the trained instances of the child neural network, adjusts the current values of the controller parameters of the controller neural network to improve the expected performance of the architectures defined by the output sequences (corresponding to a different hyperparameter of the architecture of the child neural network) generated by the controller neural network 110 on the task, and repeatedly updating the values of the controller parameters to increase performance of child neural networks on the particular task, therefore, the combination of Surazhsky and Zoph teaches the claim limitation “the learning coach ML system determines the one or more revised hyperparameter values for the student ML system based on the observations about the first node of the first inner layer of the student ML system from training on the set of additional training data generated by the reference system to improve operation of the student ML system”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the method of training a machine learning model of Surazhsky to include a learning coach ML system determines the one or more revised hyperparameter values for the student ML system based on the observations about the performance of the student ML system of Zoph. Doing so would help increasing the performance metric of the student ML system using the updated hyperparameter values (Zoph, 0071).
Li, Rosswog, Forman, Bazrafkan and Chaudhari and Zoph do not teach
the input to the learning coach ML system from the student ML system is input to the input layer of the learning coach ML system;
Li (2018/0240257) teaches
the input to the learning coach ML system [Fig. 2, the loss neural network] from the student ML system [Fig. 2, the generator neural network] is input to the input layer of the learning coach ML system [Fig. 2 shows output 212 (from the generator neural network/student ML system) which is the input to the loss neural network/learning coach ML is input to the input layer of the loss neural network/learning coach ML; paragraph 0047, “the generator neural network 210 outputs an image, referred to herein as an intermediate output 212, based on the input noise vector. The intermediate output 212 then becomes an input to the loss neural network 240. In turn, the loss neural network 240 outputs style features of the intermediate output 212, referred to herein as intermediate style features 242”; Since Bazrafkan teaches in Fig. 1 that the network A/ learning coach ML system for training a target network B/student ML system, and in paragraph 0036 that output from the network B is used to train the network A, while Li teaches the input from the generator neural network/student ML system to the loss neural network/learning coach ML is input to the input layer of the loss neural network/learning coach ML, therefore, the combination of Bazrafkan and Li read on the claim limitations];
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the method of training a neural network of Bazrafkan to include the input to the learning coach ML system from the student ML system is input to the input layer of the learning coach ML system of Li. Doing so would help iteratively updating the parameters of the generator neural network to minimize the losses (Li, 0004).
Prior Art
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
Kang et al. (US Pub. 2017/0083829) describes a method of selecting one or more teacher models to train a student model.
Lin et al. (US Patent 8,706,656) describes a method of using a plurality of classifiers for multi-label modelling, wherein, the predictive outputs from a set of classifiers are used to train other set of classifiers.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TRI T NGUYEN whose telephone number is 571-272-0103. The examiner can normally be reached M-F, 8 AM-5 PM, (CT).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, OMAR FERNANDEZ can be reached at 571-272-2589. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TRI T NGUYEN/Examiner, Art Unit 2128
/OMAR F FERNANDEZ RIVAS/Supervisory Patent Examiner, Art Unit 2128
