DETAILED ACTION
This action is responsive to the following communications: Original Application filed on February 23, 2024, and the Preliminary Amendment filed on March 6, 2024. All references to this application refer to the U.S. Patent Application Publication No. 2024/0202535 A1.
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-20 are pending in this case. Claims 1-4, 6-12, and 14-16 were amended, and claims 17-20 were added via the Preliminary Amendment. Claims 1, 9, and 17 are the independent claims. Claims 1-20 are rejected.
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55. Applicants have perfected priority to PCT/CN2022/111734, filed on August 11, 2022, and Chinese Patent Application No. 202110977567.4, filed on August 24, 2021.
Drawings
The drawings are objected to for the following informalities.
The Specification recites additional sub-steps for the process of Fig. 5 (S5062 and S5064, in paragraphs 0137 and 0139, respectively). Therefore, the drawings fail to comply with 37 CFR 1.84(p)(5) because they do not include these reference signs mentioned in the description.
This object can be overcome in two ways: either amending Fig. 5 to include the listed sub-steps, or removing the content from the written description.
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the Examiner, the Applicants will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
INFORMATION ON HOW TO EFFECT DRAWING CHANGES
Replacement Drawing Sheets
Drawing changes must be made by presenting replacement sheets which incorporate the desired changes and which comply with 37 CFR 1.84. An explanation of the changes made must be presented either in the drawing amendments section, or remarks, section of the amendment paper. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). A replacement sheet must include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of the amended drawing(s) must not be labeled as “amended.” If the changes to the drawing figure(s) are not accepted by the Examiner, Applicants will be notified of any required corrective action in the next Office action. No further drawing submission will be required, unless Applicants are notified.
Identifying indicia, if provided, should include the title of the invention, inventor’s name, and application number, or docket number (if any) if an application number has not been assigned to the application. If this information is provided, it must be placed on the front of each sheet and within the top margin.
Annotated Drawing Sheets
A marked-up copy of any amended drawing figure, including annotations indicating the changes made, may be submitted or required by the Examiner. The annotated drawing sheet(s) must be clearly labeled as “Annotated Sheet” and must be presented in the amendment or remarks section that explains the change(s) to the drawings.
Timing of Corrections
Applicants are required to submit acceptable corrected drawings within the time period set in the Office action. See 37 CFR 1.85(a). Failure to take corrective action within the set period will result in ABANDONMENT of the application.
If corrected drawings are required in a Notice of Allowability (PTOL-37), the new drawings MUST be filed within the THREE MONTH shortened statutory period set for reply in the “Notice of Allowability.” Extensions of time may NOT be obtained under the provisions of 37 CFR 1.136 for filing the corrected drawings after the mailing of a Notice of Allowability.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
The following title is suggested: “Training Heterogeneous Machine Learning Models using Knowledge Distillation”
The disclosure is objected to because of the following informalities:
As indicated above, paragraphs 0137 and 0139 contain sub-steps that do not appear in the figures. This objection can be overcome by amending the figure (as described above), or by removing the sub-steps from the written description.
Appropriate correction is required.
Claim Objections
Claims 5-7 are objected to because of the following informalities:
The preambles of claims 5-7 recite “The method according to any one of claim…” This is an artifact from the multiple dependent claims from the PCT claimset. Each of the preambles to claims 5-7 should be amended to recite “The method according to claim…”
Appropriate corrections are required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
With regard to claim 1,
Step 2A, Prong 1
This part of the eligibility analysis evaluates whether the claim recites a judicial exception. As explained in MPEP 2106.04, subsection II, a claim “recites” a judicial exception when the judicial exception is “set forth” or “described” in the claim.
Claim 1 recites:
An artificial intelligence (AI) model training method, wherein the method comprises:
Determining a first model and a second model to be trained, wherein the first model and the second model are two heterogeneous Al models;
Inputting training data into the first model and the second model, to obtain a first output by performing inference on the training data by the first model and a second output by performing inference on the training data by the second model;
Iteratively updating a model parameter of the first model by using the second output as a supervision signal of the first model and with reference to the first output, until the first model satisfies a first preset condition.
The broadest reasonable interpretation of the bolded limitations above are directed to mental process able to be performed in the human mind or by a human using pen and paper and mathematical calculations.
A human can determine a determine first and second models to be trained, and determine when the first model satisfies a preset condition. A human can perform those tasks mentally or with pen and paper. See MPEP 2016.04(a)(2)(III)(B)-(C).
Additionally, performing inference(s) to obtain first and second output from the first and second models, and iteratively updating a model parameter of the first model using the second output as a supervision signal constitute the performance of mathematical calculations. See MPEP 2106.04(a)(2)(I)(C). Specifically, the recitation is equivalent to Example 47, claim 2 (ineligible)1. In that example, the recitation of training the artificial neural network (ANN) using backpropagation and gradient descent algorithm constituted mathematical calculations. Here, the iterative updating of a model parameter using the second output as a supervision signal is incremental learning using an algorithm, and therefore a mathematical calculation.
Step 2A, Prong 1 (Yes).
Step 2A, Prong 2
This part of the eligibility analysis evaluates whether the claim as a whole integrates the recited judicial exception into a practical application of the exception or whether the claim is “directed to” the judicial exception. This evaluation is performed by (1) identifying whether there are any additional elements recited in the claim beyond the judicial exception, and (2) evaluating those additional elements individually and in combination to determine whether the claim as a whole integrates the exception into a practical application. See MPEP 2106.04(d).
The additional elements in this claim are the first and second models (which are heterogeneous AI models). These elements are recited at a high level of generality and thus is a generic computer component performing computer functions. Thus these are mere instructions to apply the exception using generic computer components. See MPEP 2106.04(f). Additionally, the claims recite inputting training data into the first and second models, which merely using a computer as a tool to perform an abstract idea.
Even when viewed in combination the additional element does not integrate the recited judicial exception into a practical application.
Step 2A, Prong 2 (Yes).
Step 2B
This part of the eligibility analysis evaluates whether the claim as a whole amounts to significantly more than the recited exception i.e., whether any additional element, or combination of additional elements, adds an inventive concept to the claim. See MPEP 2106.05.
As explained with respect to Step 2A, the additional elements are the first and second heterogeneous AI models, and inputting training data into those models. The models are not described with specificity. See MPEP 2106.05(I)(A). As for the inputting of training data, the courts have recognized that as a well-understood, routine, and convention function that does not integrate the abstract idea into a practical application. See MPEP 2106.05(d)(II).
Step 2B (Yes).
Claim 1 is ineligible.
With respect to independent claims 9 and 17,
These claims are similar in scope to Claim 1 and are rejected under a similar rationale. The computing device cluster comprising at least one computing device comprising processors and memory (claim 9) and the non-transitory computer-readable medium (claim 17) are also generic computing components.
Claims 9 and 17 are ineligible.
Dependent Claims:
Claims 2-8, 10-16, and 18-20: These claims only recite further abstract ideas (mental processes and mathematical calculations) and thus are ineligible.
To expedite a complete examination of the instant application, the claims rejected above under 35 U.S.C. 101, as relating to judicial exceptions without significantly more, are further rejected as set forth below in anticipation of amendments to these claims to place them within the four statutory categories of invention.
Examiner’s Note
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the Examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicants are advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the Examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-6, 9-14, and 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over Non-Patent Literature reference entitled “Transformer to CNN: Label-scarce distillation for efficient text classification,” by Chia et al., published on September 8, 2019 (hereinafter Chia), in view of U.S. Patent Application Publication No. 2020/0272905 A1, filed by Saripalli et al., on June 24, 2019, and published on August 27, 2020 (hereinafter Saripalli).
With respect to independent claim 1, Chia discloses an artificial intelligence (AI) model training method, wherein the method comprises:
Determining a first model and a second model to be trained, wherein the first model and the second model are two heterogeneous Al models; Chia discloses determining first and second heterogeneous AI models to be trained (see Chia, section 2 [OpenAI Transformer is selected as the teacher model and three different CNN model architectures were tested as student models (Chia focuses on the BlendCNN embodiment)].
Inputting training data into the first model and the second model, to obtain a first output by performing inference on the training data by the first model and a second output by performing inference on the training data by the second model; Chia discloses inputting training data into both models to obtain first and second outputs from the models by performing inferences (see Chia, Fig. 1; see also, Chia, Section 3 [each model was trained and tested on standard datasets]; see also, Chia, Table 2 [describing the scores/results for the experiments]; see also, Chai, Section 2, described supra).
Iteratively updating a model parameter of the first model by using the second output as a supervision signal of the first model and with reference to the first output…, Chia discloses iteratively updating a model parameter of the first model using the second output as a supervision signal (see Chia, Section 4 [describing the results of the trained model, and its performance]; see also, Chia, Sections 2 and 3, described supra; see also, Chia, Table 2, described supra).
Chia fails to expressly disclose iteratively updating …until the first model satisfies a first preset condition.
However, Saripalli teaches training a model until the model converges (e.g., a preset condition is satisfied) (see Saripalli, Fig. 12; see also, Saripalli, paragraphs 0029 [generally describing teacher-student distillation, network architecture, reward calculations/prediction, and convergence], 0031 [providing more detail about when and how the reward is calculated or predicted], and 0065 [describing the algorithm of Fig. 12]).
Accordingly, it would have been obvious to one of ordinary skill in the art, having the teachings of Chia and Saripalli before him before the effective filing date of the claimed invention, to modify the method of Chia to incorporate convergence criteria as taught by Saripalli. One would have been motivated to make such a combination because convergence conditions indicate that the optimal outcome has been achieved or approximated, as taught by Saripalli (see Saripalli, paragraph 0010 [“The RL agent component can, in some cases, update (e.g., via policy gradient methods) the compression policy until an optimal compression policy is substantially approximated (e.g., convergence).”]).
With respect to dependent claim 2, Chia, as modified by Saripalli, teaches the method according to claim 1, as described above.
Chia and Saripalli further teaches the method wherein the method further comprises: iteratively updating a model parameter of the second model by using the first output as a supervision signal of the second model and with reference to the second output, until the second model satisfies a second preset condition.
Chia further teaches iteratively updating model parameters of the second model using the first output as a supervision signal (see Chia, Sections 2-4, described supra, claim 1; see also, Chia, Table 2, described supra, claim 1).
Additionally, Saripalli further teaches determining when a model has converged (see Saripalli, Fig. 12; see also, Saripalli, paragraphs 0029, 0031, and 0065, described supra, claim 1).
With respect to dependent claim 3, Chia, as modified by Saripalli, teaches the method according to claim 1, as described above.
Chia and Saripalli further teach the method
Wherein the first output comprises at least one of a first feature extracted by the first model from the training data and a first probability distribution inferred based on the first feature, and the second output comprises at least one of a second feature extracted by the second model from the training data and a second probability distribution inferred based on the second feature; Chia further teaches the first output comprising an extracted feature of the training data by the first model and a first probability distribution inferred by the feature, and the second output comprising an extracted feature of the training data by the second model and a second probability distribution inferred by the feature (see Chia, Sections 2-4, described supra, claim 1; see also, Chia, Table 2, described supra, claim 1).
Wherein the iteratively updating a model parameter of the first model by using the second output as a supervision signal of the first model and with reference to the first output comprises:
Determining a first contrastive loss based on the first feature and the second feature and determining a first relative entropy loss based on the first probability distribution and the second probability distribution; Saripalli further teaches determining contrastive losses based on the first and second feature and a first entropy loss based on the distribution probabilities (see Saripalli, paragraphs 0037 [models are trained using cross-entropy loss and distillation loss], 0038 [each model comprises a distribution], and 0055 [describing how one of ordinary skill recognizes that other loss functions could be used]).
Iteratively updating the model parameter of the first model based on at least one of the first contrastive loss and the first relative entropy loss; Saripalli further teaches iteratively updating the model parameter of the first model using at least the entropy or distillation loss (see Saripalli, paragraphs 0037, 0038, and 0055, described supra).
With respect to dependent claim 4, Chia, as modified by Saripalli, teaches the method according to claim 3, as described above.
Saripalli further teaches the method wherein the iteratively updating the model parameter of the first model based on at least one of the first contrastive loss and the first relative entropy loss comprises:
Iteratively updating the model parameter of the first model based on a gradient of the first contrastive loss and a gradient of the first relative entropy loss; Saripalli further teaches iteratively updating the model parameter based on a gradient of the loss functions (see Saripalli, paragraph 0052 [describing how the loss gradients are used to iteratively update the model parameters]).
In response to determining that a difference between a supervised loss of the first model and a supervised loss of the second model is less than a first preset threshold, stopping iteratively updating the model parameter of the first model based on the gradient of the first contrastive loss; Saripalli further teaches performing the iterative updates until the convergence criteria are satisfied (see Saripalli, paragraphs 0029, 0031, and 0065, described supra, claim 1).
With respect to dependent claim 5, Chia, as modified by Saripalli, teaches the method according to [] claim 1, as described above.
Chia further teaches the method, wherein the first model is a transformer model, and the second model is a convolutional neural network model.
Chia further teaches the first model is a transformer model and the second is a CNN (see Chia, Fig. 1; see also, Chia, Section 2, described supra, claim 1).
With respect to dependent claim 6, Chia, as modified by Saripalli, teaches the method according to [] claim 1, as described above.
Saripalli further teaches the method wherein the determining a first model and a second model to be trained comprises: determining the first model and the second model based on a selection made via a user interface or a type of an AI task.
Saripalli further teaches determining the models based on the type of AI task (see Saripalli, paragraph 0028 [describing various use cases for which AI models are developed]).
Independent claim 9, and its respective dependent claims 10-14, recite a computing device cluster comprising at least one computing device, the at least one computing device comprises at least one processor and at least one memory, the at least one memory is coupled to the at least one processor and stores instructions for execution by the at least one processor to execute the instructions to enable the computing device cluster to perform operations comprising the method of independent claim 1, and its respective dependent claims 2-6. Accordingly, independent claim 9, and its respective dependent claims 10-14, are rejected under the same rationales used to reject independent claim 1, and its respective dependent claims 2-6, which are incorporated herein.
Independent claim 17, and its respective dependent claims 18-20, recite a non-transitory, computer-readable medium storing one or more instructions executable by at least one processor to perform operations comprising the method of independent claim 1, and its respective dependent claims 2-4. Accordingly, independent claim 17, and its respective dependent claims 18-20, are rejected under the same rationales used to reject independent claim 1, and its respective dependent claims 2-4, which are incorporated herein.
Claims 7, 8, 15, and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Chia, in view of Saripalli, further in view of U.S. Patent Application Publication No. 2020/0097847 A1, filed by Convertino et al., on September 21, 2018, and published on March 20, 2020 (hereinafter Convertino).
With respect to dependent claim 7, Chia, as modified by Saripalli, teaches the method according to [] claim 6, as described above.
Chia and Saripalli fail to further teach the method wherein the method further comprises: Receiving a training parameter configured by the user via the user interface.
However, Convertino teaches a graphical user interface that allows a user to select or otherwise enter hyperparameters, parameters, policies, metrics, etc., associated with training ML models (see Convertino, Figs. 17-28; see also, Convertino, paragraphs 0003 [defining hyperparameter and metric tuning], 0042 [describing steps 104 and 106 of Fig. 1, in which the model and relevant features of the model are selected and defined, based on the task to be achieved], 0046 [describing step 2020, in which the user can tune hyperparameters through the GUI], 0072 [describing the platform as a web based interface or a desktop GUI], 0094 [describing step 1002 of Fig. 10, in which users can select and set values of the hyperparameters through text boxes, sliders, knob, and other GUI controls], and 0123-0129 [describing the GUIs of Figs. 17-19, which present the tuning dashboard, hyperparameter panel, and performance metric panel, including types of settable parameters, metrics, and values]).
Accordingly, it would have been obvious to one of ordinary skill in the art, having the teachings of Chia, Saripalli, and Convertino before him before the effective filing date of the claimed invention, to modify the method of Chia, as modified by Saripalli, to incorporate GUI as taught by Convertino, in order to permit user tuning/entry of parameters and values. One would have been motivated to make such a combination because this allow users to have greater control over training models to perform tasks, as taught by Convertino (see Convertino, paragraph 0002 [“As the demand for application-specific ML models increases, tools to enable users to efficiently and confidently build ML models have become increasingly important.”]).
Chia, as modified by Saripalli and Convertino, further teaches determining the training parameter based on the type of the AI task, the first model, and the second model.
Convertino further teaches determining the training parameter based on the task and model type (see Convertino, Figs. 17-28; see also, Convertino, paragraphs 0003, 0046, 0072, 0094, and 0123-0129, described supra).
With respect to dependent claim 8, Chia, as modified by Saripalli and Convertino, teaches the method according to claim 7, as described above.
Convertino further teaches the method wherein the training parameter comprises one or more of: a training round, an optimizer type, a learning rate update policy, a model parameter initialization manner, or a training policy.
Convertino further teaches the training parameter comprising at least batch size, number of epoch, dropout rate, learning rate, accuracy, loss, etc. (see Convertino, Figs. 17-19; see also, Convertino, paragraphs 0003, 0046, 0072, 0094, and 0123-0129, described supra, claim 7).
Dependent claims 15 and 16 recite a computing device cluster comprising at least one computing device, the at least one computing device comprises at least one processor and at least one memory, the at least one memory is coupled to the at least one processor and stores instructions for execution by the at least one processor to execute the instructions to enable the computing device cluster to perform operations comprising the method of dependent claims 7 and 8. Accordingly, dependent claims 15 and 16 are rejected under the same rationales used to reject dependent claims 7 and 8, which are incorporated herein.
Conclusion
The prior art made of record and not relied upon is considered pertinent to Applicants’ disclosure. See PTO-892.
It is noted that any citation to specific pages, columns, figures, or lines in the prior art references any interpretation of the references should not be considered to be limiting in any way. A reference is relevant for all it contains and may be relied upon for all that it would have reasonably suggested to one having ordinary skill in the art. In re Heck, 699 F.2d 1331-33, 216 USPQ 1038-39 (Fed. Cir. 1983) (quoting In re Lemelson, 397 F.2d 1006, 1009, 158 USPQ 275, 277 (CCPA 1968)).
Any inquiry concerning this communication or earlier communications from the Examiner should be directed to ERIC J. BYCER whose telephone number is (571) 270-3741. The Examiner can normally be reached Monday - Thursday 9am-6pm, and alternate Fridays 9am-5pm.
Examiner interviews are available via a variety of formats. See MPEP § 713.01. To schedule an interview, Applicants are encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/InterviewPractice.
If attempts to reach the Examiner by telephone are unsuccessful, the Examiner’s supervisor, MATT ELL can be reached on (571) 270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center. Status information for published applications may be obtained from Patent Center. Status information for unpublished applications is available through Patent Center to authorized users only. Should you have questions about access to the USPTO patent electronic filing system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
/ERIC J. BYCER/
Primary Examiner
Art Unit 2141
1 July 2024 Subject Matter Eligibility Examples, available at https://www.uspto.gov/sites/default/files/documents/2024-AI-SMEUpdateExamples47-49.pdf
Prosecution Insights