DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims 1-20 are pending of which claims 1, 10, and 16 are in independent form.
Claims 5 and 14 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph.
Claims 1-20 are rejected under 35 U.S.C. 103.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 5 and 14 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claims 5 and 14, these claims depend on claims 2 and 11 (respectively), directly or indirectly incorporating claims 2/11 and 1/10, but recites increasing the number of third resources “based on the third to fifth throughput information”. Claims 1/10, 2/11 introduce first and second throughput information; neither introduce or identify third or fourth throughput information. Those terms have been introduced in claims 3 and 12, which claims 5 and 14 don’t depend on. Consequently the scope of claims 5 and 14 is unclear because it cannot be determined from those claims what the referenced third and fourth throughput information represents.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 2, 10, 11, 16 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over “Large-scale knowledge distillation with elastic heterogeneous computing resources” Ji Liu, July 14, 2022 [Liu], in view of HONG; Hyesun et al. (US 20220121912 A1) [Hong].
Regarding claims 1, 10, and 16, Liu discloses, an electronic device training a second model by using an inference result of a first model (the architecture of EDL-Dist, which consists of a student module, a teacher module and a coordinator module. This architecture decouples the training process in the student module and the inference process in the teacher module, which can accelerate the training process in the student module with elastic heterogeneous resources in the teacher modules. … dynamic hybrid resource scheduling method, the design of fault-tolerance and the EDL-Dist algorithm. The dynamic hybrid resource scheduling method can choose appropriate teacher GPU cards for each student GPU card while avoiding the risk to store many unused soft labels (see details in Section 2). The design of fault-tolerance enables knowledge distillation with the consideration of the failures of teacher GPU cards during the training process. In addition, the design exploits a checkpoint mechanism to continue the training process when there are failures or exceptions in the student module. The EDL-Dist algorithm enables the training process with student GPU cards distributed in multiple servers while exploiting the soft labels from the teacher module [pages 2-3], also see [sec. 3.1, pages 5-6], [sec. 3.3, pages 7-8], algorithms 2, page 8 ), when executed by the at least one processor cause the electronic device to: acquire first throughput information for an inference operation of the first model and second throughput information for a training operation of the second model (historical throughput of inference in a teacher server, expressed as input data processed per unit time and training a student server [See sec. 3.2, pages 6-7 and more specifically Algorithm 1]),
a first resource to perform the inference operation of the first model and a second resource to perform the training operation of the second model (identifies dedicated student training resources [sec. 3.1, pages 5-6]);
allocate the identified first resource as a resource to perform the inference operation of the first model (deploys the teaches on inference resources and schedules teacher resources to supply soft labels to the student [sec. 3.1, pages 5-6], [sec. 3.2, Algorithm 2]), and allocate the identified second resource as a resource to perform the training operation of the second model (dedicated computing resources to student training ¶ [sec, 3.1, page 5]), and
train, based on the allocated first and second resources, the second model by using the inference result of the first model (student receives teacher generated soft labels and using them to update its parameters [sec. 3.1, 5-6], [sec. 3.3, pages 7-8, algorithm 2, steps 3-5]).
However Liu does not explicitly facilitate the electronic device comprising: a memory storing information for the first model and second model and one or more instructions; and at least one processor operatively coupled to the memory and configured to execute the one or more instructions stored in the memory, wherein the one or more instructions; identify, based on at least the first throughput information and the second throughput information.
Hong discloses, the electronic device comprising: a memory storing information for the first model and second model and one or more instructions; and at least one processor operatively coupled to the memory and configured to execute the one or more instructions stored in the memory, wherein the one or more instructions (see Fig. 1, ¶ [0047]-[0054]).
identify, based on at least the first throughput information and the second throughput information (In operation 440, the scheduler may select a single candidate kernel estimated to have the best performance from among a plurality of candidate kernels for the selected layer. The scheduler may select a candidate kernel estimated to have the best accelerator performance when running each candidate kernel based on a current situation of the accelerator and kernel information of candidate kernels. A criterion of performance may include any one or any combination of a throughput of each model, a latency, a fairness, a power consumption amount, and a utilization rate of the accelerator based on a situation or a selection of a user. In the example illustrated on the right of FIG. 4, a single candidate kernel may be selected from among the plurality of candidate kernels for layer 2 of selected model 1 ¶ [0077]. Through this, in a run time, the scheduler of one or more embodiments may minimize contention between models and may maximize a resource utilization rate of the accelerator by adaptively changing a number of cores used to run each layer based on a resource situation of the accelerator. For example, scheduling may be performed to quickly complete a specific model by increasing the overall system throughput through the overall system performance prioritization scheme in a general situation in which specific requirements are absent, and by selectively applying the greedy scheme to the specific model in response to an urgent request for executing the specific model ¶ [0083]).
It would have been obvious, to one ordinary skilled in the art, before the effective filing date of the claimed invention to combine the teachings of the cited references because Hong's system would have allowed Liu to explicitly facilitate the electronic device comprising: a memory storing information for the first model and second model and one or more instructions; and at least one processor operatively coupled to the memory and configured to execute the one or more instructions stored in the memory, wherein the one or more instructions; identify, based on at least the first throughput information and the second throughput information. The motivation to combine is apparent in the Liu’s reference, because a need to improve generating the plurality of candidate kernels to run each layer using a different accelerator resource use amount.
Regarding claims 2, 11, and 17, the combination of Liu and Hong discloses, identify information for a plurality of resource allocation combinations, acquire, for each of the plurality of resource allocation combinations, the first throughput information for the inference operation of the first model and the second throughput information for the training operation of the second model, identify a resource allocation combination having a highest throughput among the plurality of resource allocation combinations based on the first throughput information and the second throughput information for each of the plurality of resource allocation combinations, and identify the first resource and the second resource based on the identified resource allocation combination (Hong: In operation 440, the scheduler may select a single candidate kernel estimated to have the best performance from among a plurality of candidate kernels for the selected layer. The scheduler may select a candidate kernel estimated to have the best accelerator performance when running each candidate kernel based on a current situation of the accelerator and kernel information of candidate kernels. A criterion of performance may include any one or any combination of a throughput of each model, a latency, a fairness, a power consumption amount, and a utilization rate of the accelerator based on a situation or a selection of a user. In the example illustrated on the right of FIG. 4, a single candidate kernel may be selected from among the plurality of candidate kernels for layer 2 of selected model 1 ¶ [0077]. Through this, in a run time, the scheduler of one or more embodiments may minimize contention between models and may maximize a resource utilization rate of the accelerator by adaptively changing a number of cores used to run each layer based on a resource situation of the accelerator. For example, scheduling may be performed to quickly complete a specific model by increasing the overall system throughput through the overall system performance prioritization scheme in a general situation in which specific requirements are absent, and by selectively applying the greedy scheme to the specific model in response to an urgent request for executing the specific model ¶ [0083]).
Claim(s) 3-9, 12-15, 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over Liu, in view of Hong in view of Tan; Mingxing et al. (US 20220245928 A1) [Tan].
Regarding claims 3, 12, and 18, the combination of Liu and Hong discloses, acquire, while training the second model , third throughput information for the inference operation of the first model and fourth throughput information for the training operation of the second model , and re-identify at least one of the first resource to perform the inference operation of the first model or the second resource to perform the training operation of the second model based on a determination that, based on the third and fourth throughput information (Liu: monitoring the supply of teacher generated soft labels during distillation and adds an available teacher resources when the student runs out [sec. 3.2, pp. 6-7 and Algorithm 1, lines 3-12]).
However, neither Liu nor Hong explicitly facilitate an inference speed of the first model is slower than a training speed of the second model by a critical speed or more.
Tan discloses, an inference speed of the first model is slower than a training speed of the second model by a critical speed or more (FIG. 3 depicts a graphical diagram 300 of an example neural architecture search approach to emphasize training speed and accuracy according to example embodiments of the present disclosure … ¶ [0095], [0098], [0099]).
It would have been obvious, to one ordinary skilled in the art, before the effective filing date of the claimed invention to combine the teachings of the cited references because Tan's system would have allowed Liu and Hong to explicitly facilitate an inference speed of the first model is slower than a training speed of the second model by a critical speed or more. The motivation to combine is apparent in the Liu and Hong’s reference, because a need to optimize training speed and parameter efficiency, which would substantially increase the availability of computational resources for other tasks.
Regarding claims 4, 13, and 19, the combination of Liu, Hong, and Tan discloses, based on the determination that the [inference speed of the first model is slower than the training speed of the second model by the critical speed or more], determine whether an increase condition to increase the number of the first resources to improve a processing speed of the inference operation of the first model is satisfied, and based on a determination the increase condition is satisfied, re-identify the first resource by increasing the number of the first resources, and based on a determination the increase condition is not satisfied, re-identify the second resource by reducing the number of the second resources (Liu: increasing teacher resources when the student lacks soft labels [sec. 3.2, pp. 6-7, Algorithm 1, lines 7-9])
inference speed of the first model is slower than the training speed of the second model by the critical speed or more (Tan: EfficientNet is a family of models that are optimized for FLOPs and parameter efficiency. It leverages NAS to search for the baseline EfficientNet-B0 that has better trade-off on accuracy and FLOPs. The baseline model is then scaled up with a compound scaling strategy to obtain a family of models B1-B7. While recent works have claimed large gains on training or inference speed, they are often worse than EfficientNet in terms of parameters and FLOPs efficiency. The present disclosure improves the training speed while maintaining the parameter efficiency ¶ [0040]-[0041]).
Regarding claims 5, 14 and 20, the combination of Liu, Hong, and Tan discloses, acquire fifth throughput information for a preprocessing operation of data input to the first model and the second model (Liu: Liu: monitoring the supply of teacher generated soft labels during distillation and adds an available teacher resources when the student runs out [sec. 3.2, pp. 6-7 and Algorithm 1, lines 3-12]); and
based on a determination that at least one of an inference speed of the first model or a training speed of the second model is faster than a preprocessing speed by a critical speed or more, increase, based on the third to fifth throughput information, a number of third resources to perform the preprocessing operation (The operations can include performing a model training process on the first machine-learned model to obtain first training data descriptive of a first training speed. The operations can include generating a second machine-learned model from the defined model search space based at least in part on the first training data, wherein the second machine-learned model comprises one or more second values for the one or more searchable parameters, wherein at least one of the one or more second values is different than the one or more first values. The operations can include performing the model training process on the second machine-learned model to obtain second training data descriptive of a second training speed, wherein the second training speed is faster than the first training speed ¶ [0006]-[0007], [0030], [0034], [0099], [0103]. A model training process (e.g., the previously described progressive regularization, etc.) can be applied to the first machine-learned model to obtain training data descriptive of a first training speed. Based at least in part on the first training data, a second machine-learned model can be generated from the defined model search space that includes one or more second values different than the one or more first values. The model training process can be performed on the second machine-learned model to obtain second training data. The second training data can describe a second training speed faster than the first training speed. In such fashion, an architecture search technique can be utilized to optimize for training speed, therefore significantly increasing the overall training speed of the machine-learned model(s) ¶ [0022]).
Regarding claims 6, and 15, the combination of Liu, Hong, and Tan discloses, a graphics processing unit (GPU) configured to perform the inference operation by using the first model or the training operation by using the second model, and a central processing unit (CPU) configured to perform the preprocessing operation, and wherein the one or more instructions, when executed by the at least one processor , cause the device to: increase the number of the third resources to perform the preprocessing operation by connecting a remote CPU in addition to the CPU (Hong: Dedicated hardware for AI may be implemented by, for example, a central processing unit (CPU) and a graphics processing unit (GPU) and may also be implemented by a field programmable gate array (FPGA) and an application specific integrated circuit (ASIC) ¶ [0004], [0048], [0049]. in response to a request for executing the plurality of neural networks, the electronic device 100 and/or the host processor 110 of one or more embodiments may improve the utilization rate of the accelerator 120 by minimizing an idle time occurring in the middle of execution by allocating a resource for processing each neural network based on a situation of the accelerator 120 ¶ [0056]-[0057], [0116]-[0117]).
Regarding claim 7, the combination of Liu, Hong, and Tan discloses, acquire, while training the second model, third throughput information for the inference operation of the first model and fourth throughput information for the training operation of the second model , and adjust a parameter included in the second model based on a determination that (i) a training speed of the second model is slower than an inference speed of the first model and (ii) the training speed of the second model is slower than a critical speed based on the third and fourth throughput information (Tan: Based on this evaluation, feedback can be provided to the controller 302. Depending on the feedback, the controller 302 can then increase the regularization magnitude progressively at the trainer 304 and the complexity of the training samples. The controller 302 can then progressively increase regularization and sample complexity implemented by the trainer 304 during model training. As such, by starting with relatively weak regularization and complexity, the trainer 304 can train the machine-learned model 306 substantially faster than would be possible using a static level of regularization and sample complexity, therefore increasing training speed and substantially decreasing associated computational resource cost ¶ [0099]-[0103]).
Regarding claim 9, the combination of Liu, Hong, and Tan discloses, wherein the first throughput information comprises at least one of information for a time required per iteration, the number of samples processed per hour, floating-point operations per second (FLOPs), in an inference operation process of the first model, or an amount of data stored in the memory by the inference operation of the first model, and the second throughput information comprises at least one of information for a time required per iteration, the number of samples processed per hour, floating-point operations per second (FLOPs), in the training operation of the second model, or an amount of data stored in the memory by the training operation of the second model (Tan: EfficientNet is a family of models that are optimized for FLOPs and parameter efficiency. It leverages NAS to search for the baseline EfficientNet-B0 that has better trade-off on accuracy and FLOPs. The baseline model is then scaled up with a compound scaling strategy to obtain a family of models B1-B7. While recent works have claimed large gains on training or inference speed, they are often worse than EfficientNet in terms of parameters and FLOPs efficiency. The present disclosure improves the training speed while maintaining the parameter efficiency ¶ [0040]-[0042]).
Claim(s) 8 is rejected under 35 U.S.C. 103 as being unpatentable over Liu, in view of Hong in view of Tan in view of KHATTAK; Muhammad Uzair et al. (US 20240220722 A1) [Khattak].
Regarding claim 8, the combination of Liu, Hong, and Tan teaches all the elements of claim 7.
However, neither one of Liu, Hong, or Tan explicitly facilitate adjust the parameter included in the second model by performing at least one of pruning, freezing, or parameter tuning for the second model.
Khattak discloses, adjust the parameter included in the second model by performing at least one of pruning, freezing, or parameter tuning for the second model (The multi-modal prompt learning framework 200 can fine-tune both the language and vision branches where only learnable prompt tokens (e.g., the learnable text prompt tokens 218A-218T and the learnable image prompt tokens 238A—238T) are learned, while the other parts of the model are frozen. The multi-modal prompt learning framework 200 conditions vision prompts on language prompts via the vision-language coupling functions 250A, 250B, and 250J ¶ [0030]).
It would have been obvious, to one ordinary skilled in the art, before the effective filing date of the claimed invention to combine the teachings of the cited references because Khattak's system would have allowed Liu, Hong, and Tan to explicitly facilitate adjust the parameter included in the second model by performing at least one of pruning, freezing, or parameter tuning for the second model. The motivation to combine is apparent in Liu, Hong, and Tan’s reference, because a need to improve multi-modal training of pre-trained vision-language models.
Conclusion
The examiner requests, in response to this Office action, support be shown for language added to any original claims on amendment and any new claims. That is, indicate support for newly added claim language by specifically pointing to page(s) and line no(s) in the specification and/or drawing figure(s). This will assist the examiner in prosecuting the application.
When responding to this office action, Applicant is advised to clearly point out the patentable novelty which he or she thinks the claims present, in view of the state of the art disclosed by the references cited or the objections made. He or she must also show how the amendments avoid such references or objections See 37 CFR 1.111(c).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MOHAMMAD S ROSTAMI whose telephone number is (571)270-1980. The examiner can normally be reached Mon-Fri From 9 a.m. to 5 p.m..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Boris Gorney can be reached at (571)270-5626. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
9/30/2026
/MOHAMMAD S ROSTAMI/ Primary Examiner, Art Unit 2154
