DETAILED ACTION
Claims 1-19 are presented for examination.
This office action is in response to submission of application on 03/21/2024.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 03/21/2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 5-8 and 13-16 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 5 recites the limitation "… sequentially aligning the tasks based on the execution times between the measuring and creating the second deep learning model …". It is unclear whether “the measuring” refers to “…measuring execution times of tasks…” in claim 1, or some other measurement such as the measurement of criterion being met in claim 3. There is insufficient antecedent basis for this limitation. Dependent claims 6-8 inherit the deficiency and therefore are rejected on the same basis.
Claim 13 recites the limitation "… sequentially aligning the tasks based on the execution times between the measuring and creating the second deep learning model …". It is unclear whether the “measuring” refers to “…measuring execution times of tasks…” in claim 9 or “…measuring accuracy…”in claim in claim 11, or some other measurement such as the measurement of criterion being met in claim 12 . There is insufficient antecedent basis for this limitation. Dependent claims 14-16 inherit the deficiency and therefore are rejected on the same basis.
For examination purposes, the examiner is interpreting “the measuring” in claims 5 and 13 to refer to “…measuring execution times of tasks…”.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-19 are rejected under AIA 35 U.S.C. 102(a)(1) as being anticipated by Kim et al. (CPrune: Compiler-Informed Model Pruning for Efficient Target-Aware DNN Execution), hereafter Kim.
Regarding claim 1, Kim discloses:
A compiler-based deep learning model pruning method, comprising (page 10, paragraph 3, lines 1-9 “we carry out various experiments to evaluate the performance of CPrune for different DNN models on different mobile CPU/GPU devices. We conduct experiments over different pre-trained DNN models, including ResNet-18 [12], MobileNetV2 [34], and MnasNet1.0 [37], with ImageNet [18] or CIFAR-10 [17] datasets on various resource-constrained mobile devices (Samsung Galaxy S8 (Kryo 280 CPU), S9 (Kryo 385 CPU and Mali-G72 GPU), S20+ (Kryo 585 CPU), or Google Pixel 3 XL (Kryo 385 CPU)). We also use multiple host PCs with NVIDIA GeForce RTX 1080 Ti or 2080 Ti. All pruned models are optimized by stochastic gradient descent (SGD)”),
extracting multiple subgraphs from a first deep learning model (Fig. 3 step 2 teaches extracting multiple subgraphs from the first deep learning model in step 1),
generating programs representing respective tasks allocated to the extracted multiple subgraphs (Fig. 3 step 2 teaches generating different programs that represent tasks allocated to the extracted multiple subgraphs),
compiling the first deep learning model based on selected programs and measuring execution times of tasks of the first deep learning model on respective devices (Fig. 3 and page 5, final paragraph, lines 7-9 “a DNN compiler creates numerous intermediate representations for each task and selects the fastest program on the target device” and page 10, paragraph 3, lines 1-2 “we carry out various experiments to evaluate the performance of CPrune for different DNN models on different mobile CPU/GPU devices.” teaches compiling the DNN model based on selected candidate programs and measures execution times of tasks on respective target devices),
creating a second deep learning model by pruning a subgraph corresponding to at least one task selected from among the tasks from the first deep learning model based on the execution times on the respective devices (Fig. 3 and page 6, first paragraph, lines 8-10 “CPrune prunes subgraphs of the selected task while ensuring their code structures follow the structure of the fastest program of that task” teaches creating a pruned model as a second deep learning model by pruning a subgraph corresponding to at least one task selected from among the tasks from the first deep learning model based on the execution times on the respective devices).
Regarding claim 2, Kim discloses the method of claim 1 (and thus the rejection of claim 1 is incorporated). Kim further discloses:
wherein generating the programs comprises: allocating an identical task to two or more subgraphs having an identical form (Fig. 3 teaches allocating identical tasks to identical subgraphs in step 2).
Regarding claim 3, Kim discloses the method of claim 1 (and thus the rejection of claim 1 is incorporated). Kim further discloses:
short-term training the compiled first deep learning model and thereafter measuring accuracy of the first deep learning model (Fig. 3 and Algorithm 1, line 11),
determining whether the accuracy of the first deep learning model and an execution time of the first deep learning model meet certain criteria (Fig. 3 and page 5 last two lines “CPrune checks if the model meets the minimum execution and accuracy requirements”),
wherein, when it is determined that the accuracy of the first deep learning model and the execution time of the first deep learning model meet the certain criteria, proceeding to creating the second deep learning model (Fig. 3 steps 2-3 teach that if the accuracy and execution criteria are met, the steps proceed to creation of the second pruned model in step 5).
Regarding claim 4, Kim discloses the method of claim 3 (and thus the rejection of claim 3 is incorporated). Kim further discloses:
the first deep learning model is selected from among multiple candidate deep learning models, and (Fig. 3 and page 10, paragraph 3, lines 2-3 “We conduct experiments over different pre-trained DNN models,” teaches selection of the DNN model from multiple candidate deep learning models),
when it is determined that the execution time of the first deep learning model does not meet a certain criterion, the first deep learning model is changed to one of the multiple candidate deep learning models, and thereafter a procedure starting from extracting subgraphs from the changed first deep learning model is performed again (Fig. 3 steps 2-3 teaches that when the criteria are not met, the first deep learning model is changed to one of the multiple candidate deep learning models, and the procedure restarts from step 1).
Regarding claim 5, Kim discloses the method of claim 4 (and thus the rejection of claim 4 is incorporated). Kim further discloses:
sequentially aligning the tasks based on the execution times between the measuring and creating the second deep learning model (Fig. 3 step 3 teaches task ordering as sequentially aligning the tasks based on the execution times between the measuring and creating the second deep learning model).
Regarding claim 6, Kim discloses the method of claim 5 (and thus the rejection of claim 5 is incorporated). Kim further discloses:
the aligning comprises: updating a first table in which a program executed at a highest speed for each task, an execution time of the program executed at the highest speed, a number of subgraphs, and a total execution time for each task calculated by multiplying the execution time of the corresponding program by the number of subgraphs are mapped to each task (Fig. 3 and page 7, paragraph 5, lines 2-3 “CPrune sorts tasks according to the pruning impact (task’s execution time × number of subgraphs associated with the task).” Teaches updating the task ordering table in which a program executed at a highest speed for each task, an execution time of the program executed at the highest speed, a number of subgraphs, and a total execution time for each task calculated, by multiplying the execution time of the corresponding program by the number of subgraphs are mapped to each task),
aligning the tasks in descending order based on total execution times for respective tasks and storing the aligned tasks in a task list (Fig. 3 step 3 teaches a ranked order of tasks based on total execution times for respective tasks and storing the aligned tasks in a task list R).
Regarding claim 7, Kim discloses the method of claim 6 (and thus the rejection of claim 6 is incorporated). Kim further discloses:
updating a second table in which at least one subgraph allocated to each task and a fastest program are mapped to each task, between the aligning and creating the second deep learning model (Fig. 3, step 4 teaches updating a second table in which at least one subgraph allocated to each task and a fastest program are mapped to each task, between the aligning and creating the second deep learning model),
wherein the pruning comprises: creating at least one second deep learning model by pruning a selected subgraph while maintaining a fastest program for an at least one high-ranked task selected from among the aligned tasks (Fig. 3, and page 7, paragraph 2 teaches creating pruned model by pruning a selected subgraph while maintaining a fastest program for an at least one high-ranked task selected from among the aligned tasks).
Regarding claim 8, Kim discloses the method of claim 7 (and thus the rejection of claim 7 is incorporated). Kim further discloses:
updating the multiple candidate deep learning models to at least one second deep learning model (Algorithm 1 and Fig. 3 teaches updating the multiple candidate deep learning models to at least one second deep learning model),
selecting one of updated multiple candidate deep learning models as a first deep learning model, and repeatedly performing again a procedure starting from extracting subgraphs from the first deep learning model (Algorithm 1 teaches selecting one of updated multiple candidate deep learning models as a first deep learning model, and repeatedly performing again a procedure starting from extracting subgraphs from the first deep learning model),
wherein the procedure is repeatedly performed until the execution time of the first deep learning model meets a certain criterion and a task to be pruned is not present in a previously updated task list (Algorithm 1 and page 6, first paragraph, last 2 lines “This process continues to make the most efficient pruned DNN model satisfying the accuracy requirement” teaches repeatedly performing the procedure until the execution time of the first deep learning model meets a certain criterion and a task to be pruned is not present in a previously updated task list).
Claims 9-16 are substantially similar to claims 1-8, and thus are rejected on the same basis as claims 1-8.
Regarding claim 17, Kim discloses:
A compiler-based deep learning model pruning method, comprising (page 10, paragraph 3, lines 1-9 “we carry out various experiments to evaluate the performance of CPrune for different DNN models on different mobile CPU/GPU devices. We conduct experiments over different pre-trained DNN models, including ResNet-18 [12], MobileNetV2 [34], and MnasNet1.0 [37], with ImageNet [18] or CIFAR-10 [17] datasets on various resource-constrained mobile devices (Samsung Galaxy S8 (Kryo 280 CPU), S9 (Kryo 385 CPU and Mali-G72 GPU), S20+ (Kryo 585 CPU), or Google Pixel 3 XL (Kryo 385 CPU)). We also use multiple host PCs with NVIDIA GeForce RTX 1080 Ti or 2080 Ti. All pruned models are optimized by stochastic gradient descent (SGD)”),
extracting multiple subgraphs from a first deep learning model selected from among multiple candidate deep learning models (Fig. 3 step 2 teaches extracting multiple subgraphs from the first deep learning model in step 1 and page 10, paragraph 3, lines 2-3 “We conduct experiments over different pre-trained DNN models,” teaches selection of the DNN model from multiple candidate deep learning models),
generating programs representing respective tasks allocated to the extracted multiple subgraphs (Fig. 3 step 2 teaches generating different programs that represent tasks allocated to the extracted multiple subgraphs),
compiling the first deep learning model based on selected programs and measuring execution times of tasks of the first deep learning model on respective tasks (Fig. 3 and page 5, final paragraph, lines 7-9 “a DNN compiler creates numerous intermediate representations for each task and selects the fastest program” teaches compiling the DNN model based on selected candidate programs and measures execution times of tasks on respective tasks),
short-term training the compiled first deep learning model and thereafter measuring accuracy of the first deep learning model (Fig. 3 and Algorithm 1, line 11),
determining whether the accuracy of the first deep learning model and an execution time of the first deep learning model measured in compiling the first deep learning model meet certain criteria (Fig. 3 and page 5 last two lines “CPrune checks if the model meets the minimum execution and accuracy requirements”),
sequentially aligning the tasks based on the execution times (Fig. 3 step 3 teaches task ordering as sequentially aligning the tasks based on the execution times).
creating at least one second deep learning model by pruning a selected subgraph while maintaining a fastest second program for at least one high-ranked task selected from among the aligned tasks (Fig. 3, and page 7, paragraph 2 teaches creating pruned model by pruning a selected subgraph while maintaining a fastest program for an at least one high-ranked task selected from among the aligned tasks).
wherein the multiple candidate deep learning models are updated to at least one second deep learning model (Algorithm 1 and Fig. 3 teaches updating the multiple candidate deep learning models to at least one second deep learning model),
wherein one of the updated multiple candidate deep learning models is selected as a first deep learning model, and a procedure starting from extracting subgraphs from the first deep learning model is repeatedly performed again (Algorithm 1 teaches selecting one of updated multiple candidate deep learning models as a first deep learning model, and repeatedly performing again a procedure starting from extracting subgraphs from the first deep learning model),
wherein the procedure is repeatedly performed until the execution time of the first deep learning model meets a certain criterion and a task to be pruned is not present (Algorithm 1 and page 6, first paragraph, last 2 lines “This process continues to make the most efficient pruned DNN model satisfying the accuracy requirement” teaches repeatedly performing the procedure until the execution time of the first deep learning model meets a certain criterion and a task to be pruned is not present in a previously updated task list).
Regarding claim 18, Kim discloses the method of claim 17 (and thus the rejection of claim 17 is incorporated). Kim further discloses:
the aligning comprises: updating a first table in which a program executed at a highest speed for each task, an execution time of the program executed at the highest speed, a number of subgraphs, and a total execution time for each task calculated by multiplying the execution time of the corresponding program by the number of subgraphs are mapped to each task (Fig. 3 and page 7, paragraph 5, lines 2-3 “CPrune sorts tasks according to the pruning impact (task’s execution time × number of subgraphs associated with the task).” Teaches updating the task ordering table in which a program executed at a highest speed for each task, an execution time of the program executed at the highest speed, a number of subgraphs, and a total execution time for each task calculated, by multiplying the execution time of the corresponding program by the number of subgraphs are mapped to each task),
aligning the tasks in descending order based on total execution times for respective tasks and storing the aligned tasks in a task list (Fig. 3 step 3 teaches a ranked order of tasks based on total execution times for respective tasks and storing the aligned tasks in a task list R).
Regarding claim 19, Kim discloses the method of claim 17 (and thus the rejection of claim 17 is incorporated). Kim further discloses:
updating a second table in which at least one subgraph allocated to each task and a fastest program are mapped to each task, between the aligning and creating the second deep learning model (Fig. 3, step 4 teaches updating a second table in which at least one subgraph allocated to each task and a fastest program are mapped to each task, between the aligning and creating the second deep learning model).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's
disclosure.
Yuan et. al (“NPAS: A Compiler-aware Framework of Unified Network Pruning and Architecture Search for Beyond Real-Time Mobile Acceleration”) teaches compiler based pruning of deep learning models.
Yang et. al (“NetAdapt: Platform-Aware Neural Network Adaptation for Mobile Applications”) teaches compiler based pruning of deep learning models.
Li et al. (“Optimizing the Deep Neural Networks by Layer-Wise Refined Pruning and the Acceleration on FPGA”) teaches subgraph wise pruning of deep learning models.
U.S. Pub No. 20200151579 A1: Yang et al. teaches compiler and task optimization, and subgraphs in deep learning models.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HUMAIRA ZAHIN MAUNI whose telephone number is (703)756-5654. The examiner can normally be reached Monday - Friday, 9 am - 5 pm (ET).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, MATT ELL can be reached at (571) 270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/H.Z.M./Examiner, Art Unit 2141
/MATTHEW ELL/Supervisory Patent Examiner, Art Unit 2141
Prosecution Insights