DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-20 are presented for examination.
Response to Amendment
Applicant’s amendment has obviated the objection to the abstract (though it has caused new grounds of objection to be made, see claim objection infra) and the rejections under 35 USC § 112(b). Therefore, those objections and rejections are withdrawn. However, Applicant has not made the required corrections to the drawings and specification. Therefore, those objections are maintained.
Drawings
The drawings are objected to because (a) Fig. 5 contains text on a shaded background, see 37 CFR § 1.84(p)(3); (b) in Fig. 9A, text in reference characters 960, 964, and 996 mingles with the lines of the drawings, see id.; (c) in Figs. 17A, 36, and 37, reference characters are oriented both horizontally and vertically, see 37 CFR § 1.84(p)(1); and (d) reference character 1318 (Fig. 8) appears in the drawings but not the specification. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specification
The lengthy specification has not been checked to the extent necessary to determine the presence of all possible minor errors. Applicant’s cooperation is requested in correcting any errors of which applicant may become aware in the specification.
The abstract of the disclosure is objected to because the phrase “one or more previously less than all previously” is nonsensical. A corrected abstract of the disclosure is required and must be presented on a separate sheet, apart from any other text. See MPEP § 608.01(b).
The use of the terms AMAZON, GOOGLE, and MICROSOFT (paragraph 106), which are trade names or marks used in commerce, has been noted in this application. The terms should be accompanied by the generic terminology; furthermore, the terms should be capitalized wherever they appear or, where appropriate, include a proper symbol indicating use in commerce such as ™, SM , or ® following the terms.
Although the use of trade names and marks used in commerce (i.e., trademarks, service marks, certification marks, and collective marks) is permissible in patent applications, the proprietary nature of the marks should be respected and every effort made to prevent their use in any manner which might adversely affect their validity as commercial marks.1
Claim Rejections - 35 USC § 103
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claims 1-5, 7-12, and 14-19 are rejected under 35 U.S.C. 103 as being unpatentable over Shen et al. (US 20220292360) (“Shen”) in view of Zhuo et al. (US 11030528) (“Zhuo”).
Regarding claim 1, Shen discloses “[a] processor, comprising:
one or more circuits to:
compute initial scores of one or more portions of one or more neural networks (process for a system of neural network pruning involves determining a sub-network [i.e., less than all portions of the neural networks] and calculating an early pruning indicator (EPI) value [initial score]; system determines a sub-network by ranking neurons based on calculated importance scores for each neuron; system then determines whether EPI is greater than a threshold and EPI from past epochs [previously evaluated portions of the networks]; the system then sets a status to prune and neurons are pruned such that only the top k neurons of the neural network remain – Shen, paragraphs 125-30 and Figs. 7-8) …;
compare the initial scores with scores of one or more previously evaluated portions of the one or more neural networks to identify a subset of the previously evaluated portions to use (system then determines whether EPI is greater than a threshold and EPI from past epochs [previously evaluated portions of the networks]; the system then sets a status to prune and neurons are pruned such that only the top k neurons [subset of the previously evaluated portions] of the neural network remain – Shen, paragraphs 125-30 and Figs. 7-8 ) …;
deactivate at least one portion of the one or more portions of the one or more neural networks identified according to the … scores (system then determines whether EPI is greater than a threshold and EPI [score] from past epochs; the system then sets a status to prune and neurons are pruned [deactivated] such that only the top k neurons of the neural network remain – Shen, paragraphs 125-30 and Figs. 7-8); and
perform one or more inferencing tasks with the one or more neural networks with at least the one portion being deactivated (if no epochs remain, a neural network is returned and utilized to perform various processes, such as image classification, object detection, segmentation, data analysis, and/or similar processes [inferencing tasks] – Shen, paragraph 133 and Figs. 7-8).”
Shen appears not to disclose explicitly the further limitations of the claim. However, Zhuo discloses “comput[ing] … scores of one or more portions of one or more neural networks based, at least in part, on respective output of the one or more portions (the highest accuracy of verification sets before and after pruning is compared, and if the highest accuracy of the verification set after pruning is greater than or equal to the highest accuracy of the verification set [output] before pruning, the current pruned ratio [score] is taken as a new lower limit of the pruned ratio and increased – Zhuo, claim 1)… [and]
modify[ing] the initial scores (he highest accuracy of verification sets before and after pruning is compared, and if the highest accuracy of the verification set after pruning is greater than or equal to the highest accuracy of the verification set [output] before pruning, the current pruned ratio [score] is taken as a new lower limit of the pruned ratio and increased [modified] – Zhuo, claim 1) ….”
Zhuo and the instant application both relate to pruning neural networks and are analogous. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Shen to calculate a pruning score based on output of the networks and then modify the score, as disclosed by Zhuo, and an ordinary artisan could reasonably expect to have done so successfully. Doing so would speed up the computation of the network and reduce hardware requirements. See Zhuo. Col. 1, l. 65-col. 2, l. 2.
Claim 8 is a system claim corresponding to processor claim 1 and is rejected for the same reasons as given in the rejection of that claim. Similarly, claim 15 is a non-transitory machine-readable medium claim corresponding to processor claim 1 and is rejected for the same reasons as given in the rejection of that claim.
Regarding claim 2, Shen/Zhuo discloses that “the one or more portions of the one or more neural networks include one or more first neurons of a layer of the one or more neural networks, and the one or more previously evaluated portions of the one or more neural networks include one or more second neurons of one or more previously evaluated layers of the one or more neural networks (if a system for neural network pruning determines that a calculated early pruning indicator value is greater than or equal to a stability threshold and is greater than or equal to calculated early pruning indicator values for one or more past epochs [previous evaluations], the system indicates that the neural network is to be pruned – Shen, paragraph 89; sub-network of a first neural network is formed by one or more neurons per layer of a first neural network – id. at paragraph 75; difference between sub-networks for an lth layer may be defined – id. at paragraph 86 [i.e., the system evaluates some neurons in some layers in one epoch and other neurons in other layers in other epochs]).”
Claim 9 is a system claim corresponding to processor claim 2 and is rejected for the same reasons as given in the rejection of that claim. Similarly, claim 16 is a non-transitory machine-readable medium claim corresponding to processor claim 2 and is rejected for the same reasons as given in the rejection of that claim.
Regarding claim 3, Shen discloses that “the deactivation of the one or more portions of the one or more neural networks comprises removing the one or more portions based, at least in part, on a threshold and the … scores representing importance of the one or more portions within the one or more neural networks (process for a system of neural network pruning involves determining a sub-network and calculating an early pruning indicator (EPI) value; system determines a sub-network by ranking neurons based on calculated importance scores for each neuron [portion]; system then determines whether EPI is greater than a threshold and EPI from past epochs; the system then sets a status to prune and neurons are pruned [deactivated] such that only the top k neurons of the neural network remain – Shen, paragraphs 125-30 and Figs. 7-8).”
Zhuo discloses “modified scores,” as shown in the rejection of claim 1. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Shen to modify the pruning score, as disclosed by Zhuo, for substantially the same reasons as given in the rejection of claim 1.
Claim 10 is a system claim corresponding to processor claim 3 and is rejected for the same reasons as given in the rejection of that claim. Similarly, claim 17 is a non-transitory machine-readable medium claim corresponding to processor claim 3 and is rejected for the same reasons as given in the rejection of that claim.
Regarding claim 4, Shen discloses that “the threshold is set based, at least in part, on ranking the … scores of the one or more portions within the one or more neural networks (system determines a sub-network by ranking neurons based at least in part on calculated importance scores for said neurons , and the system calculates an EPI value for the sub-network; the system then determines whether the EPI is greater than a threshold – Shen, paragraphs 125-26; grid search is utilized to determine the stability threshold – id. at paragraph 112; grid search analyzes every neuron during one or more epochs to determine a most optimal set of neurons to remove – id. at paragraph 107 [i.e., the grid search used to determine the threshold determines which neurons to remove, meaning that it is based on the ranking that determines which neurons to remove]).”
Zhuo discloses “modified scores,” as shown in the rejection of claim 1. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Shen to modify the pruning score, as disclosed by Zhuo, for substantially the same reasons as given in the rejection of claim 1.
Claim 11 is a system claim corresponding to processor claim 4 and is rejected for the same reasons as given in the rejection of that claim. Similarly, claim 18 is a non-transitory machine-readable medium claim corresponding to processor claim 4 and is rejected for the same reasons as given in the rejection of that claim.
Regarding claim 5, Shen/Zhuo discloses that “the one or more circuits are further to:
calculate one or more first metrics associated with the one or more first neurons of the layer (system determines a sub-network by ranking [calculating metrics on] neurons based on calculated importance scores for each neuron – Shen, paragraph 125);
calculate one or more second metrics associated with the one or more second neurons of the one or more previously evaluated layers (system determines a sub-network by ranking neurons based on calculated importance scores for each neuron; system then determines whether EPI is greater than a threshold and EPI from past epochs [previously evaluated portions of the networks] – Shen, paragraphs 125-30; sub-network of a first neural network is formed by one or more neurons per layer of a first neural network – id. at paragraph 75; difference between sub-networks for an lth layer may be defined – id. at paragraph 86 [i.e., the system evaluates some neurons in some layers in one epoch and other neurons in other layers in other epochs]); and
deactivate the one or more first neurons based, at least in part, on the one or more first metrics and the one or more second metrics (if EPI is greater than EPI values from past epochs, the system sets a status to prune [deactivate neurons] – Shen, paragraphs 126-27; see also paragraph 125 (disclosing that the system calculates the EPI value based on the sub-network formed by the ranking [i.e., based on the metrics])).”
Claim 12 is a system claim corresponding to processor claim 5 and is rejected for the same reasons as given in the rejection of that claim. Similarly, claim 19 is a non-transitory machine-readable medium claim corresponding to processor claim 4 and is rejected for the same reasons as given in the rejection of that claim.
Regarding claim 7, Shen/Zhuo discloses that “the one or more first metrics and the one or more second metrics are based, at least in part, on an L2-norm (magnitude-based criterion refers to a criterion to rank neurons that uses an l2-norm of neuron weights to measure a relevance of a neuron in a network – Shen, paragraph 70).”
Claim 14 is a system claim corresponding to processor claim 7 and is rejected for the same reasons as given in the rejection of that claim.
Claims 6, 13, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Shen in view of Zhuo and further in view of Miret et al. (US 20220092425) (“Miret”).
Regarding claim 6, Shen/Zhuo appears not to disclose explicitly the further limitations of the claim. However, Miret discloses that “to compare the initial scores with the scores of the one or more previously evaluated portions, the one or more circuits are to determine a sum of a portion of the one or more second metrics having higher values than the one or more first metrics (pruning module selects a subset of the filters based on the pruning ratio; for instance, where the pruning ratio is 10%, the filter pruning module selects 10% of the filters based on the ranking, e.g., the 10% filters that have lower absolute magnitude sum than the remaining 90% filters [second metric = absolute magnitude of 90% of filters; first metric = sum of absolute magnitudes of 10% of filters] – Miret, paragraph 71).”
Miret and the instant application both relate to pruning of neural networks and are analogous. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Shen/Zhuo to perform the pruning based on a ratio of a metric related to lower-performing neurons to that of higher-performing neurons, as disclosed by Miret, and an ordinary artisan could reasonably expect to have done so successfully. Doing so would increase the sparsity in the hidden layers, thereby reducing the memory footprint and the processor resources consumed in executing the model. See Miret, paragraph 71.
Claim 13 is a system claim corresponding to processor claim 6 and is rejected for the same reasons as given in the rejection of that claim. Similarly, claim 20 is a non-transitory machine-readable medium claim corresponding to processor claim 6 and is rejected for the same reasons as given in the rejection of that claim.
Response to Arguments
Applicant’s arguments with respect to the claims have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RYAN C VAUGHN whose telephone number is (571)272-4849. The examiner can normally be reached M-R 7:00a-5:00p ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kamran Afshar, can be reached at 571-272-7796. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RYAN C VAUGHN/ Primary Examiner, Art Unit 2125
1 Applicant’s argument that the recitation of these trademarks is compliant with the MPEP is unconvincing at least because the very section of the MPEP that Applicant cites as allegedly supporting its position (viz., § 608.01(v)) states that “[m]arks should be identified by capitalizing each letter of the mark” (emphasis added). The current version of the specification only capitalizes the first letter of each word of the marks. Properly speaking, the marks should read “AMAZON WEB SERVICES, GOOGLE CLOUD and MICROSOFT AZURE”.