DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 6/16/2026 has been entered.
This action is responsive to the original application filed on 9/21/2022 and the Remarks and Amendments and RCE filed on 6/16/2026.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. § 103 as being obvious over Liu et al. (US 20210264278 A1, hereinafter “Liu”) in view of Narang et al. (US 20190130271 A1, hereinafter “Narang”).
Regarding claim 1, Liu discloses [a] processor, comprising: ([0149]; “executable by processors of one or more computing devices”)
one or more circuits to iteratively training candidate neural network portions, wherein for a training iteration the one or more circuits are to: ([0026]; “progressively prune a neural network until one or more training conditions are satisfied. For example, the neural network pruning system can jointly train and prune a neural network for a set number of iterations or for a set time amount”, which discloses updating the scaling parameters during training. The scaling parameter is an iteratively updated portion of a neural network used to decide which layers or network portions remain; and [0149]; the hardware components include a circuit).
Liu fails to explicitly disclose but Narang discloses selectively identify one or more of the candidate neural network portions to be pruned based, at least in part, on a performance threshold ([0044]; “In one or more embodiments, the threshold (c) is monotonically growing to cause more blocks to be pruned as training progress. In one or more embodiments, pruning more blocks stops when a threshold is reached, e.g., after around a predetermined percentage (such as 40%) of training epochs has completed. Any blocks that had been zeroed out are held at zero even after pruning has ended resulting in a sparse model at the end of training”, which discloses selectively identifying one or more of the candidate neural network portions or blocks of weights to be pruned based on a performance threshold “c”)
increase the performance threshold for a subsequent training iteration based, at least in part, on a number of elapsed training iterations ([0044]; “In one or more embodiments, the threshold (c) is monotonically growing to cause more blocks to be pruned as training progress. In one or more embodiments, pruning more blocks stops when a threshold is reached, e.g., after around a predetermined percentage (such as 40%) of training epochs has completed. Any blocks that had been zeroed out are held at zero even after pruning has ended resulting in a sparse model at the end of training”, which discloses increasing or monotonically growing a performance threshold “c” for s subsequent training iteration or training epoch based on a number of training iterations. Raising the threshold “c” causes a larger fraction of the network to be pruned at each subsequent iteration. The word “monotonically growing” means that the threshold is increased for each subsequent training iteration; and [0047]; “The threshold (∈) is continuous and monotonically growing, comprising an initial section 405 with ∈ setting as zero for iterations below a start iteration number (Start_iter), a start section 410 with ∈ linearly increasing with a start slope θ for iterations between the start iteration number and a ramp iteration number (Ramp_iter), a ramp section 415 with ∈ linearly increasing with a ramp slope ϕ for iterations between the ramp iteration number and an end iteration number (End_iter)”).
Liu and Narang are analogous art because both are concerned with neural network pruning. Before the effective filing date of the claimed invention, it would have been obvious to one skilled in neural network pruning to combine the performance thresholds and elapsed training iterations of Narang with the method of Liu to yield to the predictable result of selectively identify one or more of the candidate neural network portions to be pruned based, at least in part, on a performance threshold and increase the performance threshold for a subsequent training iteration based, at least in part, on a number of elapsed training iterations. The motivation for doing so would be to cause more blocks to be pruned as training progress (Narang; [0044]).
Regarding claim 2, the rejection of claim 1 is incorporated and Liu discloses wherein the one or more circuits are to increase the performance metrics based on training progress for a current training epoch ([0026]; “For example, the neural network pruning system can jointly train and prune a neural network for a set number of iterations or for a set time amount. In another example, the neural network pruning system can jointly train and prune a neural network until the neural network converges and/or a minimum amount of network loss is achieved”, wherein the scaling parameter is updated after each training iteration or epoch, so the metric is calculated with respect to the current training epoch).
Regarding claim 3, the rejection of claim 1 is incorporated and Liu discloses calculate one or more weights corresponding to the one or more candidate neural network portions; and select the one or more candidate neural network portions based, at least in part, on the one or more weights ([0027]; “the neural network pruning system can jointly learn network weights and scaling parameters for each portion (e.g., layers or channels within those layers) of the neural network, determine a total loss, and back-propagate the loss to reduce total loss in the next iteration”, which discloses that each network portion or layer has associated weights and that pruning can consider those weights together
Regarding claim 4, the rejection of claim 1 is incorporated and Liu fails to explicitly disclose but Narang discloses remove a subset of a first set of neural network portions based on the performance threshold; and obtain a first neural network layer from the first set of candidate neural network portions ([0044] and [0047]).
The motivation to combine Liu and Narang is the same as discussed above with respect to claim 1.
Regarding claim 5, the rejection of claim 1 is incorporated and Liu fails to explicitly disclose but Narang discloses wherein the one or more circuits are to linearly increase the performance threshold ([0044]; and [0047]).
The motivation to combine Liu and Narang is the same as discussed above with respect to claim 1.
Regarding claim 6, the rejection of claim 1 is incorporated and Liu discloses wherein the processor is part of one or more graphics processing units (GPUs) ([0149]; [0168]; [0174]; [0180]).
Regarding claim 7, the rejection of claim 1 is incorporated and Liu discloses generate a neural network comprising one or more of the candidate neural network portions ([0028-0029]).
Regarding claim 8, it is a system claim corresponding to the steps of claim 1, and is rejected for the same reasons as claim 1.
Regarding claim 9, the rejection of claim 8 is incorporated and Liu discloses obtain the set of candidate neural network portions; iteratively reduce the set of candidate neural network layers based, at least in part, on the one or more iteratively increasing neural network performance metrics; and select one or more neural network layers from the set of candidate neural network portions ([0028]; “the neural network pruning system can remove one or more portions associated with the lowest scaling parameters after a training iteration”).
Regarding claim 10, the rejection of claim 8 is incorporated and Liu fails to explicitly disclose but Narang discloses at each training epoch, calculate a value of the one or more the performance threshold ([0044]; and [0047]).
The motivation to combine Liu and Narang is the same as discussed above with respect to claim 1.
Regarding claim 11, the rejection of claim 8 is incorporated and Liu discloses wherein the one or more processors are to selectively identify the one or more of the candidate neural network portions to be pruned based, at least in part, on one or more latency constraints ([0030]; “Accordingly, in many implementations, the neural network pruning system can gradually and automatically prune and morph a deep and wide neural network into a shallow and thin neural network that is tailored to a particular task and dataset while also maintaining overall accuracy and increasing efficiency”).
Regarding claim 12, the rejection of claim 8 is incorporated and Liu discloses iteratively update a set of weights based, at least in part, on training data; and selectively identify the one or more of the candidate neural network portions to be pruned based, at least in part, on the set of weights ([0027]; “For example, as described below, the neural network pruning system can jointly learn network weights and scaling parameters for each portion (e.g., layers or channels within those layers) of the neural network, determine a total loss, and back-propagate the loss to reduce total loss in the next iteration”, wherein the weights are updated during training and used with scaling parameter for pruning decisions).
Regarding claim 13, the rejection of claim 8 is incorporated and Liu discloses wherein the one or more processors are to train the candidate neural network portions to perform one or more computer vision tasks ([0030]; “tailored to a particular task and dataset while also maintaining overall accuracy and increasing efficiency”, the particular task being computer vision; and [0006]).
Regarding claim 14, the rejection of claim 8 is incorporated and Liu fails to explicitly disclose but Narang discloses wherein the performance threshold is iteratively increases based on training progress ([0044]; and [0047]).
The motivation to combine Liu and Narang is the same as discussed above with respect to claim 1.
Regarding claim 15, it is a method claim corresponding to the steps of claim 1, and is rejected for the same reasons as claim 1.
Regarding claim 16, the rejection of claim 15 is incorporated and Liu fails to explicitly disclose but Narang discloses calculating a set of values corresponding to a set of the candidate neural network portions; reducing the set of candidate neural network layers based, at least in part, on the set of values and iteratively increasing neural network performance threshold; and selecting one or more neural network layers from the set of candidate neural network portions ([0044]; and [0047]).
The motivation to combine Liu and Narang is the same as discussed above with respect to claim 1.
Regarding claim 17, the rejection of claim 15 is incorporated and Liu discloses training the candidate neural network portions to perform one or more natural language processing (NLP) tasks ([0030]; “tailored to a particular task and dataset while also maintaining overall accuracy and increasing efficiency”, the particular task being NLP; and [0006]).
Regarding claim 18, the rejection of claim 15 is incorporated and Liu discloses wherein the one or more candidate neural network portions correspond to one or more blocks of a data structure ([0024-0026]; describes layers/channels/blocks of the network data structure subjected to pruning).
Regarding claim 19, the rejection of claim 15 is incorporated and Liu discloses calculating the one or more candidate neural network portions by at least, at one or more times during training, reducing a set of the candidate neural network portions ([0028]).
Regarding claim 20, Liu discloses A non-transitory computer readable medium having stored thereon a set of instructions, which if performed by one or more processors, cause the one or more processors to at least perform the method of claim 15 (Liu; [0149]; and [0026-0028]; and Narang; [0044]; and [0047]).
The motivation to combine Liu and Narang is the same as discussed above with respect to claim 1.
Response to Arguments
Applicant’s arguments and amendments, filed on 6/16/2026, with respect to the 35 USC § 103 rejection of the pending claim have been fully considered but are moot because the arguments do not apply to the references used to reject the amended claims of the present application. Liu and Narang are now being used to render the independent claims obvious under 35 USC § 103.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Pan et al. (US 20220092645 A1).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Brent Hoover whose telephone number is (303)297-4403. The examiner can normally be reached Monday - Friday 9-5 MST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abdullah Kawsar can be reached on 571-270-3169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BRENT JOHNSTON HOOVER/Primary Examiner, Art Unit 2127
Prosecution Insights