Detailed Action
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-23 are pending for examination. Claims 1 and 12-13 are independent.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 6-8, 10-13, 18-20, and 22-23 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by PARTOVI NIA et al. (US 2021/0073643 A1, hereinafter "Nia").
Regarding Claim 1
Nia discloses: A method for filter based pruning of a convolutional neural network (CNN) ([Abstract]), comprising:
initializing a CNN, the CNN including a plurality of filters, each filter associated with a weight and a filter factor ([Para 0051, 0053-0063, 0084, Fig 2, Fig 5] a subnetwork with convolution operations (i.e. CNN) associated with a scaling factor (i.e. filter factor) and at least one filter each filter including a plurality of weights. [Para 0089 and Fig 6A-D] further describes initializing a CNN.);
providing the CNN with a training input ([Para 0051, 0053, 0087, Fig 2 and Fig 5] describes providing an input feature map during training.);
adjusting a weight of a filter of the plurality of filters in response to processing the training input ([Para 0050, 0053-0054, 0072-0073, Fig 2, and Fig 5-6] describes training/learning weights (i.e. adjusting).);
adjusting a filter factor of the filter of the plurality of filters in response to processing the training input; ([Para 0050, 0057-0066, 0085, Claim 2, Fig 2, and Fig 5-6] describes during training for each scaling factor corresponding to a filter, learning the scaling factor by minimizing loss of a loss function (i.e. adjusting).)
pruning the CNN by removing the filter in response to detecting that a value of the filter factor is below a predefined threshold after training is complete ([Para 0021, 0049, 0056, 0062, 0069, Fig 2, and Fig 5-6] describes when the scaling factor satisfies a predetermined criterion (i.e. below threshold), selectively pruning the filter corresponding to the scaling factor by masking the filter from the convolution operation. [Para 0062] states “Once training is completed, the weights (e.g. , W ) that correspond to any masked filters ( e.g. , Fj ) can be removed from the block weights W.”);
storing a trained pruned CNN based on the initialized CNN ([Para 0012, 0045, 0050] disclose storing the pruned network.); and
processing an input with the trained CNN ([Para 0054 0088 0103] describes an inference stage for the trained CNN).
Regarding Claim 12
Nia discloses: A non-transitory computer-readable medium storing a set of instructions for filter based pruning of a convolutional neural network (CNN), the set of instructions comprising: one or more instructions that, when executed by one or more processors of a device ([Para 0027, 0092, 0105, Claims 11, and Fig 7]), cause the device to: (Claim 12 is a non-transitory computer-readable medium claim that corresponds to claim 1 and the rest of the limitations are rejected on the same ground)
Regarding Claim 13
Nia discloses: A system for filter based pruning of a convolutional neural network (CNN) comprising: a processing circuitry; and a memory, the memory containing instructions that, when executed by the processing circuitry ([Para 0090-0094, Claim 11, and Fig 7]), configure the system to: (Claim 13 is a system claim that corresponds to claim 1 and the rest of the limitations are rejected on the same ground)
Regarding Claim 6
Nia discloses: The method of claim 1, wherein the filter factor of each filter of the plurality of filters includes a value selected between a lower limit value and an upper limit value. ([Para 0063-0065], Nia describes the scaling factor having a range between -t to t.)
Regarding Claim 7
Nia discloses: The method of claim 1, wherein a weight value, a filter factor value, and a combination thereof is stored as any one of: a fixed point value, a floating point value, an integer value, and any combination thereof. ([Para 0083], Nia discloses an integer value. [Para 0012 and 0079] Nia discloses floating-point operations.)
Regarding Claim 8
Nia discloses: The method of claim 1, wherein the trained pruned CNN includes only filters having a filter factor above a predefined threshold. ([Para 0021, 0049, 0056, 0062, 0069, Fig 2, and Fig 5-6] describes Nia only pruning filters with scaling factor that satisfies a predetermined criterion.)
Regarding Claim 10
Nia discloses: The method of claim 1, further comprising: applying a hyperparameter value in training the CNN ([Para 0020 and Claim 5] Nia describes hyperparameters that controls a respective influence of the regularization parameter on the loss function.).
Regarding Claim 11
Nia discloses: The method of claim 1, wherein a loss function of the CNN includes a base loss function and a filter based pruning loss function. ([Para 0063-0065 Para 0072-0073 0080] Nia describes a loss function with weight regularization function and scaling factor regularization function.)
Regarding Claim 18
(Claim 18 recites analogous limitations to claim 6 and therefore is rejected on the same ground as claim 6.)
Regarding Claim 19
(Claim 19 recites analogous limitations to claim 7 and therefore is rejected on the same ground as claim 7.)
Regarding Claim 20
(Claim 20 recites analogous limitations to claim 8 and therefore is rejected on the same ground as claim 8.)
Regarding Claim 22
(Claim 22 recites analogous limitations to claim 10 and therefore is rejected on the same ground as claim 10.)
Regarding Claim 23
(Claim 23 recites analogous limitations to claim 11 and therefore is rejected on the same ground as claim 11.)
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 2-5, and 14-17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nia in view of Koivisto et al. (US 20190251442 A1, hereinafter "Koivisto").
Regarding Claim 2
Nia discloses: The method of claim 1, further comprising: determining a number of single instruction multiple data (SIMD) processing units; removing a number of filters based on the filter factor ([Para 0049])
Nia does not explicitly disclose: such that a second number of remaining filters is a whole multiple of the number of SIMD processing units
However, Koivisto discloses in the same field of endeavor: determining a number of single instruction multiple data (SIMD) processing units; removing a number of filters based on the filter factor, such that a second number of remaining filters is a whole multiple of the number of SIMD processing units. ([Para 0033, 0049, 0076-0077, 0085, 0101, and Fig 5-6] describes optimizing for numbers of channels that are multiples of eight/two the number of filters remaining in each convolutional layer after pruning is a multiple of eight/two.)
It would have been obvious to a person of ordinary skill in art before the effective filling date of the invention to implement the function of Pruning Convolution Neural networks disclosed by Koivisto into the method of Neural Network pruning disclosed by Nia to remove filters based on processing units. The modification would have been obvious because one of the ordinary skills of the art would be motivated to utilize the feature of Pruning Convolution Neural networks disclosed by Koivisto as all the references are in the field of machine learning. A person of ordinary skill of the art would have been motivated to perform the combination for being able to optimize pruning by considering hardware architecture and a number of channels.
Regarding Claim 3
Nia in view of Koivisto discloses: The method of claim 2, further comprising:
selecting a number of second filters, each having a filter factor value which exceeds the predefined threshold; and removing a number of filters, the number of filters equal to a number of filters having a filter factor value below the predefined threshold added to the number of second filters. ([Para 0033, 0049, 0076-0077, 0085, 0101, and Fig 5-6], Koivisto describes optimizing for numbers of channels that are multiples of eight/two the number of filters remaining in each convolutional layer after pruning is a multiple of eight/two.)
Regarding Claim 4
Nia in view of Koivisto discloses: The method of claim 1, further comprising:
determining a number of single instruction multiple data (SIMD) processing units; removing a number of filters based on the filter factor, such that the number of removed filters is a whole multiple of the number of SIMD processing units. ([Para 0033, 0049, 0076-0077, 0085, 0101, and Fig 5-6], Koivisto describes optimizing for numbers of channels that are multiples of eight/two the number of filters remaining in each convolutional layer after pruning is a multiple of eight/two.)
Regarding Claim 5
Nia in view of Koivisto discloses: The method of claim 4, further comprising:
selecting a number of second filters, each having a filter factor value which exceeds the predefined threshold; and removing a number of filters, the number of filters equal to a number of filters having a filter factor value below the predefined threshold added to the number of second filters. ([Para 0033, 0049, 0076-0077, 0085, 0101, and Fig 5-6], Koivisto describes optimizing for numbers of channels that are multiples of eight/two the number of filters remaining in each convolutional layer after pruning is a multiple of eight/two.)
Regarding Claim 14
(Claim 14 recites analogous limitations to claim 2 and therefore is rejected on the same ground as claim 2.)
Regarding Claim 15
(Claim 15 recites analogous limitations to claim 3 and therefore is rejected on the same ground as claim 3.)
Regarding Claim 16
(Claim 16 recites analogous limitations to claim 4 and therefore is rejected on the same ground as claim 4.)
Regarding Claim 17
(Claim 17 recites analogous limitations to claim 5 and therefore is rejected on the same ground as claim 5.)
Claim(s) 9 and 21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nia in view of Li (US 20230162477 A1, hereinafter "Nia").
Regarding Claim 9
Nia discloses: The method of claim 1,
Nia does not explicitly disclose: further comprising: applying a pruning technique only on a predetermined number of layers of the CNN.
However, Li discloses in the same field of endeavor: applying a pruning technique only on a predetermined number of layers of the CNN. ([Para 0074] states “In addition, the model to be pruned and compressed is generally fixed, that is, the number of layers of the student model is determined before training” [Para 0069])
It would have been obvious to a person of ordinary skill in art before the effective filling date of the invention to implement the function of Training a model disclosed by Li into the method of Neural Network pruning disclosed by Nia to prune a predetermined CNN. The modification would have been obvious because one of the ordinary skills of the art would be motivated to utilize the feature of Training a model disclosed by Li as all the references are in the field of machine learning. A person of ordinary skill of the art would have been motivated to perform the combination for being able to evaluate a fixed number of layers for pruning.
Regarding Claim 21
(Claim 21 recites analogous limitations to claim 9 and therefore is rejected on the same ground as claim 9.)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Choe et al. (US 20230259776 A1) describes filter based pruning.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TEWODROS E MENGISTU whose telephone number is (571)270-7714. The examiner can normally be reached Mon-Fri 9:30-5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ABDULLAH KAWSAR can be reached at (571)270-3169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TEWODROS E MENGISTU/ Examiner, Art Unit 2127