Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 20 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim does/do not fall within at least one of the four categories of patent eligible subject matter because the claimed invention falls under signal per se. The claim fails to add "non-transitory" to the CRM and the specification fails to disavow the signals per se by excluding signals.
Step 1: Claim 20 is a signal per se. Therefore, claim 20 does not fall within one of the statutory categories and is not eligible subject matter.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, 4, 9-12, 14, 19, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Ahmed et al, NPL (PCA driven mixed filter pruning for efficient convNets), published January 24, 2022 in view of Partovi Nia et al, U.S. PG PUB (US 20210073643 A1), published September 4, 2020 and Singh et al, NPL (Stability Based Filter Pruning for Accelerating Deep CNNs), published March 7, 2019.
With regard to independent claim 1,
Ahmed teaches, “A computer-implemented method comprising:” (Pg.10, Section: Results and discussion, Subsection: Methods; EN: This denotes that the system is run on an NVIDIA Tesla K80 GPU), “obtaining data specifying an initial neural network configured to perform a machine learning task,” (Pg. 13, Section: Results and Discussion, Subsection: VGG-19 on CIFAR-100; EN: This denotes the use of the CIFAR-100 dataset being used with the VGG-19 deep convolutional neural network). “wherein the initial neural network comprises a plurality of neural network layers,” (Pg. 13, Section: Results and Discussion, Subsection: VGG-19 on CIFAR-100; EN: This denotes that the VGG-19 deep convolutional neural network comprises various types of layers).
“determining, …, a representativeness measure for each of the plurality of filters, wherein the representativeness measure indicates how representative each filter is of all of the plurality of filters;” (: Pg. 5-6, Section: Mixed filter pruning, paragraph 1-2; EN: This denotes the use of PCA (principal component analysis) to analyze a pretrained neural network, compress the number of layers and filters, and to optimize the neural network. This can be used to determine how well the filters can represent other filters).
“determining a central tendency measure for each of the plurality of filters based on processing a batch of network inputs using the initial neural network,” (Pg. 9, Section: Mixed filter pruning, Subsection: Step 1: Pruning filters via PCA, paragraph “Optimizing width and depth of the network.”; EN: This denotes that after the neural network is analyzed by the PCA, significant dimensions are used for each layer to determine the number of filters per layer in the neural network and which filter in the layer contributes the most to the output. This is done by utilizing the mean vector in the PCA). “wherein determining the central tendency measure comprises,…” (Pg. 9, Section: Mixed filter pruning, Subsection: Step 1: Pruning filters via PCA, paragraph “Optimizing width and depth of the network.”; EN: This denotes that after the neural network is analyzed by the PCA, significant dimensions are used for each layer to determine the number of filters per layer in the neural network and which filter in the layer contributes the most to the output. This is done by utilizing the mean vector in the PCA). “and computing the central tendency measure for each of the plurality of filters based on output values included in the layer outputs of the first linear transformation layer for the network inputs in the batch;” (Pg. 9, Section: Mixed filter pruning, subsection: Step 1: Pruning filters via PCA, paragraph “Optimizing width and depth of the network.”; EN: This denotes that after the neural network is analyzed by the PCA, significant dimensions are used for each layer to determine the number of filters per layer in the neural network and which filter in the layer contributes the most to the output. This is done by utilizing the mean vector in the PCA).
“…, based on the representativeness measures and the central tendency measures,…;” (Pg. 5-6, Section: Mixed filter pruning, paragraph 1-2 and Pg. 9, Section: Mixed filter pruning, subsection: Step 1: Pruning filters via PCA, paragraph “Optimizing width and depth of the network.”; EN: This denotes the use of PCA to analyze a pretrained neural network, compress the number of layers and filters, and to optimize the neural network. It also shows how after the neural network is analyzed by the PCA, significant dimensions are used for each layer to determine the number of filters per layer in the neural network and which filter in the layer contributes the most to the output, which is done by utilizing the mean vector in the PCA).
“selecting, …, a proper subset of the plurality of filters;” (Pg. 9, Section: Mixed filter pruning, Subsection Step 2: Pruning filters via geometric median; EN: This denotes the use of geometric median to find redundant filters and prune them). “and generating a pruned neural network configured to perform the machine learning task,” (Pg. 13, section: Results and Discussion, subsection: VGG-19 on CIFAR-100; EN: This denotes pruning the VGG-19 deep convolutional neural network). “wherein the pruned neural network comprises a pruned feed-forward neural network layer having the proper subset of the plurality of filters” (Pg. 13, Section: Results and Discussion, Subsection: VGG-19 on CIFAR-100; EN: This denotes that the pruned VGG-19 deep convolutional neural network has reached its required accuracy and has removed redundant filters).
However, Ahmed fails to disclose “wherein the plurality of neural network layers comprise a feed-forward neural network layer that comprises”, “(i) a first linear transformation layer that has a plurality of first linear transformation parameters followed by”, “(ii) a nonlinear activation layer followed by”, ” from the second linear transformation parameters”, ”…, for each network input in the batch of network inputs: receiving a layer input of the first linear transformation layer;”, “and processing the layer input in accordance with the plurality of first linear transformation parameters to generate a layer output of the first linear transformation layer;”, “determining,…, a cumulative importance score for each of the plurality of filters;”, and “…,based on the cumulative importance scores,…;” .
Partovi Nia teaches, “wherein the plurality of neural network layers comprise a feed-forward neural network layer that comprises” (Paragraph 0004, 0051; EN: This denotes the use of a CNN, which operates as a feed-forward neural network). “(i) a first linear transformation layer that has a plurality of first linear transformation parameters followed by” (Fig. 5, Paragraph 0053-0055,0082-0084; EN: This denotes a convolutional layer making a feature map by using convolution operations, which is configured to operate as a plurality of filters, and a weight matrix). “(ii) a nonlinear activation layer followed by” (Fig. 5, paragraph 0053-0055,0082-0084; EN: This denotes the use of an activation operation such as ReLU). “(iii) a second linear transformation layer that has a plurality of second linear transformation parameters, the plurality of first linear transformation parameters and the plurality of second linear transformation parameters defining a plurality of filters of the feed-forward neural network layer;” (Fig. 5, paragraph 0053-0055,0082-0084; EN: This denotes the use of a second convolutional layer, which uses convolution operations, which is configured to operate as a plurality of filters, and a weight matrix).
“from the second linear transformation parameters” (Fig. 5, paragraph 0053-0055,0082-0084; EN: This denotes the use of a second convolutional layer, which uses convolution operations and is configured to operate as a plurality of filters, and a weight matrix). “…, for each network input in the batch of network inputs: receiving a layer input of the first linear transformation layer;” (paragraph 0051; EN: This denotes that the convolution operations within the convolutional layer will receive an input feature map). “and processing the layer input in accordance with the plurality of first linear transformation parameters to generate a layer output of the first linear transformation layer;” (paragraph 0051; EN: This denotes that the convolution operations within the convolutional layer will receive an input feature map and generate a respective activation map).
Ahmed and Partovi Nia are considered to be analogous to the claimed invention due to the fact that they are both generally related to pruning neural networks. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the mixed filter pruning of Ahmed with the pruning of a neural network block of Partovi Nia. One would be motivated to do so to improve the method of pruning neural networks.
However, Ahmed and Partovi both fail to explicitly disclose “determining,…, a cumulative importance score for each of the plurality of filters;” and “…,based on the cumulative importance scores,…;”
Singh teaches, “determining,…, a cumulative importance score for each of the plurality of filters;” (Pg.1168, Section 3, subsection 3.2.2; EN: This denotes calculating the importance of the filters in each layer and ranking them based on the ratio of the sum of the absolute value of the filters after and before applying the auxiliary loss). “…,based on the cumulative importance scores,…;” (Pg.1168, Section 3, subsection 3.2.2; EN: This denotes calculating the importance of the filters in each layer and ranking them based on the ratio of the sum of the absolute value of the filters after and before applying the auxiliary loss).
Ahmed, Partovi Nia, and Singh are considered to be analogous to the claimed invention due to the fact that they are each generally related to pruning neural networks. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the mixed filter pruning of Ahmed and the pruning of a neural network block of Partovi Nia with the filter pruning through the stability of a filter of Singh. One would be motivated to do so to improve the method of pruning neural networks.
With regard to dependent claim 2,
Partovi Nia teaches, “The method of claim 1, wherein generating the pruned neural network comprises: generating a mask that assigns a non-zero value to each of one or more of the plurality of second linear transformation parameters that define the proper subset of the plurality of filters;” (Paragraph 0023, 0084-085; EN: This denotes the use of a mask function, which can have a value of 1 or 0, alongside a scaling factor. This scaling operation happens after the second convolutional output and when the neural network is being trained). “and applying the mask to a layer output of the second linear transformation layer” (Paragraph 0084-0085; EN: This denotes the use of a mask function, which can have a value of 1 or 0, alongside a scaling factor. This scaling operation happens after the second convolutional output and when the neural network is being trained).
With regard to dependent claim 4,
Partovi Nia teaches, “The method of claim 2, wherein the non-zero values in the mask are one” (Paragraph 0023; EN: This denotes that the mask function can have a value of 1).
With regard to dependent claim 9,
Ahmed teaches, “and selecting, …, the proper subset of the plurality of filters” (Pg. 9, Section: Mixed filter pruning, Subsection Step 2: Pruning filters via geometric median; EN: This denotes the use of geometric median to find redundant filters and prune them).
However, Ahmed and Partovi Nia fails to explicitly teach “The method of claim 1, wherein selecting the proper subset of the plurality of filters comprises: receiving data defining a resource constraint that specify how many computational resources can be consumed by the pruned neural network when performing the machine learning task;”, “generating a ranking of the plurality of filters based on the cumulative importance score for each of the plurality of filters;”, and “…, in accordance with the ranking and the resource constraint, …”.
Singh teaches, “The method of claim 1, wherein selecting the proper subset of the plurality of filters comprises: receiving data defining a resource constraint that specify how many computational resources can be consumed by the pruned neural network when performing the machine learning task;” (Pg.1169, Section 3 and 4, subsection 3.3 and 4.4; EN: This denotes the receiving data in regards to the layers in a CNN and using it to calculate the FLOPS (Floating-Point Operations per inference/training) and TRM (Run Time Memory). It also shows the various other constraints affected by using CNN). “generating a ranking of the plurality of filters based on the cumulative importance score for each of the plurality of filters;” (Pg.1168, Section 3, subsection 3.2.2; EN: This denotes calculating the importance of the filters in each layer and ranking them based on the ratio of the sum of the absolute value of the filters after and before applying the auxiliary loss). “…, in accordance with the ranking and the resource constraint, …” (Pg.1168-1169 and 1172-1173, Section 3 and 4, subsection 3.2.2, 3.3, and 4.4; EN: This denotes the calculation of FLOPS, TRM, filter importance, and filter ranking, while also showing the constraints that occur when using CNN and compressing the CNN).
With regard to dependent claim 10,
Partovi Nia teaches, “The method of claim 1, wherein the nonlinear activation layer comprises a Gaussian error linear unit (GELU) activation layer or a rectified linear unit (RELU) activation layer” (Paragraph 0082-0083; EN: This denotes the use of ReLU).
With regard to independent claim 11,
The rest of this claim is similar in scope to claim 1 and is rejected under a similar rationale.
With regard to dependent claim 12,
This claim is similar in scope to claim 2 and is rejected under a similar rationale.
With regard to dependent claim 14,
This claim is similar in scope to claim 4 and is rejected under a similar rationale.
With regard to dependent claim 19,
This claim is similar in scope to claim 9 and is rejected under a similar rationale.
With regard to independent claim 20,
The rest of this claim is similar in scope to claim 1 and is rejected under a similar rationale.
Claims 3, 5, 13, and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Ahmed et al, NPL (PCA driven mixed filter pruning for efficient convNets), published January 24, 2022 in view of Partovi Nia et al, U.S. PG PUB (US 20210073643 A1), published September 4, 2020 and Singh et al, NPL (Stability Based Filter Pruning for Accelerating Deep CNNs), published March 7, 2019 as applied to claims 1, 2, 11, and 12 above, and further in view of Djokovic et al, U.S. PG PUB (US 20210021866 A1), published September 20, 2020.
With regard to dependent claim 3,
As discussed above, Ahmed in view of Partovi Nia and Singh teaches the computer-implemented methods of claims 1 and 2. Ahmed in view of Partovi Nia and Singh fail to explicitly teach “The method of claim 2, wherein applying the mask comprises determining a Hadamard product between the layer output and the mask”.
Djokovic teaches, “The method of claim 2, wherein applying the mask comprises determining a Hadamard product between the layer output and the mask” (Paragraph 0112-0119; EN: This denotes the use of a Hadamard product in regard to a scaling coefficient matrix).
Ahmed, Partovi Nia, Singh, and Djokovic are considered to be analogous to the claimed invention due to the fact that they are each generally related to pruning neural networks. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the mixed filter pruning of Ahmed, the pruning of a neural network block of Partovi Nia, and the filter pruning through the stability of a filter of Singh with the encoding and decoding of image data of Djokovic. One would be motivated to do so to improve the method of pruning neural networks.
With regard to dependent claim 5,
As discussed above, Ahmed in view of Partovi Nia and Singh teaches the computer-implemented methods of claims 1 and 2. Ahmed in view of Partovi Nia and Singh fail to explicitly teach “The method of claim 2, wherein the non-zero values in the mask are different from each other”.
Djokovic teaches, “The method of claim 2, wherein the non-zero values in the mask are different from each other” (Fig. 15, paragraph 0112-0119; EN: This denotes that the scaling coefficient can be values between 0 and 1, such as 0.2).
The motivation to combine Ahmed, Partovi Nia, Singh, and Djokovic is the same as discussed above with respect to claim 3.
With regard to dependent claim 13,
This claim is similar in scope to claim 3 and is rejected under a similar rationale.
With regard to dependent claim 15,
This claim is similar in scope to claim 5 and is rejected under a similar rationale.
Claims 6 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Ahmed et al, NPL (PCA driven mixed filter pruning for efficient convNets), published January 24, 2022 in view of Partovi Nia et al, U.S. PG PUB (US 20210073643 A1), published September 4, 2020 and Singh et al, NPL (Stability Based Filter Pruning for Accelerating Deep CNNs), published March 7, 2019 as applied to claims 1 and 11 above, and further in view of Deng et al, NPL (PERMDNN: Efficient Compressed DNN Architecture with Permuted Diagonal Matrices), published April 23, 2020.
With regard to dependent claim 6,
As discussed above, Ahmed in view of Partovi Nia and Singh teach the computer-implemented methods of claim 1. Ahmed teaches “…the representativeness measures for the plurality of filters” (Pg. 5-6, Section: Mixed filter pruning, paragraph 1-2; EN: This denotes the use of PCA (principal component analysis) to analyze a pretrained neural network, compress the number of layers and filters, and to optimize the neural network. This can be used to determine how well the filters can represent other filters).
However, Ahmed and Singh both fail to explicitly teach “The method of claim 1, wherein determining the representativeness measure for each of the plurality of filters comprises: generating a coefficient matrix having horizontal and vertical dimensions equal to a number of the plurality of filters of the feed-forward neural network layer;”, “determining updates to coefficients in the coefficient matrix based on minimizing a difference between (i) the plurality of second linear transformation parameters and (ii) a product of the plurality of second linear transformation parameters and coefficient matrix;”, and “and using updated coefficients along a diagonal of the coefficient matrix as…”.
Partovi Nia teaches, “The method of claim 1, wherein determining the representativeness measure for each of the plurality of filters comprises: generating a coefficient matrix having horizontal and vertical dimensions equal to a number of the plurality of filters of the feed-forward neural network layer;” (Paragraph 0007, 0051-0053, ; EN: This denotes the use of a weight matrix, which is utilized by a Matmul (matrix multiplication) operation. This matrix contains a plurality of filters that correspond to a respective set of weights and will utilize an input feature map using that respective filter). “…(i) the plurality of second linear transformation parameters and (ii) a product of the plurality of second linear transformation parameters and coefficient matrix;” (Paragraph 0007, 0051-0053, ,0082-0084; EN: This denotes the use of a weight matrix utilizing a Matmul (matrix multiplication) operation and that the convolution operations within the convolutional layer will receive an input feature map with the respective filters to generate a respective activation map).
However, Ahmed in view of Partovi Nia and Singh fail to explicitly teach “determining updates to coefficients in the coefficient matrix based on minimizing a difference between…” and “and using updated coefficients along a diagonal of the coefficient matrix as the …”.
Deng teaches, “determining updates to coefficients in the coefficient matrix based on minimizing a difference between…” (Pg. 191-193, Section: 3, Subsection: A-D; EN: This denotes updating the weights of the weight matrix and compressing the layers). “and using updated coefficients along a diagonal of the coefficient matrix as …” (Pg. 3-5, Section: 3, Subsection: A-C; EN: This denotes that the structured sparse weight matrix consists of multiple permuted diagonal matrices used as sub-matrices).
Ahmed, Partovi Nia, Singh, and Deng are considered to be analogous to the claimed invention due to the fact that they are each generally related to reducing the size of various neural networks. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the mixed filter pruning of Ahmed, the pruning of a neural network block of Partovi Nia, and the filter pruning through the stability of a filter of Singh with the matrices and compression of layers of Deng. One would be motivated to do so to improve the method of pruning neural networks and reducing the size of those same neural networks.
With regard to dependent claim 16,
This claim is similar in scope to claim 6 and is rejected under a similar rationale.
Claims 7 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Ahmed et al, NPL (PCA driven mixed filter pruning for efficient convNets), published January 24, 2022, Partovi Nia et al, U.S. PG PUB (US 20210073643 A1), published September 4, 2020, Singh et al, NPL (Stability Based Filter Pruning for Accelerating Deep CNNs), published March 7, 2019, and Deng et al, NPL (PERMDNN: Efficient Compressed DNN Architecture with Permuted Diagonal Matrices) as applied to claims 1, 6, 11, and 16 above, and further in view of Lao et al, NPL (Regression and Classification of Alzheimer’s Disease Diagnosis Using NMF-TDNet Features From 3D Brain MR Image), published September 20, 2021.
With regard to dependent claim 7,
As discussed above, Ahmed in view of Partovi Nia, Singh, and Deng teaches the computer-implemented methods of claim 1 and 6. However, Ahmed in view of Partovi Nia and Deng fail to explicitly teach “The method of claim 6, wherein determining updates to coefficients in the coefficient matrix comprises: applying a non-negative matrix factorization (NMF) update rule, a semi-NMF update rule, or nonnegative least square update rule”.
Lao teaches, “The method of claim 6, wherein determining updates to coefficients in the coefficient matrix comprises: applying a non-negative matrix factorization (NMF) update rule, a semi-NMF update rule, or nonnegative least square update rule” (Pg.1106, Section 4, Subsection B; EN: This denotes the use of NMF with regards to a multiplication update rule).
Ahmed, Partovi Nia, Singh, Deng, and Lao are considered to be analogous to the claimed invention due to the fact that they are each generally related to reducing the size and dimensionality of various neural networks. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the mixed filter pruning of Ahmed, the pruning of a neural network block of Partovi Nia, the filter pruning through the stability of a filter of Singh, and the matrices and compression of layers of Deng with the data dimensionality reduction by utilizing NMF-TDNet of Lao. One would be motivated to do so to improve the method of pruning neural networks and reducing the size and dimensionality of those same neural networks.
With regard to dependent claim 17,
This claim is similar in scope to claim 7 and is rejected under a similar rationale.
Claims 8 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Ahmed et al, NPL (PCA driven mixed filter pruning for efficient convNets), published January 24, 2022 in view of Partovi Nia et al, U.S. PG PUB (US 20210073643 A1), published September 4, 2020 and Singh et al, NPL (Stability Based Filter Pruning for Accelerating Deep CNNs), published March 7, 2019 as applied to claims 1 and 11 above, and further in view of Yu et al, NPL (Accelerating Deep Unsupervised Domain Adaptation with Transfer Channel Pruning), published September 30, 2019.
With regard to dependent claim 8,
As discussed above, Ahmed, in view of Partovi Nia and Singh teach the computer-implemented methods of claim 1. However, Ahmed in view of Partovi Nia fail to explicitly teach “The method of claim 1, wherein processing the batch of network inputs using the initial neural network comprises: obtaining the batch of network inputs from an unlabeled dataset”.
Yu teaches, “The method of claim 1, wherein processing the batch of network inputs using the initial neural network comprises: obtaining the batch of network inputs from an unlabeled dataset” (Pg.6-7, Section 4, Subsection C; EN: This denotes the use of an unlabeled dataset).
Ahmed, Partovi Nia, Singh, and Yu are considered to be analogous to the claimed invention due to the fact that they are each generally related to pruning neural networks. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the mixed filter pruning of Ahmed, the pruning of a neural network block of Partovi Nia, and the filter pruning through the stability of a filter of Singh with the transfer channel pruning of Yu. One would be motivated to do so to improve the method of pruning neural networks.
With regard to dependent claim 18,
This claim is similar in scope to claim 8 and is rejected under a similar rationale.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SETH CAPRIANO-UMARI SHELTON whose telephone number is (571)270-0213. The examiner can normally be reached 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Ell can be reached at (571) 270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SETH CAPRIANO-UMARI SHELTON/Examiner, Art Unit 2141
/BEN M RIFKIN/Primary Examiner, Art Unit 2123
Prosecution Insights