Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119
(a)-(d). The certified copy has been filed in parent Application No. CN202110183936.2, filed on
02/10/2021.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 01/02/2025 and 07/25/2025 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-2, 4-5, 10, 12, 14-16, and 18 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Wang et al. Pub. No.: JP2020107042A.
Regarding Claim 1 Wang teaches A method of model training comprising: obtaining a first neural network model comprising, a first convolutional layer; (Wang, paragraph 0007, teaches the selecting of at least one convolutional layer, which can be the first convolutional layer.)
obtaining a plurality of second neural network models based on the first neural network model, wherein each second neural network model is obtained by replacing the first convolutional layer in the first neural network model with a linear operation that is equivalent to a convolutional layer; (Wang, paragraph 0007 – 0008, teaches a method of generating a new neural network by replacing convolutional neural network layers with other layers with different convolutional neural network layers that can be linear operations. Wang, paragraph 0016, teaches the generation of several different models by changing the structure of the convolutional layer that is put in in place of the old convolutional layer.)
and performing model training on the plurality of second neural network models, to obtain a target neural network model being a neural network model with a highest model precision in a plurality of trained second neural network models. (Wang, paragraph 0008, teaches the model relearning for the plurality of second models based on the new replacement layers of the neural networks and then selecting a model based on the recognition performance of the model (i.e. model precision).)
Regarding Claim 2 Wang teaches The method according to claim 1, wherein a receptive field of the convolutional layer equivalent to the linear operation is less than or equal to a receptive field of the first convolutional layer. (Wang, paragraphs 0020-0023, and 0061-0064, teaches the layers that are replacing the old layers having the same input structure or an input structure that is smaller than the layer that is to be replaced.)
Regarding Claim 4 Wang teaches The method according to claim 1, wherein the linear operation in equivalent to the convolutional layer is different from the first convolutional layer, and linear operations comprised in different second neural network models are different from one another. (Wang, paragraph 0008, teaches the plurality of second models based on the new replacement layers of the neural networks that are all different from the original layer and are all different from each other.)
Regarding Claim 5 Wang teaches The method according to claim 1, wherein the convolutional layer equivalent to the linear operation and the linear operation obtain same processing results when processing same data. (Wang, paragraph 0037, teaches the output of the models with the replaced layers having similar accuracy with the original model meaning that the layers that are replaced are producing the same results as the original layers that were present in the original model.)
Regarding Claim 10 Wang teaches The method according to claim 1, wherein the linear operation comprises a plurality of sub-linear operations, and an operation type of the plurality of sub-linear operations comprises at least one of an addition operation, a null operation, an identity operation, a convolution operation, a batch normalization (BN) operation, or a pooling operation. (Wang, pargraph 0013, 0075, teaches that a pooling layer maybe used in the substitution of layers and sub-layers.)
Regarding Claim 12 Wang teaches A model method of model training, comprising: obtaining a first neural network model comprising: a first convolutional layer used to implement a target task; (Wang, paragraph 0007, teaches the selecting of at least one convolutional layer, which can be the first convolutional layer.) determining a target linear operation for replacing the first convolutional layer based on at least one of: a network structure of the first neural network model, the target task, a network structure of the first neural network model, the target task, and or a location of the first convolutional layer in the first neural network model, wherein the target linear operation is equivalent to a convolutional layer; (Wang, paragraph 0007 – 0008, teaches a method of generating a new neural network by replacing convolutional neural network layers with other layers with different convolutional neural network layers that can be linear operations. Wang, paragraph 0016, teaches the generation of several different models by changing the structure of the convolutional layer that is put in in place of the old convolutional layer. ) obtaining a second neural network model based on the first neural network model, wherein the second neural network model is obtained by replacing the first convolutional layer in the first neural network model with the target linear operation; and performing model training on the second neural network model, to obtain a target neural network model. (Wang, paragraph 0008, teaches the model relearning for the plurality of second models based on the new replacement layers of the neural networks and then selecting a model based on the recognition performance of the model (i.e. model precision).)
Regarding Claim 14 Wang teaches The method according to claim 12, wherein a receptive field of the convolutional layer equivalent to the target linear operation is less than or equal to a receptive field of the first convolutional layer. (Wang, paragraphs 0020-0023, and 0061-0064, teaches the layers that are replacing the old layers having the same input structure or an input structure that is smaller than the layer that is to be replaced.)
Regarding Claim 15 Wang teaches The method according to claim 12, wherein the target linear operation is different from the first convolutional layer. (Wang, paragraph 0008, teaches the plurality of second models based on the new replacement layers of the neural networks that are all different from the original layer and are all different from each other.)
Regarding Claim 16 Wang teaches The method according to claim 12, wherein the convolutional layer equivalent to the target linear operation and the target linear operation obtain same processing results when processing same data. (Wang, paragraph 0037, teaches the output of the models with the replaced layers having similar accuracy with the original model meaning that the layers that are replaced are producing the same results as the original layers that were present in the original model.)
Regarding Claim 18 Wang teaches A model training apparatus, comprising: a processor; and a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to: obtain a first neural network model comprising, a first convolutional layer; (Wang, paragraph 0007, teaches the selecting of at least one convolutional layer, which can be the first convolutional layer.)
obtain a plurality of second neural network models based on the first neural network model, wherein each second neural network model is obtained by replacing the first convolutional layer in the first neural network model with a linear operation that, is equivalent to is equivalent to one a convolutional layer; (Wang, paragraph 0007 – 0008, teaches a method of generating a new neural network by replacing convolutional neural network layers with other layers with different convolutional neural network layers that can be linear operations. Wang, paragraph 0016, teaches the generation of several different models by changing the structure of the convolutional layer that is put in in place of the old convolutional layer.)
and perform model training on the plurality of second neural network models, to obtain a target neural network model being, a neural network model with a highest model precision in a plurality of trained second neural network models. (Wang, paragraph 0008, teaches the model relearning for the plurality of second models based on the new replacement layers of the neural networks and then selecting a model based on the recognition performance of the model (i.e. model precision).)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 3is/are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. Pub. No.: JP2020107042A in view of Guo et al. “ExpandNets: Linear Over-parameterization to Train Compact Convolutional Networks” 2020.
Regarding Claim 3, 6-9, 11, 13, and 17 Wang teaches The method according to claim 1, wherein the linear operation comprises a plurality of operation branches, an input of each operation branch is an input of the linear operation, (Wang, paragraph 0020 – 0023, teaches the inputting of data into the layers of a neural network) 1…or the linear operation comprises an operation branch configured to process input data of the linear operation 2… (Wang, paragraph 0020 – 0023, teaches the inputting of data into the layers of a neural network)
Wang does not teach 1… each operation branch comprises at least one serial sub- linear operation, and an equivalent receptive field of the at least one serial sub-linear operation is less than or equal to the receptive field of the first convolutional layer; However, Guo in analogous art teaches this limitation (Guo, page 3-5, section 3, teaches a method of expanding convolutional neural network layers, meaning that they are splitting the layers by the individual operations that take place within the layer (i.e. operation branch). These sub layers are processed in order making them serial operations (i.e. serial sub-linear operations).)
Further Wang does not teach 2…, the operation branch comprises at least one serial sub- linear operation, and an equivalent receptive field of the at least one serial sub-linear operation is less than or equal to the receptive field of the first convolutional layer. Guo in analogous art teaches this limitation (Guo, page 3-5, section 3, teaches the receptive field of the expanded convolutional layers to be the same or less than the unexpanded convolutional layers)
It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine Guo’s teaching of expanding convolutional layers into separate operations with Wang’s teaching of substitution of layers in a neural network. The motivation to do so would be to help reduce the number of parameters in a neural network and to increase the performance of the neural networks.
Regarding Claim 6 Wang teaches The method according to claim 1, wherein a second neural network model corresponding to the target neural network model is obtained by replacing the first convolutional layer in the first neural network model with a target linear operation, the target neural network model comprises a trained target linear operation; and the method further comprises: (Wang, paragraph 0007 – 0008, teaches a method of generating a new neural network by replacing convolutional neural network layers with other layers with different convolutional neural network layers that can be linear operations. Wang, paragraph 0016, teaches the generation of several different models by changing the structure of the convolutional layer that is put in in place of the old convolutional layer.)
Wang does not teach replacing the trained target linear operation with a second convolutional layer equivalent to the trained target linear operation, to obtain a third neural network model. Guo in analogous art teaches this limitation (Guo, page 3-5, section 3, teaches the ability of the system to contract the expanded layer back into the original layer and retrain the model to create a third neural network model.)
It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine Guo’s teaching of expanding convolutional layers into separate operations with Wang’s teaching of substitution of layers in a neural network. The motivation to do so would be to help reduce the number of parameters in a neural network and to increase the performance of the neural networks.
Regarding Claim 7 the combination of Wang and Guo teaches The method according to claim 6, wherein a size of the second convolutional layer is same as a size of the first convolutional layer. (Guo, page 3-5, section 3, teaches the ability of the system to contract the expanded layer back into the original layer and retrain the model to create a third neural network model. This contracted convolutional layer is the same size as the original convolutional neural network layer.)
Regarding Claim 8 the combination of Wang and Guo teaches The method according to claim 6, further comprising: fusing, based on a data processing sequence of a plurality of sub-linear operations comprised in the trained target linear operation, each sub-linear operation into an adjacent sub-linear operation that follows the sub-linear operation in the data processing sequence, until fusion of a last sub- linear operation in the data processing sequence is completed, to obtain the second convolutional layer. (Guo, page 3-5, section 3, teaches the expanding of convolutional layers into multiple operation layers these operation layers are then multiplied, subtracted, divided, or added together to get an output of each layer until all the layers are eventually combined to produce the final output (i.e. fusion).)
Regarding Claim 9 the combination of Wang and Guo teaches The method according to claim 8, wherein the trained target linear operation comprises a first sub-linear operation and a second sub- linear operation that are adjacent to each other, the second sub-linear operation follows the first sub-linear operation in the data processing sequence, the first sub-linear operation comprises a first operation parameter, and the second sub-linear operation comprises a second operation parameter; (Guo, page 3-5, section 3, teaches the expanding of convolutional layers into the different operations that make up the convolutional layers as a whole. Further Guo teaches the expanding of a convolutional layer as “Y =X∗Fn×m×k×k =X∗F1 p1× m× 3× 3 ∗···∗Fl−1 pl−1×pl−2×3×3 ∗ Fl n× pl−1× 3× 3” where each F can be seen as an operation and those operations are each fused by multiplying them together in a separate operation. These operations are done in the equations order making them serial operations and can be broken up into different equations making branches.)
and fusing each sub-linear operation into the adjacent sub-linear operation that follows the sub-linear operation in the data processing sequence comprises: obtaining a fusion parameter of the first sub-linear operation, wherein if-when input data of the first sub-linear operation is input data of the trained target linear operation, the fusion parameter of the first sub-linear operation is the first operation parameter, or if-when input data of the first sub-linear operation is output data of a third sub-linear operation that is adjacent to the first sub- linear operation and that is followed by the first sub-linear operation in the sequence, the fusion parameter of the first sub-linear operation is obtained based on a fusion parameter of the third sub- linear operation and the first operation parameter; (Guo, page 3-5, section 3, teaches the expanding of a convolutional layer as “Y =X∗Fn×m×k×k =X∗F1 p1× m× 3× 3 ∗···∗Fl−1 pl−1×pl−2×3×3 ∗ Fl n× pl−1× 3× 3” where each F can be seen as an operation and those operations are each fused by multiplying them together in a separate operation. These operations are done in the equations order making them serial operations and can be broken up into different equations making branches.)
and obtaining a fusion parameter of the second sub-linear operation based on the fusion parameter of the first sub-linear operation, the second operation parameter, and an operation type of the second sub-linear operation, wherein if-when the second sub-linear operation is the last sub-linear operation in the data processing sequence, the fusion parameter of the second sub-linear operation is used as an operation parameter of the second convolutional layer. (Guo, page 3-5, section 3, teaches the expanding of a convolutional layer as “Y =X∗Fn×m×k×k =X∗F1 p1× m× 3× 3 ∗···∗Fl−1 pl−1×pl−2×3×3 ∗ Fl n× pl−1× 3× 3” where each F can be seen as an operation and those operations are each fused by multiplying them together in a separate operation. These operations are done in the equations order making them serial operations and can be broken up into different equations making branches. When the final equation/operation is multiplied together the output is then passed to the next convolutional layer in order to continue the process of the neural network)
Regarding Claim 11 the combination of Wang and Guo teaches teaches The method according to claim 9, wherein if-when the operation type of the second sub-linear operation is a convolution operation or a batch normalization (BN) operation, the fusion parameter of the second sub-linear operation is obtained by performing an inner product calculation on the fusion parameter of the first sub-linear operation and the operation parameter of the second sub-linear operation; or when if the operation type of the second sub-linear operation is an addition operation, pooling operation, an identity operation, or a null operation, the fusion parameter of the second sub-linear operation is obtained by performing calculation corresponding to the operation type of the second sub-linear operation on the fusion parameter of the first sub-linear operation. (Guo, page 3-5, section 3, teaches the expanding of a convolutional layer as “Y =X∗Fn×m×k×k =X∗F1 p1× m× 3× 3 ∗···∗Fl−1 pl−1×pl−2×3×3 ∗ Fl n× pl−1× 3× 3” where each F can be seen as an operation and those operations are each fused by multiplying them together in a separate operation. These operations are done in the equations order making them serial operations and can be broken up into different equations making branches.)
Regarding Claim 13 Wang teaches The method according to claim 12, wherein the target linear operation comprises a plurality of sub-linear operations and M operation branches, (Wang, paragraph 0020 – 0023, teaches the inputting of data into the layers of a neural network)
an input of each operation branch is an input of the target linear operation, and the M operation branches meet at least one of the following an input of at least one of a first plurality of sub-linear operations comprised in the M operation branches is an output of a second plurality of sub-linear operations of the plurality of sub-linear operations; quantities of sub-linear operations comprised in at least two of the M operation branches are different from one another; or operation types of sub-linear operations comprised in the at least two of the M operation branches are different from one another. However, Guo in analogous art teaches this limitation (Guo, page 3-5, section 3, teaches a method of expanding convolutional neural network layers, meaning that they are splitting the layers by the individual operations that take place within the layer (i.e. operation branch). These sub layers are processed in order making them serial operations (i.e. serial sub-linear operations). Guo further teaches teaches the expanding of convolutional layers into the different operations that make up the convolutional layers as a whole. Further Guo teaches the expanding of a convolutional layer as “Y =X∗Fn×m×k×k =X∗F1 p1× m× 3× 3 ∗···∗Fl−1 pl−1×pl−2×3×3 ∗ Fl n× pl−1× 3× 3” where each F can be seen as an operation and those operations are each fused by multiplying them together in a separate operation. These operations are done in the equations order making them serial operations and can be broken up into different equations making branches.)
Regarding Claim 17 Wang teaches The method according to claim 12, wherein the target neural network model comprises a trained target linear operation, and the method further comprises: (Wang, paragraph 0007 – 0008, teaches a method of generating a new neural network by replacing convolutional neural network layers with other layers with different convolutional neural network layers that can be linear operations. Wang, paragraph 0016, teaches the generation of several different models by changing the structure of the convolutional layer that is put in in place of the old convolutional layer.)
Wang does not teach replacing the trained target linear operation in the target neural network model with a second convolutional layer equivalent to the trained target linear operation, to obtain a third neural network model. Guo in analogous art teaches this limitation (Guo, page 3-5, section 3, teaches the ability of the system to contract the expanded layer back into the original layer and retrain the model to create a third neural network model.)
It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine Guo’s teaching of expanding convolutional layers into separate operations with Wang’s teaching of substitution of layers in a neural network. The motivation to do so would be to help reduce the number of parameters in a neural network and to increase the performance of the neural networks.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to THOMAS B LANE whose telephone number is (571)272-1872. The examiner can normally be reached M-Th: 6:40am-4:40pm; F: Out of Office.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, MARIELA REYES can be reached at (571) 270-1006. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/THOMAS BERNARD LANE/ Examiner, Art Unit 2142
/HAIMEI JIANG/ Primary Examiner, Art Unit 2142
Prosecution Insights