Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 01/10/2023 and 03/25/2026 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-9 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The term “closer” in claim 1, lines 5 and 13 a relative term which renders the claim indefinite. The term “closer” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. It's not clear what a “layer closer to an input side than the convolution layer” or a “layer closer to an output side than the pooling layer”, since the specification lacks definition. For examination purpose, the term “layer closer to an input side than the convolution layer” will be view as any layer that process the input data before it reaches the convolutional layer and the term “layer closer to an output side than the pooling layer” will be view as any layer that proceed the pooling layer.
Claim 1 recites the limitation "first input data output" in line 4. However, it’s not clear what the "first input data output" is. Is the "first input data output" output generated by a layer or is the "first input data output" input data. For purposes of examination "first input data output" will be view as an output from a previous layer. In addition, claim 1 recites “the first input data and second input data” in line 12. For examination purposes, examiner is interpreting these limitations as “the first input data output and a second input data” in line 12.
Claim 2 recites the limitation “the first input data” in line 5. There is insufficient antecedent basis for this limitation in the claim. For examination purposes, examiner is interpreting the limitations as “the first input data output”.
Claim 3 is dependent on claim 2, and thus are rejected for reasons set forth in the rejection of claim 2.
Claims 2 and 3 are dependent on claim 1, and thus are rejected for reasons set forth in the rejection of claim 1.
Claims 4 and 7 are independent and recites similar elements as claim 1. Therefore, are rejected for reasons set forth in the rejection of claim 1.
Claims 5 and 8 recites similar elements as claim 2. Therefore, are rejected for reasons set forth in the rejection of claim 2.
Claims 5-6 and 8-9 are dependent on claims 4 and 7, and thus are rejected for reasons set forth in the rejection of claim 4 and 7.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-9 are rejected under 35 U.S.C. 103 as being unpatentable over Sarma et al. Exploiting Activation based Gradient Output Sparsity to Accelerate Backpropagation in CNNs (hereinafter Sarma) in further view of Dai et al. SparseTrain: Exploiting Dataflow Sparsity for Efficient Convolutional Neural Networks Training (hereinafter Dai) in further view of Nakahara et al. US 2021/0224640 A1 (hereinafter Nakahara) as cited in the Information Disclosure Statement (IDS) date 01/10/2023.
Regarding claim 1:
Sarma teaches causing a convolution layer included in a network to execute a convolution calculation of forward propagation on first input data output from a layer closer to an input side than the convolution layer; (
PNG
media_image1.png
355
595
media_image1.png
Greyscale
Examiner will like to also emphasize applicant specification [0087] teaches a forward propagation is convolution calculation. Sarma pg. 1, sec: 1 Introduction, para. 2 teaches a CNN architecture is composed of n layers where most of the layers are composed of a weight layer followed by an activation layer. [where] ...a convolution (CONV) acts as the weight layer (W)”, while a Rectified Linear Unit (ReLU) is widely used as an activation function” and Fig. 2 and figure 4 teaches input feature map is process by a ReLU layer (i.e., a layer closer to an input side than the convolution layer ) in the forward pass to generate the output feature map in forward pass (see Fig. 2 above)).
generating, an index in which a position of a non-zero element is set for each predetermined element of the output data; and ( Sarma pg. 6, right col., para. 3-4 teaches necessitating an indexing stage that enables identification of non-zero (NZ) neuron location and teaches “indexing” being performed once per layer such that NZ indexing is being performed on the generated feature map (i.e., output data)).
causing, when the convolution layer is caused to execute a convolution calculation of backward propagation of the first input data and second input data that is output from a layer closer to an output side the convolution layer to execute a convolution calculation of a non-zero element based on the index, and to skip a convolution calculation of a zero element. (Examiner will like to emphasize “back pass”, “backward propagation” and “backpropagation” will be used interchangeably for purposes of examination. Sarma Fig. 2, pg. 2, para. 1, teaches a convolutional layer f3 utilizes sparsity to skip “the zero-valued computation” and pg. 6, right colm., para. 3-5 teaches how the index values are used to skip convolution calculation of a zero element “The NZ index values are used to index into the synapse field, and thereby selective (only non-zero) neuron”. In particular, pg. 7, left colm., para 1 teaches the bitmap location (i.e., index) being used during backpropagation to skip non-zero elements).
Sarma does not explicitly teaches A non-transitory computer-readable recording medium storing a training program for causing a computer to execute processing comprising: generating, when a pooling layer included in the network is caused to execute a pooling calculation of forward propagation on output data that serves as an execution result of the convolution calculation, an index and causing, when the convolution layer is caused to execute a convolution calculation of backward propagation of the first input data and second input data that is output from a layer closer to an output side than the pooling layer, the convolution layer to execute a convolution calculation based on the index.
Nonetheless Dai teaches the following:
generating, when a pooling layer included in the network is caused to execute a pooling calculation of forward propagation on output data that serves as an execution result of the convolution calculation, an index in which a position of a non-zero element is set for each predetermined element of the output data; and (Dai pg. 2, left colm. sec: II Preliminary, para. 1-3, continuing on right colm., para 1 teaches a MaxPool layer (i.e., pooling layer) is being executed in a Forward Stage to generate non-zero patterns that are being record as mask (i.e., index)).
causing, when the convolution layer is caused to execute a convolution calculation of backward propagation of the first input data and second input data that is output from a layer closer to an output side than the pooling layer, the convolution layer to execute a convolution calculation of a non-zero element based on the index,
PNG
media_image2.png
27
176
media_image2.png
Greyscale
. In addition, Dai pg, 4, left colm, par. 1; pg. 2, left colm. para. 3 teaches non-zero patterns (i.e., non-zero element) generated based on the execution result of the convolution calculation (i.e., forward propagation) are stored as mask (i.e., index) and are adopted in backward stage (i.e., backward propagation)).
Dai is also in the same field of endeavor as Sarma and Nakahara (Sparsity in machine learning). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include the functionality of a forward stage that include input data and kernel as well a back stage that updates the kernel using SDG, as being disclosed and taught by Dai, in the system taught by Sarma and Nakahara to yield the predictable results of “improve CNN training speed and efficiency significantly by exploiting different types of sparsity” (see Dai pg. 6, left colm., para. 1).
Neither Sarma or Dai teaches A non-transitory computer-readable recording medium storing a training program for causing a computer to execute processing comprising: causing ...the convolution layer to execute a convolution calculation of a non-zero element based on the index, the first input data, and the second input data, and to skip a convolution calculation of a zero element.
However, Nakahara teaches the following:
A non-transitory computer-readable recording medium storing a training program for causing a computer to execute processing comprising: (Nakahara [0206] teaches a storage medium).
generating...an index in which a position of a non-zero element is set... ( Nakahara Fig. 4B, Fig. 7A and [0087] teaches an index (idx) is associated with the position of nonzero weight (i.e., nonzero element) is set).
causing ...the convolution layer to execute a convolution calculation of a non-zero element based on the index, the first input data, and the second input data, and to skip a convolution calculation of a zero element ( Nakahara [0193] teaches skipping a weight Wi having a zero weight (i.e., skip a convolution calculation of a zero element) and performing a convolution operation based on a nonzero weight (i.e., second input data) and an input value Xi (i.e., first input data ) corresponding to the nonzero weight. In particular, [0086-0087] teaches “an operation in which a zero weight is skipped, the nonzero convolution operation circuit 21: reads a nonzero weight of interest and a relative address corresponding thereto from the weight/address memory 213”... More specifically, the weight/address memory 213 stores therein a nonzero weight w1, . . . and a relative address adr1, . . . , for each index idx”, thus Nakahara teaches the convolution calculation of a non-zero element based on the nonzero weight that includes an index and the input value).
Nakahara is also in the same field of endeavor as Sarma and Dai (Sparsity in machine learning). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include the functionality of a index and convolutional calculations, as being disclosed and taught by Nakahara, in the system taught by Sarma and Dai to yield the predictable results of “drastically reduce an absolute number of multiply-accumulate operations in performing a convolution operation, thus allowing a reduction in an amount of memory and a high-speed calculation time” ([0194]).
Regarding claim 2:
Nakahara, Sarma and Takahashi teach The non-transitory computer-readable recording medium according to claim 1. Nakahara specifically teaches wherein the processing of causing the convolution calculation of the non-zero element to be executed and causing the convolution calculation of the zero element to be skipped causes a convolution calculation of an element at a position of each predetermined element of the first input data and an element of the second input data, which correspond to the position of the non-zero element set in the index, to be executed ( Nakahara [0086-0087] teaches performing the convolution calculation of a non-zero element based on the nonzero weight (i.e., second input data) that includes an index and the input value (i.e., first input data)).
Regarding claim 3:
Sarma, Dai and Nakahara teach The non-transitory computer-readable recording medium according to claim 2. Nakahara specifically teaches for causing the computer to execute the processing further comprising: causing the convolution layer to execute a convolution calculation of forward propagation of a kernel and the first input data; and (Nakahara Fig. 3 and [0081] teaches causing convolution operation of an input feature map (i.e., first input data) and a kernel to be performed).
While Nakahara teaches “when relearning is necessary, a position or a value is updated”. Neither Sarma or Nakahara teach updating the kernel based on an execution result of the convolution calculation of the non-zero element.
Nonetheless, Dai teaches the following:
causing the convolution layer to execute a convolution calculation of forward propagation of a kernel and the first input data; and (For purpose of examination, the terms filter, kernel and weight will be used interchangeably. Dai pg. 2, left colm. Sec: Preliminary, para. 2, teaches the convolutional layer during the forward stage execute a convolutional calculation of a convolutional kernel (i.e., kernel) and the input activations (i.e., first input data). And Dai teaches the convolutional layer can be represented as
PNG
media_image3.png
93
445
media_image3.png
Greyscale
. Where
w
i
,
j
is the kernel and
I
j
is the first input data).
updating the kernel based on an execution result of the convolution calculation of the non-zero element ( Applicant specification [0087] teaches a forward propagation is convolution calculation for which Dai pg. 2, left colm. para. 1-4 teaches non-zero patterns (non-zero element) generated based on the execution result of the convolution calculation (forward propagation) are stored as mask and will be adapted in backward stage to update the kernel using Stochastic gradient descent (SGD)).
Regarding claim 4: Claim 4 is rejected under the same rational of claim 1. Claim 4 only recites the additional elements of A training method, for which Nakahara invention relates to a technique of a neural network processing method (see Nakahara [0002]).
Regarding claim 5: is a training method claim comprising limitations similar to those of claim 2, therefore is rejected under the same rational of claim 2.
Regarding claim 6: is a training method claim comprising limitations similar to those of claim 3, therefore is rejected under the same rational of claim 3.
Regarding claim 7: Claim 7 is rejected under the same rational of claim 1. Claim 7 only recites the additional elements of An information processing apparatus, for which Nakahara Fig. 20 element 201 teaches an information processing apparatus i.e., a computer.
Regarding claim 8: is an information processing apparatus claim comprising limitations similar to those of claim 2, therefore is rejected under the same rational of claim 2.
Regarding claim 9: is an information processing apparatus claim comprising limitations similar to those of claim 3, therefore is rejected under the same rational of claim 3.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GISEL G FACCENDA whose telephone number is (703)756-1919. The examiner can normally be reached Monday - Friday 8:00 am - 4:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abdullah Al Kawsar can be reached at (571) 270-3169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/G.G.F./Examiner, Art Unit 2127
/ABDULLAH AL KAWSAR/Supervisory Patent Examiner, Art Unit 2127
Prosecution Insights