Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
1. Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The independent claims recite performing a mathematical algorithm that reuses values from a first round of computations (forward propagation) in a subsequent round of computations (backward propagation). This judicial exception is not integrated into a practical application. The mere recitation of a system comprising generic processors fails to render the claim statutory as the additional element of using a processing device to perform the mathematical operation amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Therefore, the claim is not patent eligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
2. Claims 1, 3-5, 9, 11-13, 17, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Shen et al (US 2022/0230422, herein Shen) in view of Ling et al (US 2017/0103298).
Regarding claim 1, Shen teaches a computer-implemented method for normalizing data in a reconfigurable dataflow processor, the method comprising:
conducting layer computations in a forward-propagation pass ([0096], [0098], forward propagation layer);
saving selected computed values (xHat) from the layer computations ([0007], [0045], [0162-0163], saving computation results for reuse); and
reusing the selected computed values in a backward-propagation pass ([0007], [0098], [0118], reusing saves values for backward propagation).
Shen fails to teach wherein the computations are layer normalization computations.
Ling teaches a computer-implemented method for normalizing data in a processor comprising conducting layer normalization computations in a forward-propagation pass ([0026], [0050], [0053], normalization layer, [0049], forward propagation computations).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the teachings of Shen and Ling to utilize the computation techniques of Shen for layer normalization operations as taught by Ling. While Shen does not explicitly state that the forward propagation layer may be computing layer normalization operations, Shen does disclose normalization of vector inputs and neural network training including batch normalization (Shen [0063], [0165]). As normalization computations are a routine and conventional aspect of neural network operations in the processor art, the combination would merely entail a simple substitution of known prior art elements to achieve predictable results, and thus would have been obvious to one of ordinary skill in the art.
Regarding claim 3, the combination of Shen and Ling teaches the computer-implemented method of claim 1, further including: partitioning normalization operations including the layered normalization computations into multiple compute stages and intervening buffering stages in the reconfigurable dataflow processor (Shen [0130], [0162], performing operations in multiple stages & Ling [0026], [0029], [0061], multiple stages of computations utilizing buffers, [0006], [0039], runtime configuration).
Regarding claim 4, the combination of Shen and Ling teaches the computer-implemented method of claim 3, further including: configuring the multiple compute stages and the intervening buffering stages (Ling [0039], runtime configuration of neural network accelerator, [0029], [0061], buffering).
Regarding claim 5, the combination of Shen and Ling teaches the computer-implemented method of claim 4, further including: processing data (X) using the multiple compute stages and the intervening buffering stages (Shen [0130], [0162], performing operations in multiple stages & Ling [0029], [0061], buffering).
Claims 9 and 11-13 refer to a medium embodiment of the method embodiment of claims 1 and 3-5. Therefore, the above rejections for claims 1 and 3-5 are applicable to claims 9 and 11-13, respectively.
Claims 17 and 19 refer to a system embodiment of the method embodiment of claims 1 and 5. Therefore, the above rejections for claims 1 and 5 are applicable to claims 17 and 19, respectively.
3. Claims 2, 6-8, 10, 14-16, 18, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Shen and Ling as applied to claims above, and further in view of Labatie et al (US 2023/0098994, herein Labatie).
Regarding claim 2, the combination of Shen and Ling teaches the computer-implemented method of claim 1. Shen and Ling fail to teach wherein: the selected computed values are computed according to the equation xHat = (X - µ) / σ.
Labatie teaches a method for normalizing data wherein selected computed values are computed according to the equation xHat = (X - µ) / σ (Fig 3A, [0016], [0053], layer normalization equation).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the teachings of Shen and Ling to utilize the specific normalization algorithm disclosed by Labatie. While Ling does not explicitly disclose the details of the layer normalization operations computed by the exemplary neural network, one of ordinary skill in the art would understand that Labatie’s exemplary formula and the computations disclosed by both Shen and Ling may may interchanged as necessary to the particular application of the exemplary neural network. As both Shen and Labatie disclose computing forward and backward passes of a neural network algorithm used in machine learning, the combination would merely entail a simple substitution of known prior art elements to achieve predictable results, and thus would have been obvious to one of ordinary skill in the art.
Regarding claim 6, the combination of Shen, Ling, and Labatie teaches the computer-implemented method of claim 5. Shen and Ling fail to teach wherein: the compute stages comprise a centering stage that computes X – µ.
Labatie teaches a method for normalizing data wherein a compute stages comprises a centering stage that computes X – µ (Fig 3A, [0016], [0057], re-centering operation).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the teachings of Shen and Ling to utilize the specific normalization algorithm disclosed by Labatie. While Ling does not explicitly disclose the details of the layer normalization operations computed by the exemplary neural network, one of ordinary skill in the art would understand that Labatie’s exemplary formula and the computations disclosed by both Shen and Ling may may interchanged as necessary to the particular application of the exemplary neural network. As both Shen and Labatie disclose computing forward and backward passes of a neural network algorithm used in machine learning, the combination would merely entail a simple substitution of known prior art elements to achieve predictable results, and thus would have been obvious to one of ordinary skill in the art.
Regarding claim 7, the combination of Shen, Ling, and Labatie teaches the computer-implemented method of claim 5, wherein: the compute stages comprise a normalization stage that computes (X - µ) / σ.
Labatie teaches a method for normalizing data wherein a compute stage comprises a normalization stage that computes (X - µ) / σ (Fig 3A, [0016], [0053], layer normalization equation).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the teachings of Shen and Ling to utilize the specific normalization algorithm disclosed by Labatie. While Ling does not explicitly disclose the details of the layer normalization operations computed by the exemplary neural network, one of ordinary skill in the art would understand that Labatie’s exemplary formula and the computations disclosed by both Shen and Ling may may interchanged as necessary to the particular application of the exemplary neural network. As both Shen and Labatie disclose computing forward and backward passes of a neural network algorithm used in machine learning, the combination would merely entail a simple substitution of known prior art elements to achieve predictable results, and thus would have been obvious to one of ordinary skill in the art.
Regarding claim 8, the combination of Shen, Ling, and Labatie teaches the computer-implemented method of claim 5, wherein: the compute stages comprise a shift-add stage that computes y * ((X - µ) / σ) + β.
Labatie teaches a method for normalizing data wherein a compute stage comprises a shift-add stage that computes y * ((X - µ) / σ) + β (Fig 3A, [0067-0069], [0075], shift and scaling of normalization algorithm).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the teachings of Shen and Ling to utilize the specific normalization algorithm disclosed by Labatie. While Ling does not explicitly disclose the details of the layer normalization operations computed by the exemplary neural network, one of ordinary skill in the art would understand that Labatie’s exemplary formula and the computations disclosed by both Shen and Ling may may interchanged as necessary to the particular application of the exemplary neural network. As both Shen and Labatie disclose computing forward and backward passes of a neural network algorithm used in machine learning, the combination would merely entail a simple substitution of known prior art elements to achieve predictable results, and thus would have been obvious to one of ordinary skill in the art.
Claims 10 and 14-16 refer to a medium embodiment of the method embodiment of claims 2 and 6-8. Therefore, the above rejections for claims 2 and 6-8 are applicable to claims 10 and 14-16, respectively.
Claims 18 and 20 refer to a system embodiment of claims 2 and the combined limitations of 6-8. Therefore, the above rejections for claims 2 and 6-8 are applicable to claims 18 and 20.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Yoo (US 12,645,930) discloses a processor for performing forward and backward propagation.
Sadowski (US 2021/0056403) discloses a processor for performing layer normalization operations include forward and backward passes.
Chilimbi (US 2017/0193361) discloses a processor for performing forward and backward propagation.
Yu (US 2017/0132513) discloses a processor for reusing values between forward and backward propagation.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL J METZGER whose telephone number is (571)272-3105. The examiner can normally be reached Monday-Friday 8:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jyoti Mehta can be reached at 571-270-3995. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL J METZGER/ Primary Examiner, Art Unit 2183
Prosecution Insights