Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement(s) (IDS) filed on 02/16/2023 and 10/17/2025 fail to comply with 37 CFR 1.98(a)(2), which requires a legible copy of each cited foreign patent document; each non-patent literature publication or that portion which caused it to be listed; and all other information or that portion which caused it to be listed. It has been placed in the application file, but the information referred to therein has not been considered.
Priority
Acknowledgment is made of applicant’s claim for priority to PCT/CN2017/095621 filed August 2, 2017:
“[0001] This divisional application claims priority to U.S. Application No. 16/633,071, filed January 22, 2020, which claims priority under 35 U.S.C. 371 to International Application No. PCT/CN2017/095621 filed, August 2, 2017, entitled SYSTEM AND METHOD ENABLING ONE- HOT NEURAL NETWORKS ON A MACHINE LEARNING COMPUTE PLATFORM. The entire contents of which are hereby incorporated by reference herein.”
Status of Claims
The present application is being examined under the claims filed on 10/17/2025.
Claims 1-20 are rejected.
Claims 1-20 are pending.
Response to Arguments – 35 U.S.C. § 101
Applicant remarks:
“Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to abstract idea without significantly more. Applicant respectfully disagrees. However, applicant respectfully submits that this rejection is overcome by way of amendment, as the amended claims are clearly directed towards patentable subject matter.”
Examiner response:
Applicant’s arguments filed have been fully considered but they are not persuasive. Refer to the updated patentable subject matter rejections in this document.
Response to Arguments – 35 U.S.C. § 103
Applicant remarks:
“Claims 1-7, 9-15, and 17-19 are rejected under 35 U.S.C. 103 as being unpatentable over Courbariaux in view of Xu. Applicant respectfully disagrees. However, to advance prosecution, amended claims are presented herein.
As to amended claim 1, applicant respectfully submits that the cited combination of references fails to teach or suggest, at the least, a "graphics processing unit" including a "graphics core including circuitry having an instruction set architecture with support for operands in a one-hot encoding format," and to "perform a forward compute pass with mini batch samples to compute a loss function via execution of an instruction to perform a matrix operation, the instruction having an operand including at least a portion of the set of one-hot coded weights." The cited combination of references do not address a GPU with an ISA with support for one-hot encoded operands or a matrix operation instruction with a one-hot encoded operand.”
Examiner response:
Applicant’s arguments filed have been fully considered but they are not persuasive. While the prior art on record at the time of the previous office action (Courbariaux, Xu, Wan) does not explicitly teach an ISA with support for one-hot encoding and matrix operations, further search and consideration has yielded reference Bharati which does teach this. Refer to the updated claim mappings and motivations to combine art in the prior rejection section of this document.
Prior Art References
The short names that are used to identify the references of prior art in the analysis that follows are:
Short Name
Reference
Courbariaux
Courbariaux, M., Hubara, I., Soudry, D., El-Yaniv, R. and Bengio, Y., 2016. Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or-1. arXiv preprint arXiv:1602.02830.
Xu
Xu, X., Lu, Q., Wang, T., Liu, J., Hu, Y. and Shi, Y., 2017, July. Efficient hardware implementation of cellular neural networks with powers-of-two based incremental quantization. In Proceedings of the Neuromorphic Computing Symposium (pp. 1-10).
Wan
Wan, Y. and Wey, C.L., 1998, May. Efficient algorithms for binary logarithmic conversion and addition. In 1998 IEEE International Symposium on Circuits and Systems (ISCAS) (Vol. 5, pp. 233-236). IEEE.
Bharati
Bharati, K.S. and Jhunjhunwala, A., 2015, May. Implementation of machine learning applications on a fixed-point DSP. In 2015 IEEE 28th Canadian Conference on Electrical and Computer Engineering (CCECE) (pp. 1458-1463). IEEE.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to abstract idea without significantly more. This judicial exception is not integrated into a practical application as outlined in the 2-step analyses for each claim that follows.
In reference to claim 1.
(Step 1) Is the claim to a process, machine, manufacture or composition of matter?
Yes, a machine - “1. A graphics processing unit comprising:“.
(Step 2A Prong 1) Does the claim recite an abstract idea, law of nature, or natural phenomenon?
- "generate an approximate weight matrix including a set of one-hot coded weights;" which, but for the inclusion of generic computing equipment (i.e., graphics core, cache memory), is an evaluation that may be performed mentally by a human with the aid of pen and paper (refer to MPEP 2106.04(a)(2)(III)(C) for more information about mental processes being performed on a computer).
(Step 2A Prong 2) Does the claim recite additional elements that integrate the judicial exception into a practical application?
No, consider the following elements:
- “a cache memory; and a graphics core coupled with the cache memory, the graphics core including circuitry having an instruction set architecture with support for operands in a one-hot encoding format, the circuitry configured to:” which merely recites the words apply it (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f). The added specificity of the amended claim language for support for one-hot encoding does not preclude generic computing equipment.
- “perform a forward compute pass with mini batch samples to compute a loss function via execution of an instruction to perform a matrix operation, the instruction having an operand including at least a portion of the set of one-hot coded weights;” which merely recites the words apply it (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f). The added specificity of performing a matrix operation does not preclude the use of generic computing equipment to carry out a mental process.
- “perform a backward compute pass to compute a gradient update via stochastic gradient descent according to a loss update;” which merely recites the words apply it (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).
- “and update the approximate weight matrix based on the gradient update to generate an updated weight matrix.” which merely recites the words apply it (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).
(Step 2B) Does the claim recite additional elements that amount to significantly more than the judicial exception?
No, consider the following elements:
- “a cache memory; and a graphics core coupled with the cache memory, the graphics core including circuitry configured to:” which merely recites the words apply it (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).
- “perform a forward compute pass with mini batch samples to compute a loss function;” which merely recites the words apply it (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).
- “perform a backward compute pass to compute a gradient update via stochastic gradient descent according to a loss update;” which merely recites the words apply it (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).
- “and update the approximate weight matrix based on the gradient update to generate an updated weight matrix.” which merely recites the words apply it (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).
Thus, the claim is subject matter ineligible.
In reference to claim 2.
(Step 1) Is the claim to a process, machine, manufacture or composition of matter?
Yes, a machine - “2. The graphics processing unit as in claim 1,“.
(Step 2A Prong 1) Does the claim recite an abstract idea, law of nature, or natural phenomenon?
- “the circuitry configured to clip values in the updated weight matrix to between negative one and one after an update of the approximate weight matrix.“ which, but for the inclusion of generic computing equipment (i.e., circuitry), is an evaluation that may be performed mentally by a human with the aid of pen and paper (refer to MPEP 2106.04(a)(2)(III)(C) for more information about mental processes being performed on a computer).
Step 2A Prong 2 and Step 2B:
No additional elements are recited. Thus, the claim is subject matter ineligible.
In reference to claim 3.
(Step 1) Is the claim to a process, machine, manufacture or composition of matter?
Yes, a machine - “3. The graphics processing unit as in claim 1,“.
(Step 2A Prong 1) Does the claim recite an abstract idea, law of nature, or natural phenomenon?
Yes, it inherits the abstract idea of the parent claim.
(Step 2A Prong 2) Does the claim recite additional elements that integrate the judicial exception into a practical application?
No, consider the following elements:
- “the circuitry configured to store the approximate weight matrix in the cache memory.” which merely recites the words apply it (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).
(Step 2B) Does the claim recite additional elements that amount to significantly more than the judicial exception?
No, consider the following elements:
- “the circuitry configured to store the approximate weight matrix in the cache memory.” which merely recites the words apply it (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).
Thus, the claim is subject matter ineligible.
In reference to claim 4.
(Step 1) Is the claim to a process, machine, manufacture or composition of matter?
Yes, a machine - “4. The graphics processing unit as in claim 1,“.
(Step 2A Prong 1) Does the claim recite an abstract idea, law of nature, or natural phenomenon?
- “the circuitry configured to generate the approximate weight matrix based on an initialized weight matrix having weights between negative one and one.“ which, but for the inclusion of generic computing equipment (i.e., circuitry), is an evaluation that may be performed mentally by a human with the aid of pen and paper (refer to MPEP 2106.04(a)(2)(III)(C) for more information about mental processes being performed on a computer).
Step 2A Prong 2 and Step 2B:
No additional elements are recited. Thus, the claim is subject matter ineligible.
In reference to claim 5.
(Step 1) Is the claim to a process, machine, manufacture or composition of matter?
Yes, a machine - “5. The graphics processing unit as in claim 1,“.
(Step 2A Prong 1) Does the claim recite an abstract idea, law of nature, or natural phenomenon?
- “the circuitry configured to generate the approximate weight matrix based on a pre-trained weight matrix having weights between negative one and one.“ which, but for the inclusion of generic computing equipment (i.e., circuitry), is an evaluation that may be performed mentally by a human with the aid of pen and paper (refer to MPEP 2106.04(a)(2)(III)(C) for more information about mental processes being performed on a computer).
Step 2A Prong 2 and Step 2B:
No additional elements are recited. Thus, the claim is subject matter ineligible.
In reference to claim 6.
(Step 1) Is the claim to a process, machine, manufacture or composition of matter?
Yes, a machine - “6. The graphics processing unit as in claim 5,“.
(Step 2A Prong 1) Does the claim recite an abstract idea, law of nature, or natural phenomenon?
- "the weights of the pre-trained weight matrix quantized to between negative one and one." which, but for the inclusion of generic computing equipment (i.e., graphics processor), is an evaluation that may be performed mentally by a human with the aid of pen and paper (refer to MPEP 2106.04(a)(2)(III)(C) for more information about mental processes being performed on a computer).
Step 2A Prong 2 and Step 2B:
No additional elements are recited. Thus, the claim is subject matter ineligible.
In reference to claim 7.
(Step 1) Is the claim to a process, machine, manufacture or composition of matter?
Yes, a machine - “7. The graphics processing unit as in claim 6,“.
(Step 2A Prong 1) Does the claim recite an abstract idea, law of nature, or natural phenomenon?
- “each one-hot coded weight in the set of one-hot coded weights including a sign bit and a power value, the circuitry configured to determine the power value for a one-hot coded weight.“ which, but for the inclusion of generic computing equipment (i.e., graphics processor), is an evaluation that may be performed mentally by a human with the aid of pen and paper (refer to MPEP 2106.04(a)(2)(III)(C) for more information about mental processes being performed on a computer).
Step 2A Prong 2 and Step 2B:
No additional elements are recited. Thus, the claim is subject matter ineligible.
In reference to claim 8.
(Step 1) Is the claim to a process, machine, manufacture or composition of matter?
Yes, a machine - “8. The graphics processing unit as in claim 7,“.
(Step 2A Prong 1) Does the claim recite an abstract idea, law of nature, or natural phenomenon?
- “the circuitry configured to determine a nearest neighbor to a weight value via a look-up table to determine the power value.“ which, but for the inclusion of generic computing equipment (i.e., circuitry), is an evaluation that may be performed mentally by a human with the aid of pen and paper (refer to MPEP 2106.04(a)(2)(III)(C) for more information about mental processes being performed on a computer).
Step 2A Prong 2 and Step 2B:
No additional elements are recited. Thus, the claim is subject matter ineligible.
In reference to claim 9.
(Step 1) Is the claim to a process, machine, manufacture or composition of matter?
Yes, a method - “9. A method of performing machine learning operations, the method comprising:“.
(Step 2A Prong 1) Does the claim recite an abstract idea, law of nature, or natural phenomenon?
- "generating an approximate weight matrix including a set of one-hot coded weights via a graphics processing unit including circuitry having an instruction set architecture with support for operands in a one-hot encoding format;" which is an evaluation that may be performed mentally by a human with the aid of pen and paper. Thus, the limitation recites a mental process and therefore an abstract idea. The added specificity of the amended claim language for support for one-hot encoding does not preclude generic computing equipment.
(Step 2A Prong 2) Does the claim recite additional elements that integrate the judicial exception into a practical application?
No, consider the following elements:
- “performing a forward compute pass with mini batch samples to compute a loss function, including executing an instruction via the graphic processing unit to perform a matrix operation, the instruction having an operand including at least a portion of the set of one-hot coded weights;” which merely recites the words apply it (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f). The added specificity of performing a matrix operation does not preclude the use of generic computing equipment to carry out a mental process.
- “performing a backward compute pass to compute a gradient update via stochastic gradient descent according to a loss update;” which merely recites the words apply it (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).
- “and updating the approximate weight matrix based on the gradient update to generate an updated weight matrix.” which merely recites the words apply it (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).
(Step 2B) Does the claim recite additional elements that amount to significantly more than the judicial exception?
No, consider the following elements:
- “performing a forward compute pass with mini batch samples to compute a loss function;” which merely recites the words apply it (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).
- “performing a backward compute pass to compute a gradient update via stochastic gradient descent according to a loss update;” which merely recites the words apply it (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).
- “and updating the approximate weight matrix based on the gradient update to generate an updated weight matrix.” which merely recites the words apply it (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).
Thus, the claim is subject matter ineligible.
In reference to claim 10.
(Step 1) Is the claim to a process, machine, manufacture or composition of matter?
Yes, a method - “10. The method as in claim 9,“.
(Step 2A Prong 1) Does the claim recite an abstract idea, law of nature, or natural phenomenon?
- “additionally comprising clipping values in the updated weight matrix to between negative one and one after updating the approximate weight matrix.“ which is an evaluation that may be performed mentally by a human with the aid of pen and paper. Thus, the limitation recites a mental process and therefore an abstract idea.
Step 2A Prong 2 and Step 2B:
No additional elements are recited. Thus, the claim is subject matter ineligible.
In reference to claim 11.
(Step 1) Is the claim to a process, machine, manufacture or composition of matter?
Yes, a method - “11. The method as in claim 9,“.
(Step 2A Prong 1) Does the claim recite an abstract idea, law of nature, or natural phenomenon?
- “additionally comprising generating an approximate weight matrix based on an initialized weight matrix having weights between negative one and one.“ which is an evaluation that may be performed mentally by a human with the aid of pen and paper. Thus, the limitation recites a mental process and therefore an abstract idea.
Step 2A Prong 2 and Step 2B:
No additional elements are recited. Thus, the claim is subject matter ineligible.
In reference to claim 12.
(Step 1) Is the claim to a process, machine, manufacture or composition of matter?
Yes, a method - “12. The method as in claim 9,“.
(Step 2A Prong 1) Does the claim recite an abstract idea, law of nature, or natural phenomenon?
- “additionally comprising generating an approximate weight matrix based on a pre-trained weight matrix having weights between negative one and one.“ which is an evaluation that may be performed mentally by a human with the aid of pen and paper. Thus, the limitation recites a mental process and therefore an abstract idea.
Step 2A Prong 2 and Step 2B:
No additional elements are recited. Thus, the claim is subject matter ineligible.
In reference to claim 13.
(Step 1) Is the claim to a process, machine, manufacture or composition of matter?
Yes, a method - “13. The method as in claim 12,“.
(Step 2A Prong 1) Does the claim recite an abstract idea, law of nature, or natural phenomenon?
- "the weights of the pre-trained weight matrix quantized to between negative one and one." which is an evaluation that may be performed mentally by a human with the aid of pen and paper. Thus, the limitation recites a mental process and therefore an abstract idea.
Step 2A Prong 2 and Step 2B:
No additional elements are recited. Thus, the claim is subject matter ineligible.
In reference to claim 14.
(Step 1) Is the claim to a process, machine, manufacture or composition of matter?
Yes, a method - “14. The method as in claim 13,“.
(Step 2A Prong 1) Does the claim recite an abstract idea, law of nature, or natural phenomenon?
- "wherein each one-hot coded weight in the set of one-hot coded weights include a sign bit and a power value." which is an evaluation that may be performed mentally by a human with the aid of pen and paper. Thus, the limitation recites a mental process and therefore an abstract idea.
Step 2A Prong 2 and Step 2B:
No additional elements are recited. Thus, the claim is subject matter ineligible.
In reference to claim 15.
(Step 1) Is the claim to a process, machine, manufacture or composition of matter?
Yes, a method - “15. The method as in claim 14,“.
(Step 2A Prong 1) Does the claim recite an abstract idea, law of nature, or natural phenomenon?
- "additionally comprising determining the power value for a one-hot coded weight." which is an evaluation that may be performed mentally by a human with the aid of pen and paper. Thus, the limitation recites a mental process and therefore an abstract idea.
Step 2A Prong 2 and Step 2B:
No additional elements are recited. Thus, the claim is subject matter ineligible.
In reference to claim 16.
(Step 1) Is the claim to a process, machine, manufacture or composition of matter?
Yes, a method - “16. The method as in claim 15,“.
(Step 2A Prong 1) Does the claim recite an abstract idea, law of nature, or natural phenomenon?
- "wherein determining the power value includes determining a nearest neighbor to a weight value via a look-up table." which is an evaluation that may be performed mentally by a human with the aid of pen and paper. Thus, the limitation recites a mental process and therefore an abstract idea.
Step 2A Prong 2 and Step 2B:
No additional elements are recited. Thus, the claim is subject matter ineligible.
In reference to claim 17.
(Step 1) Is the claim to a process, machine, manufacture or composition of matter?
Yes, a machine - “17. A data processing system comprising:“.
(Step 2A Prong 1) Does the claim recite an abstract idea, law of nature, or natural phenomenon?
- "generate an approximate weight matrix including a set of one-hot coded weights;" which, but for the inclusion of generic computing equipment (i.e., memory device, graphics processor), is an evaluation that may be performed mentally by a human with the aid of pen and paper (refer to MPEP 2106.04(a)(2)(III)(C) for more information about mental processes being performed on a computer).
(Step 2A Prong 2) Does the claim recite additional elements that integrate the judicial exception into a practical application?
No, consider the following elements:
- “a memory device; and a graphics processing unit coupled with the memory device, the graphics processing unit comprising a graphics core including circuitry configured to:” which merely recites the words apply it (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).
- “perform a forward compute pass with mini batch samples to compute a loss function via execution of an instruction to perform a matrix operation, the instruction having an operand including at least a portion of the set of one-hot coded weights;” which merely recites the words apply it (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f). The added specificity of performing a matrix operation does not preclude the use of generic computing equipment to carry out a mental process.
- “perform a backward compute pass to compute a gradient update via stochastic gradient descent according to a loss update;” which merely recites the words apply it (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).
- “and update the approximate weight matrix based on the gradient update to generate an updated weight matrix.” which merely recites the words apply it (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).
(Step 2B) Does the claim recite additional elements that amount to significantly more than the judicial exception?
No, consider the following elements:
- “a memory device; and a graphics processing unit coupled with the memory device, the graphics processing unit comprising a graphics core including circuitry having an instruction set architecture with support for operands in a one-hot encoding format, the circuitry configured to:” which merely recites the words apply it (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f). The added specificity of the amended claim language for support for one-hot encoding does not preclude generic computing equipment.
- “perform a forward compute pass with mini batch samples to compute a loss function;” which merely recites the words apply it (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).
- “perform a backward compute pass to compute a gradient update via stochastic gradient descent according to a loss update;” which merely recites the words apply it (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).
- “and update the approximate weight matrix based on the gradient update to generate an updated weight matrix.” which merely recites the words apply it (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f).
Thus, the claim is subject matter ineligible.
In reference to claim 18.
(Step 1) Is the claim to a process, machine, manufacture or composition of matter?
Yes, a machine - “18. The data processing system as in claim 17,“.
(Step 2A Prong 1) Does the claim recite an abstract idea, law of nature, or natural phenomenon?
- “the circuitry configured to generate the approximate weight matrix based on an initialized or pre-trained weight matrix having weights between negative one and one.“ which, but for the inclusion of generic computing equipment (i.e., circuitry), is an evaluation that may be performed mentally by a human with the aid of pen and paper (refer to MPEP 2106.04(a)(2)(III)(C) for more information about mental processes being performed on a computer).
Step 2A Prong 2 and Step 2B:
No additional elements are recited. Thus, the claim is subject matter ineligible.
In reference to claim 19.
(Step 1) Is the claim to a process, machine, manufacture or composition of matter?
Yes, a machine - “19. The data processing system as in claim 18,“.
(Step 2A Prong 1) Does the claim recite an abstract idea, law of nature, or natural phenomenon?
- “each one-hot coded weight in the set of one-hot coded weights including a sign bit and a power value, the circuitry configured to determine the power value for a one-hot coded weight.“ which, but for the inclusion of generic computing equipment (i.e., circuitry), is an evaluation that may be performed mentally by a human with the aid of pen and paper (refer to MPEP 2106.04(a)(2)(III)(C) for more information about mental processes being performed on a computer).
Step 2A Prong 2 and Step 2B:
No additional elements are recited. Thus, the claim is subject matter ineligible.
In reference to claim 20.
(Step 1) Is the claim to a process, machine, manufacture or composition of matter?
Yes, a machine - “20. The data processing system as in claim 19,“.
(Step 2A Prong 1) Does the claim recite an abstract idea, law of nature, or natural phenomenon?
- “the circuitry configured to determine a nearest neighbor to a weight value via a look-up table to determine the power value.“ which, but for the inclusion of generic computing equipment (i.e., circuitry), is an evaluation that may be performed mentally by a human with the aid of pen and paper (refer to MPEP 2106.04(a)(2)(III)(C) for more information about mental processes being performed on a computer).
Step 2A Prong 2 and Step 2B:
No additional elements are recited. Thus, the claim is subject matter ineligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103.
Courbariaux, Xu, Bharati
Claims 1-7, 9-15, and 17-19 are rejected under 35 U.S.C. 103 as being unpatentable over Courbariaux in view of Xu in further view of Bharati.
In reference to claim 1.
- “1. A graphics processing unit comprising:” (preamble)
Courbariaux teaches:
- “perform a forward compute pass with mini batch samples to compute a loss function (Courbariaux Algorithm 1, “Forward propagation”, “C is the cost function for minibatch”);”
PNG
media_image1.png
881
369
media_image1.png
Greyscale
- “perform a backward compute pass to compute a gradient update via stochastic gradient descent according to a loss update (Courbariaux Algorithm 1, “Backward propagation”);”
- “and update the approximate weight matrix based on the gradient update to generate an updated weight matrix (Courbariaux Algorithm 1, “Update”).”
Xu teaches:
- “a cache memory; and a graphics core coupled with the cache memory, the graphics core including circuitry configured to (Xu 2, “To tackle the computation challenge, CeNN accelerations on digital platforms such as ASICs, GPUs and FPGAs have been explored, with FPGA among the most popular choices due to its high flexibility and low time-to-market.”¸ Examiner notes that one of ordinary skill in the art would interpret GPU as a graphics module with some amount of onboard memory to serve as a cache distinct from the shared system memory.”):”
- “generate an approximate weight matrix including a set of one-hot coded weights (Xu Figure 3A, The approximate weight matrix is represented by powers of 2 and is thus one-hot encoded);”
PNG
media_image2.png
238
524
media_image2.png
Greyscale
Motivation to combine Courbariaux with Xu.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Courbariaux and Xu. Courbariaux discloses binarized neural networks wherein weights and activations are constrained to +/- 1. Xu discloses an efficient hardware implementation specifically geared towards cellular neural networks. One would be motivated to combine these references because both disclosures are concerned with efficient hardware implementations and one of ordinary skill in the art could reasonably expect success in combining the methodologies. Further, MPEP 2143 sets forth the Supreme Court rationales for obviousness including: (C) Use of known technique to improve similar devices (methods, or products) in the same way.
Bharati teaches:
- “having an instruction set architecture with support for operands in a one-hot encoding format, the circuitry […] via execution of an instruction to perform a matrix operation, the instruction having an operand including at least a portion of the set of one-hot coded weights”
(Bharati 1458, “The entire implementation is carried out using fixed point assembly instruction set in Blackfin-533 DSP.”)
(Bharati 1459, “In the training phase, the network is trained with known data and labels. The weights and biases are estimated by back propagation through gradient descent algorithm [10]. In the testing phase, the data is classified into one of the classes. The network uses the parameters obtained from the training phase to classify the test data. The testing phase is same as recognition, where the trained models are used to classify the data. Here, the features are propagated from the input to hidden layers and finally to the output layer. A one hot encoding scheme (encoding scheme in which there is a single high bit and rest all others are low) is usually preferred and hence the index of the node containing the maximum value at the output is declared as the classified output.”)
(Bharati 1460, “The matrix vector product is best optimized by making use of the SIMD instruction shown above.”)
Motivation to combine Courbariaux, Xu with Bharati.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Courbariaux, Xu and Bharati. Courbariaux, Xu discloses binarized neural networks wherein weights and activations are constrained to +/- 1 and an efficient hardware implementation specifically geared towards cellular neural networks. Bharati discloses an optimized system for implementing machine learning models on a specific instruction set architecture. One would be motivated to combine these references because both disclosures are concerned with efficient hardware implementations and one of ordinary skill in the art could reasonably expect success in combining the methodologies. Further, MPEP 2143 sets forth the Supreme Court rationales for obviousness including: (C) Use of known technique to improve similar devices (methods, or products) in the same way.
In reference to claim 2.
- “2. The graphics processing unit as in claim 1,” (preamble)
Courbariaux teaches:
- “the circuitry configured to clip values in the updated weight matrix to between negative one and one after an update of the approximate weight matrix (Courbariaux Algorithm 1, “Clip”, Clip is called in the algorithm with a min and max of -1 and 1 respectively and is called on the updated weight matrix.).”
In reference to claim 3.
- “3. The graphics processing unit as in claim 1,” (preamble)
Xu teaches:
- “the circuitry configured to store the approximate weight matrix in the cache memory (Xu 2, “To tackle the computation challenge, CeNN accelerations on digital platforms such as ASICs, GPUs and FPGAs have been explored, with FPGA among the most popular choices due to its high flexibility and low time-to-market.”¸ Examiner notes that one of ordinary skill in the art would interpret GPU as a graphics module with some amount of onboard memory to serve as a cache distinct from the shared system memory.”).”
In reference to claim 4.
- “4. The graphics processing unit as in claim 1,” (preamble)
Xu teaches:
- “the circuitry configured to generate the approximate weight matrix based on an initialized weight matrix having weights between negative one and one (Xu Figure 3A, The approximate weight matrix is being generated on an initialized matrix. Although the figure cited is operating on values that are outside the claimed [-1, 1] range, the circuitry in question is configured to generate the approximate weight matrix on a initialized matrix wherein the weights are constrained to that range since the algorithm in the figure is able to operate on an arbitrary weight matrix.).”
In reference to claim 5.
- “5. The graphics processing unit as in claim 1,” (preamble)
Xu teaches:
- “the circuitry configured to generate the approximate weight matrix based on a pre-trained weight matrix having weights between negative one and one (Xu Figure 3A, The approximate weight matrix is being generated on an initialized matrix. Although the figure cited is operating on values that are outside the claimed [-1, 1] range, the circuitry in question is configured to generate the approximate weight matrix on a pretrained matrix wherein the weights are constrained to that range since the algorithm in the figure is able to operate on an arbitrary weight matrix.).”
In reference to claim 6.
- “6. The graphics processing unit as in claim 5,” (preamble)
Courbariaux teaches:
- “the weights of the pre-trained weight matrix quantized to between negative one and one (Courbariaux Equation (1), The cited equation describes one such quantization scheme that exists in the art for mapping arbitrary real valued weights to the range [-1, 1].).“
PNG
media_image3.png
51
299
media_image3.png
Greyscale
In reference to claim 7.
- “7. The graphics processing unit as in claim 6,” (preamble)
Xu teaches:
- “each one-hot coded weight in the set of one-hot coded weights including a sign bit and a power value, the circuitry configured to determine the power value for a one-hot coded weight (Xu Figure 3A, The approximate weight matrix is represented by powers of 2 and includes signed values in the encoding).”
In reference to claim 9.
- “9. A method of performing machine learning operations, the method comprising:” (preamble)
Courbariaux teaches:
- “performing a forward compute pass with mini batch samples to compute a loss function (Courbariaux Algorithm 1, “Forward propagation”, “C is the cost function for minibatch”);”
- “performing a backward compute pass to compute a gradient update via stochastic gradient descent according to a loss update (Courbariaux Algorithm 1, “Backward propagation”);”
- “and updating the approximate weight matrix based on the gradient update to generate an updated weight matrix (Courbariaux Algorithm 1, “Update”).”
Xu teaches:
- “generating an approximate weight matrix including a set of one-hot coded weights via a graphics processing unit (Xu Figure 3A, The approximate weight matrix is represented by powers of 2 and is thus one-hot encoded);”
Bharati teaches:
- “including circuitry having an instruction set architecture with support for operands in a one-hot encoding format; performing […] including executing an instruction via the [graphics] processing unit to perform a matrix operation, the instruction having an operand including at least a portion of the set of one-hot coded weights;”
(Bharati 1458, “The entire implementation is carried out using fixed point assembly instruction set in Blackfin-533 DSP.”)
(Bharati 1459, “In the training phase, the network is trained with known data and labels. The weights and biases are estimated by back propagation through gradient descent algorithm [10]. In the testing phase, the data is classified into one of the classes. The network uses the parameters obtained from the training phase to classify the test data. The testing phase is same as recognition, where the trained models are used to classify the data. Here, the features are propagated from the input to hidden layers and finally to the output layer. A one hot encoding scheme (encoding scheme in which there is a single high bit and rest all others are low) is usually preferred and hence the index of the node containing the maximum value at the output is declared as the classified output.”)
(Bharati 1460, “The matrix vector product is best optimized by making use of the SIMD instruction shown above.”)
In reference to claim 10.
- “10. The method as in claim 9,” (preamble)
Courbariaux teaches:
- “additionally comprising clipping values in the updated weight matrix to between negative one and one after updating the approximate weight matrix (Courbariaux Algorithm 1, “Clip”, Clip is called in the algorithm with a min and max of -1 and 1 respectively and is called on the updated weight matrix.).”
In reference to claim 11.
- “11. The method as in claim 9,” (preamble)
Xu teaches:
- “additionally comprising generating an approximate weight matrix based on an initialized weight matrix having weights between negative one and one (Xu Figure 3A, The approximate weight matrix is being generated on an initialized matrix. Although the figure cited is operating on values that are outside the claimed [-1, 1] range, the circuitry in question is configured to generate the approximate weight matrix on a initialized matrix wherein the weights are constrained to that range.).”
In reference to claim 12.
- “12. The method as in claim 9,” (preamble)
Xu teaches:
- “additionally comprising generating an approximate weight matrix based on a pre-trained weight matrix having weights between negative one and one (Xu Figure 3A, The approximate weight matrix is being generated on an initialized matrix. Although the figure cited is operating on values that are outside the claimed [-1, 1] range, the circuitry in question is configured to generate the approximate weight matrix on a pretrained matrix wherein the weights are constrained to that range.).”
In reference to claim 13.
- “13. The method as in claim 12,” (preamble)
Courbariaux teaches:
- “the weights of the pre-trained weight matrix quantized to between negative one and one (Courbariaux Equation (1), The cited equation describes one such quantization scheme that exists in the art for mapping arbitrary real valued weights to the range [-1, 1].).”
In reference to claim 14.
- “14. The method as in claim 13,” (preamble)
Xu teaches:
- “wherein each one-hot coded weight in the set of one-hot coded weights include a sign bit and a power value (Xu Figure 3 A, The approximate weight matrix is represented by powers of 2 and includes signed values in the encoding).”
In reference to claim 15.
- “15. The method as in claim 14,” (preamble)
- “additionally comprising determining the power value for a one-hot coded weight (Xu Algorithm 1, “neighbor = log2|uq|;”).”
PNG
media_image4.png
398
469
media_image4.png
Greyscale
In reference to claim 17.
- “17. A data processing system comprising: a memory device;” (preamble)
Courbariaux teaches:
- “perform a forward compute pass with mini batch samples to compute a loss function (Courbariaux Algorithm 1, “Forward propagation”, “C is the cost function for minibatch”);”
- “perform a backward compute pass to compute a gradient update via stochastic gradient descent according to a loss update (Courbariaux Algorithm 1, “Backward propagation”);”
- “and update the approximate weight matrix based on the gradient update to generate an updated weight matrix (Courbariaux Algorithm 1, “Update”).”
Xu teaches:
- “and a graphics processing unit coupled with the memory device, the graphics processing unit comprising a graphics core including circuitry configured to (Xu 2, “To tackle the computation challenge, CeNN accelerations on digital platforms such as ASICs, GPUs and FPGAs have been explored, with FPGA among the most popular choices due to its high flexibility and low time-to-market.”¸ Examiner notes that one of ordinary skill in the art would interpret GPU as a graphics module with some amount of onboard memory to serve as a cache distinct from the shared system memory.”):”
- “generate an approximate weight matrix including a set of one-hot coded weights (Xu Figure 3A, The approximate weight matrix is represented by powers of 2 and is thus one-hot encoded);”
Bharati teaches:
- “having an instruction set architecture with support for operands in a one-hot encoding format, the circuitry […] via execution of an instruction to perform a matrix operation, the instruction having an operand including at least a portion of the set of one-hot coded weights”
(Bharati 1458, “The entire implementation is carried out using fixed point assembly instruction set in Blackfin-533 DSP.”)
(Bharati 1459, “In the training phase, the network is trained with known data and labels. The weights and biases are estimated by back propagation through gradient descent algorithm [10]. In the testing phase, the data is classified into one of the classes. The network uses the parameters obtained from the training phase to classify the test data. The testing phase is same as recognition, where the trained models are used to classify the data. Here, the features are propagated from the input to hidden layers and finally to the output layer. A one hot encoding scheme (encoding scheme in which there is a single high bit and rest all others are low) is usually preferred and hence the index of the node containing the maximum value at the output is declared as the classified output.”)
(Bharati 1460, “The matrix vector product is best optimized by making use of the SIMD instruction shown above.”)
In reference to claim 18.
- “18. The data processing system as in claim 17,” (preamble)
Xu teaches:
- “the circuitry configured to generate the approximate weight matrix based on an initialized or pre-trained weight matrix having weights between negative one and one (Xu Figure 3A, The approximate weight matrix is being generated on an initialized matrix. Although the figure cited is operating on values that are outside the claimed [-1, 1] range, the circuitry in question is configured to generate the approximate weight matrix on a pretrained matrix wherein the weights are constrained to that range.).”
In reference to claim 19.
- “19. The data processing system as in claim 18,” (preamble)
Xu teaches:
- “each one-hot coded weight in the set of one-hot coded weights including a sign bit and a power value, the circuitry configured to determine the power value for a one-hot coded weight (Xu Figure 3A, The approximate weight matrix is represented by powers of 2 and includes signed values in the encoding).”
Courbariaux, Xu, Bharati, Wan
Claims 8, 16, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Courbariaux in view of Xu in further view of Bharati in further view of Wan.
In reference to claim 8.
- “8. The graphics processing unit as in claim 7,” (preamble)
Xu teaches:
- “the circuitry configured to determine a nearest neighbor to a weight value (Xu Algorithm 1, “neighbor = log2|uq|;”) [via a look-up table to determine the power value.]”
Wan teaches:
- “[the circuitry configured to determine a nearest neighbor to a weight value] via a look-up table to determine the power value (Wan 233, “A number of binary logarithm conversion algorithms have been developed. The existing conversion algorithms can be roughly classified into two categories: Look-up table approach and Computation approach”).”
Motivation to combine Courbariaux, Xu, Bharati with Wan.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Courbariaux, Xu, Bharati and Wan. Courbariaux, Xu, Bharati discloses a number of algorithms that implement binary arithmetic. Wan discloses efficient algorithms for binary logarithmic conversion and addition. One would be motivated to combine these references because the disclosure of Wan provides an efficient implementation of some of the functionality required by Courbariaux, Xu, and Bharati. Further, MPEP 2143 sets forth the Supreme Court rationales for obviousness including: (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention.
In reference to claim 16.
- “16. The method as in claim 15,” (preamble)
Wan teaches:
- “wherein determining the power value includes determining a nearest neighbor to a weight value via a look-up table (Wan 233, “A number of binary logarithm conversion algorithms have been developed. The existing conversion algorithms can be roughly classified into two categories: Look-up table approach and Computation approach”).”
In reference to claim 20.
- “20. The data processing system as in claim 19,” (preamble)
Xu teaches:
- “the circuitry configured to determine a nearest neighbor to a weight value (Xu Algorithm 1, “neighbor = log2|uq|;”) [via a look-up table to determine the power value].”
Wan teaches:
- “[the circuitry configured to determine a nearest neighbor to a weight value] via a look-up table to determine the power value (Wan 233, “A number of binary logarithm conversion algorithms have been developed. The existing conversion algorithms can be roughly classified into two categories: Look-up table approach and Computation approach”).”
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CODY RYAN GILLESPIE whose telephone number is (571)272-1331. The examiner can normally be reached M-F, 8 AM - 5 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Viker A Lamardo can be reached at 5172705871. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CODY RYAN GILLESPIE/Examiner, Art Unit 2147
/VIKER A LAMARDO/Supervisory Patent Examiner, Art Unit 2147