DETAILED ACTION
This Office Action is responsive to the Applicant’s submission, filed on October 20, 2025, amending claims 20, 24, 28, 30-35 and 38. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 20-38 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea (e.g. a mental process and/or mathematical concept) without significantly more. As described in MPEP § 2106, the analyses as to whether a claim qualifies as eligible subject matter under 35 U.S.C. § 101 includes the following determinations:
(1) Whether the claim is to a statutory category, i.e. to a process, machine, manufacture or composition of matter (“Step 1”) – see MPEP §§ 2106, subsection III, and 2106.03
(2) If the claim is to a statutory category, whether the claim recites any judicial exceptions, including certain groupings of abstract ideas (i.e., mathematical concepts, certain methods of organizing human activity, or mental processes) (“Step 2A, Prong One”) – see MPEP §§ 2106, subsection III, and 2106.04
(3) If the claim recites a judicial exception, whether the claim recites additional elements that integrate the judicial exception into a practical application (“Step 2A, Prong Two”) – see MPEP §§ 2106, subsection III, and 2106.04
(4) If the claim does not recite additional elements that integrate the judicial exception into a practical application, whether the claim recites additional elements that amount to significantly more than the judicial exception (“Step 2B”) – see MPEP §§ 2106, subsection III, and 2106.05
Claim 20
Regarding “Step 1,” independent claim 20 is to a statutory category as claim 20 is directed to a data processing system comprising a circuit, which can be considered a machine, manufacture, or composition of matter.
Accordingly, the analysis proceeds to “Step 2A, Prong One” to determine if the claim recites a judicial exception. In this case, claim 20 recites a mathematical concept and thus recites a judicial exception. In particular the limitation for “implementing a neural network architecture…to receive a binary network input and, in dependence on the binary network input, propagate signals…via a plurality of processing nodes….in accordance with respective binary weights, to form a binary network output,” is considered a recitation of a mathematical concept. The forward-propagation of inputs through a neural network circuit is essentially a mathematical algorithm (see e.g. paragraphs 0006 and 0009). Additionally, the limitation for “train[ing] a first node of the plurality of processing nodes by implementing an error correcting function to identify a set of binary weights which minimize, for a given binary input to the first node, any error between a binary output of the first node when formed in accordance with current binary weights of the first node and a preferred output from the first node, and to update the current binary weights of the first node to be the identified set of binary weights,” is also considered a recitation of a mathematical concept. Such training entails performing mathematical calculations and other mathematical concepts (see e.g. paragraphs 0063-0073 of Applicant’s specification as filed).
Because claim 20 recites a judicial exception (i.e. a mathematical concept), the analysis proceeds to “Step2A, Prong Two.” But here the claim does not recite additional elements that integrate the judicial exception into a practical application. In particular, in addition to the above-noted mathematical concepts, claim 20 recites that the neural network architecture and training are implemented on “a circuit.” However, this represents no more than mere instructions to apply the judicial exception on a computer, and therefore does not integrate the judicial exception into a practical application. See MPEP § 2106.05(f).
Accordingly, as claim 20 does not recite additional elements that integrate the judicial exception into a practical application, the analysis proceeds to “Step 2B” to determine whether the claims recite additional elements that amount to significantly more than the judicial exception. However, in this case, the claim does not. As noted above, in addition to the above-noted mathematical concepts, claim 20 recites that the neural network architecture and training are implemented on “a circuit.” However, as further noted above, this represents no more than mere instructions to apply the judicial exception on a computer. This limitation therefore cannot be considered an additional element that amounts to significantly more than the judicial exception. See MPEP § 2106.05(f).
Consequently, claim 20 recites an abstract idea but does not include additional elements that integrate the abstract idea into a practical application or that amount to significantly more than the abstract idea. As a result, and for the reasons described above, claim 20 is rejected as being patent ineligible under 35 U.S.C. § 101.
Claim 21
In claim 21, the limitations for “form[ing] a node output that is binary and functionally equal to a sum in a binary field of the following: a first weight; the sum in a binary field of products of each of bits of a node input and a respective one of a set of second weights; and a value indicative of whether all the bits of the node input are non-zero” is considered a recitation of a mathematical concept. Besides this mathematical concept, claim 21 recites “receive a node input dependent on the binary network input.” However, this additional limitation is considered insignificant extra-solution activity, i.e. mere data gathering (See MPEP § 2106.5(g)), and therefore does not integrate the abstract idea into a practical application. Such data gathering is also well-understood, routine and conventional. See, e.g., Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014). Accordingly, claim 21 fails to recite any additional elements that integrate the abstract idea into a practical application or that amount to significantly more than the abstract idea, and as a result, claim 21 is also patent ineligible under 35 U.S.C. § 101.
Claim 22
In claim 22, the limitation “wherein the first node is configured to form a node output that is functionally equal to a coset of a first-order Reed-Muller code, wherein a coset kernel takes as input the bits of the node input,” is considered recitation of a mathematical concept. Accordingly, claim 22 fails to recite any additional elements (beyond abstract ideas) that integrate the abstract idea into a practical application or that amount to significantly more than the abstract idea, and as a result, claim 22 is also patent ineligible under 35 U.S.C. § 101.
Claim 23
Claim 23 recites general circuitry configured (i.e. via logical AND and XOR functions) to implement a node of the binary neural network. Such limitations represent no more than mere instructions to apply the judicial exception on a computer, and therefore do not integrate the judicial exception into a practical application or amount to significantly more than the judicial exception. See MPEP § 2106.05(f). Accordingly, claim 23 fails to recite any additional elements that integrate the abstract idea into a practical application or that amount to significantly more than the abstract idea, and as a result, claim 23 is also patent ineligible under 35 U.S.C. § 101.
Claim 24
Claim 24 recites general circuitry (i.e. first, second and third logic circuits) configured to implement the binary neural network. Such limitations represent no more than mere instructions to apply the judicial exception on a computer, and therefore do not integrate the judicial exception into a practical application or amount to significantly more than the judicial exception. See MPEP § 2106.05(f). Accordingly, claim 24 fails to recite any additional elements that integrate the abstract idea into a practical application or that amount to significantly more than the abstract idea, and as a result, claim 24 is also patent ineligible under 35 U.S.C. § 101.
Claim 25
Claim 25 describes the weights used in the neural network. Such weights are considered a characteristic of the mathematical concept (i.e. neural network) noted above in claim 21, and therefore do not integrate the abstract idea into a practical application or amount to significantly more than the abstract idea.
Claim 26
Claim 26 describes the weights used in the neural network. Such weights are considered a characteristic of the mathematical concept (i.e. neural network) noted above in claim 21, and therefore do not integrate the abstract idea into a practical application or amount to significantly more than the abstract idea.
Claim 27
Claim 27 recites characteristics of the general circuitry (i.e. the third logic circuit) configured to implement the binary neural network. Such limitations represent no more than mere instructions to apply the judicial exception on a computer, and therefore do not integrate the judicial exception into a practical application or amount to significantly more than the judicial exception. See MPEP § 2106.05(f). Accordingly, claim 27 fails to recite any additional elements that integrate the abstract idea into a practical application or that amount to significantly more than the abstract idea, and as a result, claim 27 is also patent ineligible under 35 U.S.C. § 101.
Claim 28
Claim 28 describes the weights used in the neural network. Such weights are considered a characteristic of the mathematical concept (i.e. neural network) noted above in claim 21, and therefore do not integrate the abstract idea into a practical application or amount to significantly more than the abstract idea.
Claim 29
Claim 29 describes the weights used in the neural network. Such weights are considered a characteristic of the mathematical concept (i.e. neural network) noted above in claim 21, or are a mathematic concept themselves, and therefore do not integrate the abstract idea into a practical application or amount to significantly more than the abstract idea.
Claim 30
Claim 30 recites characteristics of the general circuitry (i.e. a plurality of sub-systems) configured to implement the binary neural network. Such limitations represent no more than mere instructions to apply the judicial exception on a computer, and therefore do not integrate the judicial exception into a practical application or amount to significantly more than the judicial exception. See MPEP § 2106.05(f). Accordingly, claim 30 fails to recite any additional elements that integrate the abstract idea into a practical application or that amount to significantly more than the abstract idea, and as a result, claim 30 is also patent ineligible under 35 U.S.C. § 101.
Claim 31
Claim 31 recites characteristics of the general circuitry (i.e. a plurality of sub-systems) configured to implement the binary neural network. Such limitations represent no more than mere instructions to apply the judicial exception on a computer, and therefore do not integrate the judicial exception into a practical application or amount to significantly more than the judicial exception. See MPEP § 2106.05(f). Accordingly, claim 31 fails to recite any additional elements that integrate the abstract idea into a practical application or that amount to significantly more than the abstract idea, and as a result, claim 31 is also patent ineligible under 35 U.S.C. § 101.
Claim 32
In claim 32, the limitations for “forming a set of values representing, for each node input, whether the expected node output is zero, one, or indifferent” and for “adapting the current binary weights for the first node in dependence on the identification” are considered recitations of an abstract idea, particularly, a mental process. “’[T]he mental processes’ abstract idea grouping in particular is defined as concepts performed in the human mind, and examples of mental processes include observations, evaluations, judgements, and opinions.” MPEP § 2106.04(a)(2), subsection III. “If a claim recites a limitation that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper, the limitation falls within the mental processes grouping, and the claims recites an abstract idea.” MPEP § 2106.04(a)(2), subsection III,B (citations omitted). Also in claim 32, the limitation for “identifying an ith row of a Hadamard matrix that best matches the set of values” is considered a recitation of a mathematical concept. Accordingly, claim 32 fails to recite any additional elements (beyond abstract ideas) that integrate the abstract idea into a practical application or that amount to significantly more than the abstract idea, and as a result, claim 32 is also patent ineligible under 35 U.S.C. § 101.
Claim 33
In claim 33, the limitation for “form[ing] a set of values representing, for each node input, whether a respective node output obtained after the current binary weights have been adapted matches an expected node output” is considered a recitation of a mathematical concept (or a mental process). Accordingly, claim 33 fails to recite any additional elements (beyond abstract ideas) that integrate the abstract idea into a practical application or that amount to significantly more than the abstract idea, and as a result, claim 33 is also patent ineligible under 35 U.S.C. § 101.
Claim 34
In claim 34, the limitation for “adapt[ing] weights for a node in dependence on a set of expected node outputs expected for a set of node inputs by operating a fast Walsh-Hadamard function taking as input the set of expected node outputs” is considered a recitation of a mathematical concept. Accordingly, claim 34 fails to recite any additional elements (beyond abstract ideas) that integrate the abstract idea into a practical application or that amount to significantly more than the abstract idea, and as a result, claim 34 is also patent ineligible under 35 U.S.C. § 101.
Claim 35
Regarding “Step 1,” independent claim 35 is to a statutory category as claim 35 is directed to a communication terminal, which can be considered a machine or manufacture.
Accordingly, the analysis proceeds to “Step 2A, Prong One” to determine if the claim recites a judicial exception. In this case, claim 35 recites a mathematical concept and thus recites a judicial exception. In particular the limitation for “implementing a neural network architecture…to receive the binary network input and, in dependence on the binary network input, propagate signals…via a plurality of processing nodes….in accordance with respective binary weights, to form a binary network output,” is considered a recitation of a mathematical concept. The forward-propagation of inputs through a neural network circuit is essentially a mathematical algorithm (see e.g. paragraphs 0006 and 0009). Additionally, the limitation for “train[ing] a first node of the plurality of processing nodes by implementing an error correcting function to identify a set of binary weights which minimize, for a given binary input to the first node, any error between a binary output of the first node when formed in accordance with current binary weights of the first node and a preferred output from the first node, and to update current the binary weights of the first node to be the identified set of binary weights,” is also considered a recitation of a mathematical concept. Such training entails performing mathematical calculations and other mathematical concepts (see e.g. paragraphs 0063-0073 of Applicant’s specification as filed).
Because claim 35 recites a judicial exception (i.e. a mathematical concept), the analysis proceeds to “Step2A, Prong Two.” But here the claim does not recite additional elements that integrate the judicial exception into a practical application. In particular, in addition to the above-noted mathematical concepts, claim 35 recites that the neural network architecture and training are implemented on “a circuit” of a data processing system included within a communication terminal that also comprises a sensor configured to sense data and form a binary network input. However, this represents no more than mere instructions to apply the judicial exception on a computer, and therefore does not integrate the judicial exception into a practical application. See MPEP § 2106.05(f).
Accordingly, as claim 35 does not recite additional elements that integrate the judicial exception into a practical application, the analysis proceeds to “Step 2B” to determine whether the claims recite additional elements that amount to significantly more than the judicial exception. However, in this case, the claim does not. As noted above, in addition to the above-noted mathematical concept, claim 35 recites that the neural network architecture and training are implemented on “a circuit” of a data processing system included within a communication terminal that also comprises a sensor configured to sense data and form a binary network input. However, as further noted above, this represents no more than mere instructions to apply the judicial exception on a computer. This limitation therefore cannot be considered an additional element that amounts to significantly more than the judicial exception. See MPEP § 2106.05(f).
Consequently, claim 35 recites an abstract idea but does not include additional elements that integrate the abstract idea into a practical application or that amount to significantly more than the abstract idea. As a result, and for the reasons described above, claim 35 is rejected as being patent ineligible under 35 U.S.C. § 101.
Claim 36
Claim 36 recites characteristics of the communication terminal (i.e. that it comprises a camera) configured to implement the binary neural network. Such limitations represent no more than mere instructions to apply the judicial exception on a computer, and therefore do not integrate the judicial exception into a practical application or amount to significantly more than the judicial exception. See MPEP § 2106.05(f). Accordingly, claim 36 fails to recite any additional elements that integrate the abstract idea into a practical application or that amount to significantly more than the abstract idea, and as a result, claim 36 is also patent ineligible under 35 U.S.C. § 101.
Claim 37
In claim 37, the limitation for “perform[ing] error correction on data received over a communication link” is considered a recitation of a mathematical concept (or mental process). Accordingly, claim 37 fails to recite any additional elements (beyond abstract ideas) that integrate the abstract idea into a practical application or that amount to significantly more than the abstract idea, and as a result, claim 37 is also patent ineligible under 35 U.S.C. § 101.
Claim 38
Regarding “Step 1,” independent claim 38 is to a statutory category as claim 38 is directed to a non-transitory computer readable medium, which can be considered e.g. a manufacture.
Accordingly, the analysis proceeds to “Step 2A, Prong One” to determine if the claim recites a judicial exception. In this case, claim 38 recites a mathematical concept and thus recites a judicial exception. In particular the limitation for “a neural network architecture…to receive a binary network input and, in dependence on the binary network input, propagate signals…via a plurality of processing nodes….in accordance with respective binary weights, to form a binary network output,” is considered a recitation of a mathematical concept. The forward-propagation of inputs through a neural network circuit is essentially a mathematical algorithm (see e.g. paragraphs 0006 and 0009). Additionally, the limitation for “training a by implementing an error correcting function to identify a set of binary weights which minimize, for a given binary input to the node, any error between a binary output of the node when formed in accordance with current binary weights of the node and a preferred output from the node, and to update the current binary weights of the node to be the identified set of binary weights,” is also considered a recitation of a mathematical concept. Such training entails performing mathematical calculations and other mathematical concepts (see e.g. paragraphs 0063-0073 of Applicant’s specification as filed).
Because claim 38 recites a judicial exception (i.e. a mathematical concept), the analysis proceeds to “Step2A, Prong Two.” But here the claim does not recite additional elements that integrate the judicial exception into a practical application. In particular, in addition to the above-noted mathematical concepts, claim 38 recites that the neural network architecture is implemented on a system, and that the training is performed via program instructions stored on a non-transitory computer readable medium comprised in the system. However, this represents no more than mere instructions to apply the judicial exception on a computer, and therefore does not integrate the judicial exception into a practical application. See MPEP § 2106.05(f).
Accordingly, as claim 38 does not recite additional elements that integrate the judicial exception into a practical application, the analysis proceeds to “Step 2B” to determine whether the claims recite additional elements that amount to significantly more than the judicial exception. However, in this case, the claim does not. As noted above, in addition to the above-noted mathematical concepts, claim 38 recites that the neural network architecture is implemented on a system and that the training is performed via program instructions stored on a non-transitory computer readable medium. However, as further noted above, this represents no more than mere instructions to apply the judicial exception on a computer. This limitation therefore cannot be considered an additional element that amounts to significantly more than the judicial exception. See MPEP § 2106.05(f).
Consequently, claim 38 recites an abstract idea but does not include additional elements that integrate the abstract idea into a practical application or that amount to significantly more than the abstract idea. As a result, and for the reasons described above, claim 38 would be rejected as being patent ineligible under 35 U.S.C. § 101.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 20, 33, 35, 36 and 38 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Application Publication No. 2020/0005131 to Nakahara et al. (“Nakahara”), and also over the article entitled “Training Feed Forward Nets with Binary Weights via a Modified CHIR Algorithm” by Saad et al. (“Saad”).
Regarding claim 20, Nakahara describes a neural network circuit device for provision in a neural network comprising at least an input layer, one or more hidden layers, and an output layer (see e.g. paragraph 0013). Like claimed, Nakahara particularly describes a circuit implementing a neural network architecture, the circuit being configured to receive a binary network input and, in dependence on the binary network input, propagate signals in the circuit via a plurality of processing nodes of the circuit, in accordance with respective binary weights to form a binary network output (see e.g. paragraphs 0031-0035 and FIG. 1: Nakahara describes a deep neural network comprising an input layer, one or more hidden layers, and an output layer, wherein each layer includes a plurality of nodes; during forward propagation, an input received by each node is weighted and then converted and output to a next layer by using an activation function. Nakahara particularly teaches that each of the nodes of the hidden layers can be implemented via a binarized neural network circuit configured to receive binary inputs, apply binary weights thereto, and produce a binary output – see e.g. paragraphs 0049-0053, 0089-0092 and 0100-0104, and FIGS. 4 and 9. Accordingly, the nodes of the hidden layers together form a circuit implementing a neural network architecture like claimed, wherein the circuit is configured to receive a binary network input and, in dependence on the binary network input, propagate signals in the circuit via a plurality of processing nodes of the circuit, in accordance with respective binary weights to form a binary network output.). The circuit described by Nakahara for implementing the neural network architecture is thus considered a data processing system similar to that of claim 20. Nakahara, however, does not teach that the circuit is further configured to train a first node of the plurality of processing nodes by implementing an error correcting function to identify a set of binary weights which minimize, for a given binary input to the first node, any error between a binary output of the first node when formed in accordance with current binary weights of the first node and a preferred output from the first node, and to update the current binary weights of the first node to be the identified set of binary weights, as is further required by claim 20.
Saad nevertheless describes “learning by choice of internal representations” (CHIR), a learning algorithm that can be applied for training a feedforward neural network with binary weights (see e.g. the Abstract). Particularly, like claimed, Saad teaches that CHIR is employed to train nodes of a neural network by identifying a set of binary weights which minimize, for a given binary input to each node, any error (i.e. an energy function) between a binary output of the node when formed in accordance with current binary weights of the node and a preferred output from the node, and to update the current binary weights of the node to be the identified set of binary weights (see e.g. section 2 “An energy minimization approach for a discrete valued net” on pages 574-577).
It would have been obvious to one of ordinary skill in the art, having the teachings of Nakahara and Saad before the effective filing date of the claimed invention, to configure the circuit described by Nakahara so as to train the nodes of the plurality of processing nodes by implementing an error correcting function (i.e. CHIR) like taught by Saad, which identifies a set of binary weights which minimize, for a given binary input to each node, any error between a binary output of the node when formed in accordance with current binary weights of the node and a preferred output from the node, and to update the current binary weights of the node to be the identified set of binary weights. It would have been advantageous to one of ordinary skill to utilize such a combination because it would enable the neural network to be trained faster, as is suggested by Saad (see e.g. the Abstract). Accordingly, Nakahara and Saad are considered to teach, to one of ordinary skill in the art, a data processing system like that of claim 20.
As per claim 33, it would have been obvious, as is described above, to configure the circuit described by Nakahara so as to train the nodes of the plurality of processing nodes by implementing an error correcting function (i.e. CHIR) like taught by Saad. Saad suggests that the error correcting function entails forming a set of values representing, for each node input, whether a respective node output obtained after the current binary weights have been adapted matches an expected node output (see e.g. section 2 “An energy minimization approach for a discrete valued net” on pages 574-577: Saad teaches that the training is done in iterations, whereby each iteration comprising adjusting weights according to the difference between an expected node output and the actual node outputs; subsequent iterations of the training would thus entail forming a set of values representing whether the respective node output after the current binary weights have been adjusted matches the expected node output.). Accordingly, the above-described combination of Nakahara and Saad is further considered to teach a data processing system like that of claim 33.
Regarding claim 35, Nakahara describes a neural network circuit device for provision in a neural network comprising at least an input layer, one or more hidden layers, and an output layer (see e.g. paragraph 0013). Like claimed, Nakahara particularly describes a circuit implementing a neural network architecture, the circuit being configured to receive a binary network input and, in dependence on the binary network input, propagate signals in the circuit via a plurality of processing nodes of the circuit, in accordance with respective binary weights to form a binary network output (see e.g. paragraphs 0031-0035 and FIG. 1: Nakahara describes a deep neural network comprising an input layer, one or more hidden layers, and an output layer, wherein each layer includes a plurality of nodes; during forward propagation, an input received by each node is weighted and then converted and output to a next layer by using an activation function. Nakahara particularly teaches that each of the nodes of the hidden layers can be implemented via a binarized neural network circuit configured to receive binary inputs, apply binary weights thereto, and produce a binary output – see e.g. paragraphs 0049-0053, 0089-0092 and 0100-0104, and FIGS. 4 and 9. Accordingly, the nodes of the hidden layers together form a circuit implementing a neural network architecture like claimed, wherein the circuit is configured to receive a binary network input and, in dependence on the binary network input, propagate signals in the circuit via a plurality of processing nodes of the circuit, in accordance with respective binary weights to form a binary network output.). The circuit described by Nakahara for implementing the neural network architecture is considered a data processing system similar to that of claim 35. Nakahara teaches that the circuit can be used in edge assembly apparatus that comprises a sensor, i.e. a camera, which is understandably configured to sense data to form the binary network input (see e.g. paragraph 0040). Such an edge assembly apparatus is considered a communication terminal similar to that of claim 35. Nakahara, however, does not teach that the circuit is further configured to train a first node of the plurality of processing nodes by implementing an error correcting function to identify a set of binary weights which minimize, for a given binary input to the first node, any error between a binary output of the first node when formed in accordance with current binary weights of the first node and a preferred output from the first node, and to update the current binary weights of the first node to be the identified set of binary weights, as is further required by claim 35.
Saad nevertheless describes “learning by choice of internal representations” (CHIR), a learning algorithm that can be applied for training a feedforward neural network with binary weights (see e.g. the Abstract). Particularly, like claimed, Saad teaches that CHIR is employed to train nodes of a neural network by identifying a set of binary weights which minimize, for a given binary input to each node, any error (i.e. an energy function) between a binary output of the node when formed in accordance with current binary weights of the node and a preferred output from the node, and to update the current binary weights of the node to be the identified set of binary weights (see e.g. section 2 “An energy minimization approach for a discrete valued net” on pages 574-577).
It would have been obvious to one of ordinary skill in the art, having the teachings of Nakahara and Saad before the effective filing date of the claimed invention, to configure the circuit described by Nakahara so as to train the nodes of the plurality of processing nodes by implementing an error correcting function (i.e. CHIR) like taught by Saad, which identifies a set of binary weights which minimize, for a given binary input to each node, any error between a binary output of the node when formed in accordance with current binary weights of the node and a preferred output from the node, and to update the current binary weights of the node to be the identified set of binary weights. It would have been advantageous to one of ordinary skill to utilize such a combination because it would enable the neural network to be trained faster, as is suggested by Saad (see e.g. the Abstract). Accordingly, Nakahara and Saad are considered to teach, to one of ordinary skill in the art, a communication terminal like that of claim 35.
As per claim 36, Nakahara teaches that the sensor is a camera (see e.g. paragraph 0140). Accordingly, the above-described combination of Nakahara and Saad is further considered to teach a communication terminal like that of claim 36.
Regarding claim 38, Nakahara describes a system having a neural network architecture, the system comprising circuitry to receive a binary network input and, in dependence on the binary network input, propagate signals via a plurality of processing nodes of the circuitry, in accordance with respective binary weights, to form a binary network output (see e.g. paragraphs 0031-0035 and FIG. 1: Nakahara describes a deep neural network comprising an input layer, one or more hidden layers, and an output layer, wherein each layer includes a plurality of nodes; during forward propagation, an input received by each node is weighted and then converted and output to a next layer by using an activation function. Nakahara particularly teaches that the nodes of the hidden layers can each be implemented via a binarized neural network circuit configured to receive binary inputs, apply binary weights thereto, and produce a binary output – see e.g. paragraphs 0049-0053, 0089-0092 and 0100-0104, and FIGS. 4 and 9. Accordingly, Nakahara describes a system having a neural network architecture, the system comprising circuitry, i.e. the hidden layers of the deep neural network, configured to receive a binary network input and, in dependence on the binary network input, propagate signals via a plurality of processing nodes of the circuitry, in accordance with respective binary weights to form a network output.). Nakahara further suggests that a computer program can be implemented in such a system and stored on a non-transitory computer readable medium, whereby when executed by a computer, the computer program causes the computer to train the nodes of the neural network (see e.g. paragraphs 0126-0137). Nakahara, however, does not teach that the program instructions train a node by implementing an error correcting function to identify a set of binary weights which minimize, for a given binary input to the node, any error between a binary output of the node when formed in accordance with current binary weights of the node and a preferred output from the node, and to update the current binary weights of the first node to be the identified set of binary weights, as is further required by claim 38.
Nevertheless, like noted above, Saad describes “learning by choice of internal representations” (CHIR), a learning algorithm that can be applied for training a feedforward neural network with binary weights (see e.g. the Abstract). Particularly, like claimed, Saad teaches that CHIR is employed to train nodes of a neural network by identifying a set of binary weights which minimize, for a given binary input to each node, any error (i.e. an energy function) between a binary output of the node when formed in accordance with current binary weights of the node and a preferred output from the node, and to update the current binary weights of the node to be the identified set of binary weights (see e.g. section 2 “An energy minimization approach for a discrete valued net” on pages 574-577).
It would have been obvious to one of ordinary skill in the art, having the teachings of Nakahara and Saad before the effective filing date of the claimed invention, to configure the program taught by Nakahara so as to train the nodes of the neural network by implementing an error correcting function (i.e. CHIR) like taught by Saad, which identifies a set of binary weights which minimize, for a given binary input to each node, any error between a binary output of the node when formed in accordance with current binary weights of the node and a preferred output from the node, and to update the current binary weights of the node to be the identified set of binary weights. It would have been advantageous to one of ordinary skill to utilize such a combination because it would enable the neural network to be trained faster, as is suggested by Saad (see e.g. the Abstract). Accordingly, Nakahara and Saad are considered to teach, to one of ordinary skill in the art, a computer program like that of claim 38.
Claims 21 and 24-31 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Nakahara and Saad, which is described above, and also over the article entitled “Layered Neural Nets for Pattern Recognition” by Widrow et al. (“Widrow”).
Regarding claim 21, Nakahara and Saad teach a data processing system like that of claim 20, as is described above, and which comprises a circuit implementing a neural network architecture, the circuit being configured to receive a binary network input and, in dependence on the binary network input, propagate signals in the circuit via a plurality of processing nodes of the circuit in accordance with respective binary weights. Like in claim 21, Nakahara further teaches that a first node in the circuit is configured to receive a node input dependent on the binary network input, and form a node output that is binary and functionally equal to a sum in a binary field of (i) a first weight (i.e. a bias) and (ii) the sum in a binary field of products (i.e. each determined by an XNOR gate) of each of bits of a node input and a respective one of a set of second weights (see e.g. paragraphs 0049-0053, 0089-0092, and 0101-0106, and FIGS. 4 and 9). Nakahara and Saad, however, do not teach or suggest that a value indictive of whether all the bits of the node input are non-zero is also included in the sum, as is further required by claim 21.
Widrow nevertheless teaches configuring a neural network node (i.e. a neuron) to receive a plurality of binary node inputs and form a node output that is binary and functionally equal to a sum of (i) a first weight (i.e. a bias), (ii) the products of each of a node input and a respective one of a set of second weights, and (iii) the product of the node inputs together (i.e. the node inputs are multiplied together and added to the sum), inter alia (see e.g. “Linear Separability” and “Nonlinear Separability – Nonlinear Input Functions” on page 1110). The product of the node inputs together is indicative of whether all the bits of the node inputs are non-zero, as the product would be zero if any of the node inputs is zero and non-zero otherwise.
It would have been obvious to one of ordinary skill in the art, having the teachings of Nakahara, Saad and Widrow before the effective filing date of the claimed invention, to modify the data processing system taught by Nakahara and Saad such that the product of the node inputs together, i.e. a value indicative of whether all the bits of the node input are non-zero, is also included in the calculation of the sum, as is taught by Widrow. It would have been advantageous to one of ordinary skill to utilize such a combination because it would enable a single neuron to realize more logic functions, as is taught by Widrow (see e.g. “Linear Separability” and “Nonlinear Separability – Nonlinear Input Functions” on page 1110). Accordingly, Nakahara, Saad and Widrow are considered to teach, to one of ordinary skill in the art, a data processing system like that of claim 21.
As per claim 24, Nakahara suggests that the circuit comprises a plurality of blocks (i.e. a plurality of neural network nodes), wherein each block comprises a first logic circuit (i.e. a plurality of XNOR gates) that receives block inputs dependent on the network input and forms first intermediate block values (i.e. products of node inputs and respective weights) and another logic circuit (e.g. a sum circuit, considered a “third logic circuit” like claimed) that receives as input the first intermediate block values and forms an output from the block (see e.g. paragraphs 0031-0035, 0049-0053, 0089-0092, and 0101-0106, and FIGS. 1, 4 and 9). Nakahara however does not disclose that each block also comprises another logic circuit (i.e. a “second logic circuit”) that that also receives inputs dependent on the network input and forms a second intermediate block value, which is also provided to the third logic circuit to form the output (e.g. the sum), as is required by claim 24. Nevertheless, like described above, it would have been obvious to modify the data processing system taught by Nakahara and Saad such that the product of the node inputs together, i.e. a value indicative of whether all the bits of the node input are non-zero, is also included in the calculation of the sum, as is taught by Widrow, because this would enable a single neuron to realize more logic functions. It thus would have been apparent to include another logic circuit (i.e. a second logic circuit) within each block to multiply the node inputs together and provide the resulting product to be included within the sum, i.e. to include a second logic circuit that receives inputs dependent on the network input and forms a second intermediate block value (i.e. a product of the inputs together), which is also provided to the third logic circuit to form an output from the block. Accordingly, Nakahara, Saad and Widrow are further considered to teach a data processing system like that of claim 24.
As per claim 25, and like noted above, Nakahara teaches that each block (i.e. node) comprises a first logic circuit (i.e. a plurality of XNOR gates) that receives block inputs dependent on the network input and forms first intermediate block values (i.e. products of node inputs and respective weights) (see e.g. paragraphs 0031-0035, 0049-0053, 0089-0092, and 0101-0106, and FIGS. 1, 4 and 9). Nakahara teaches that such a first logic circuit is configured to apply weights as the weights of a binary perceptron (see e.g. paragraphs 0049-0053, 0089-0092, and 0101-0106, and FIGS. 1, 4 and 9). As described above, it would have been obvious to modify the data processing system taught by Nakahara and Saad such that the product of the node inputs together is also included in the calculation of the sum, as is taught by Widrow, whereby it would have been apparent to include another logic circuit (i.e. a second logic circuit) within each block to multiply the node inputs together. As Widrow teaches that a weight can be included in the product of the node inputs together (see e.g. “Nonlinear Separability – Nonlinear Input Functions” on page 1110), it would have further been apparent to configure the second logic circuit so as to include fixed weights (at least during forward propagation) to be included in the product. Accordingly, the above-described combination of Nakahara, Saad and Widrow is further considered to teach a data processing system like that of claim 25.
As per claim 26, it would have been apparent to configure the second logic circuit so as to include fixed weights to be included in the product (i.e. of the node inputs together), as is noted above. As the weights are binary (see e.g. paragraph 0052 of Nakahara), it would have been apparent to apply (at least in memory) weights of ones and zeros. Accordingly, the above-described combination of Nakahara, Saad and Widrow is further considered to teach a data processing system like that of claim 26, in which the weights of each second logic circuit are zero.
As per claim 27, and like noted above, Nakahara suggests that the circuit comprises a plurality of blocks (i.e. a plurality of neural network nodes), wherein each block comprises a first logic circuit (i.e. a plurality of XNOR gates) that receives block inputs dependent on the network input and forms first intermediate block values (i.e. products of node inputs and respective weights) and another logic circuit (e.g. a sum circuit, considered a “third logic circuit” like claimed) that receives as input the first intermediate block values and forms an output from the block (see e.g. paragraphs 0031-0035, 0049-0053, 0089-0092, and 0101-0106, and FIGS. 1, 4 and 9). Nakahara suggests that the third logic circuit (i.e. adder circuit) receives as input a single instance of each corresponding first intermediate block value (see e.g. paragraphs 0049-0053, 0089-0092, and 0101-0106, and FIGS. 4 and 9). As described above, it would have been obvious to modify the data processing system taught by Nakahara and Saad such that the product of the node inputs together is also included in the calculation of the sum, as is taught by Widrow, whereby it would have been apparent to include another logic circuit (i.e. a second logic circuit) within each block to multiply the node inputs together and to provide the resulting product to the third logic circuit. Widrow suggests that multiple instances of the node inputs multiplied together can be included in the sum (see e.g. “Nonlinear Separability – Nonlinear Input Functions” on page 1110). Accordingly, the above-described combination of Nakahara, Saad and Widrow is further considered to teach a data processing system like that of claim 27.
As per claim 28, it would have been obvious, as is described above, to modify the data processing system taught by Nakahara and Saad such that the product of the node inputs together is also included in the calculation of the sum, as is taught by Widrow, whereby it would have been apparent to include another logic circuit (i.e. a second logic circuit) within each block to multiply the node inputs together and to provide the resulting product to the third logic circuit. Widrow suggests that all the inputs included in determining the sum, with the exception of one instance of a product, can comprise fixed weights (see e.g. “Nonlinear Separability – Nonlinear Input Functions” on page 1110). Accordingly, the above-described combination of Nakahara, Saad and Widrow is further considered to teach a data processing system like that of claim 28, in which each third logic circuit is configured to apply fixed weights to all the inputs of the respective third logic circuit with the exception of one instance of the corresponding second logic circuit.
As per claim 29, it would have been obvious, as is described above, to modify the data processing system taught by Nakahara and Saad such that the product of the node inputs together is also included in the calculation of the sum, as is taught by Widrow, whereby it would have been apparent to include another logic circuit (i.e. a second logic circuit) within each block to multiply the node inputs together and to provide the resulting product to the third logic circuit. As the weights are arbitrary it would have been apparent to apply a weight equal to the sum in the binary field of the weights of the corresponding first logic circuit to the instance of the corresponding second intermediate block value. Accordingly, the above-described combination of Nakahara, Saad and Widrow is further considered to teach a data processing system like that of claim 29.
As per claim 30, Nakahara suggests that data processing system can be comprised in at least one of a plurality of sub-systems (e.g. neural network layers) of a system and is configured to provide an output of the at least one of the plurality of sub-systems as an input to at least another of the plurality of sub-systems (e.g. to a following layer) (see e.g. paragraphs 0031-0035, 0049-0053, 0089-0092, and 0101-0106, and FIGS. 1, 4 and 9). Accordingly, the above-described combination of Nakahara, Saad and Widrow is further considered to teach a data processing system like that of claim 30.
As per claim 31, Nakahara suggests that the connections between the plurality of sub-systems are configured (e.g. with weights) in dependence on a desired Boolean function to be implemented (see e.g. paragraphs 0031-0035, 0049-0053, 0089-0092, and 0101-0106, and FIGS. 1, 4 and 9). Accordingly, the above-described combination of Nakahara, Saad and Widrow is further considered to teach a data processing system like that of claim 31.
Claim 22 is rejected under 35 U.S.C. 103 as being unpatentable over the combination of Nakahara, Saad and Widrow, and also over the article entitled, “Neural Networks, Error-Correcting Codes, and Polynomials over the Binary n-Cube” by Bruck et al. (“Bruck”) It is noted that the Bruck reference is cited in Applicant’s IDS filed on September 6, 2022.
Regarding claim 22, Nakahara, Saad and Widrow teach a data processing system like that of claim 21, as is described above, and which comprises a circuit implementing a neural network architecture, the circuit being configured to receive a binary network input and, in dependence on the binary network input, propagate signals in the circuit via a plurality of processing nodes of the circuit in accordance with respective binary weights. Nakahara, Saad and Widrow, however, do not teach that a first node is configured to form a node output that is functionally equal to a coset of a first-order Reed-Muller code, wherein a coset kernel takes as input the bits of the node input, as is required by claim 22.
Bruck nevertheless generally describes an analogy between neural networks and error-correcting codes in which both error-correcting codes and neural networks can be described by polynomials over the n-cube (see e.g. section I “Introduction” on page 976). Bruck particularly describes a coset of a first-order Reed-Muller code (see e.g. the left column of page 980), and thereby demonstrates that such a coset can be functionally equal to a neural network node output.
It would have been obvious to one of ordinary skill in the art, having the teachings of Nakahara, Saad, Widrow and Bruck before the effective filing date of the claimed invention, to modify the data processing system taught by Nakahara, Saad and Widrow such that a first node is configured to form a node output that is functionally equal to a coset of a first-order Reed-Muller code (i.e. wherein a coset kernel takes as input the bits of the node input) like suggested by Bruck. It would have been advantageous to one of ordinary skill to utilize such a combination because it can help solve programming problems for neural networks, as is suggested by Bruck (see section VII “Conclusion”). Accordingly, Nakahara, Saad, Widrow and Bruck are considered to teach, to one of ordinary skill in the art, a data processing system like that of claim 22.
Claim 23 is rejected under 35 U.S.C. 103 as being unpatentable over the combination of Nakahara, Saad and Widrow, and also over the article entitled “All About XOR” by Lewin (“Lewin”).
Regarding claim 23, Nakahara, Saad and Widrow teach a data processing system like that of claim 21, as is described above, and which comprises a circuit implementing a neural network architecture, the circuit being configured to receive a binary network input and, in dependence on the binary network input, propagate signals in the circuit via a plurality of processing nodes of the circuit in accordance with respective binary weights. Nakahara particularly teaches that a first node is implemented as a set of logic circuits configured in a manner so that: (i) each bit of the node input is subject to a multiplication function (i.e. via an XNOR gate) with a respective weight to form a respective first intermediate value; and (ii) a further weight (i.e. a bias) and the first intermediate values are together subject to an addition function to form the node output (see e.g. paragraphs 0049-0053, 0089-0092, and 0101-0106, and FIGS. 1, 4 and 9). Nakahara suggests that the multiplication function can alternatively be performed in part using an AND gate (see e.g. paragraph 0096). Like described above (see the rejection for claim 21), it would have been obvious to modify the data processing system taught by Nakahara and Saad such that the product of the node inputs together is also included in the calculation of the sum, as is taught by Widrow, It thus would have been apparent to configure the set of logic circuits of the first node so that, in addition, all bits of the node input are together subject to a logical multiplication function to form a second intermedia value, which is also included in the sum to form the node output. Nakahara, Saad and Widrow are thus considered to teach a data processing system similar to that of claim 23, but do not explicitly disclose that the addition function is performed with a logical XOR function to form the node output, as is required by claim 23.
Lewin nevertheless generally teaches using a logical XOR function to add a plurality of values (see e.g. “Inside you ALU”) on page 18).
It would have been obvious to one of ordinary skill in the art, having the teachings of Nakahara, Saad, Widrow and Lewin before the effective filing date of the claimed invention, to modify the data processing system taught by Nakahara, Saad and Widrow such that a logical XOR function like taught by Lewin is used to add the further weight, the first intermediate value and the second intermediate value. It would have been advantageous to one of ordinary skill to utilize such an XOR function because it is commonly used in addition operations (e.g. in processors), as is suggested by Lewin (see e.g. “Inside you ALU” on page 18). Accordingly, Nakahara, Saad, Widrow and Lewin are considered to teach, to one of ordinary skill in the art, a data processing system like that of claim 23.
Claims 32 and 34 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Nakahara and Saad, which is described above, and also over the article entitled, “Neural Networks, Error-Correcting Codes, and Polynomials over the Binary n-Cube” by Bruck et al. (“Bruck”). As noted above, the Bruck reference is cited in Applicant’s IDS filed on September 6, 2022.
Regarding claim 32, Nakahara and Saad teach a data processing system like that of claim 20, as is described above, and which comprises a circuit implementing a neural network architecture, the circuit being configured to receive a binary network input and, in dependence on the binary network input, propagate signals in the circuit via a plurality of processing nodes of the circuit in accordance with respective binary weights. Nakahara and Saad, however, do not teach or suggest that the circuit is configured to adapt the current binary weights for a first node in dependence on a set of expected node outputs expected for a set of node inputs by the steps of: (i) forming a set of values representing, for each node input, whether an expected node output is zero, one, or indifferent; (ii) identifying an ith row of a Hadamard matrix that best matches the set of values; and (iii) adapting the current binary weights for the first node in dependence on the identification, as is required by claim 32.
Like noted above, Bruck generally describes an analogy between neural networks and error-correcting codes in which both error-correcting codes and neural networks can be described by polynomials over the n-cube (see e.g. section I “Introduction” on page 976). Bruck further suggests that the current binary weights (i.e. polynomial coefficients) for a first node can be adapted in dependence on a set of expected node outputs expected for a set of node inputs by the steps of: (i) forming a set of values representing, for each node input, whether an expected node output is zero, one, or indifferent; (ii) identifying an ith row of a Hadamard matrix that best matches the set of values; and (iii) adapting the current binary weights for the first node in dependence on the identification (see e.g. Appendix ii “Generalization to Nonlinear Codes” on pages 985-986).
It would have been obvious to one of ordinary skill in the art, having the teachings of Nakahara, Saad and Bruck before the effective filing date of the claimed invention, to modify the data processing system taught by Nakahara and Saad such that the circuit is configured to adapt the current binary weights for a first node in dependence on a set of expected node outputs expected for a set of node inputs by the steps of: (i) forming a set of values representing, for each node input, whether an expected node output is zero, one, or indifferent; (ii) identifying an ith row of a Hadamard matrix that best matches the set of values; and (iii) adapting the current binary weights for the first node in dependence on the identification, as is taught by Bruck. It would have been advantageous to one of ordinary skill to utilize such a combination because it would enable such weights to be efficiently identified, as is suggested by Bruck (see e.g. Appendix ii “Generalization to Nonlinear Codes” on pages 985-986). Accordingly, Nakahara, Saad and Bruck are considered to teach, to one of ordinary skill in the art, a data processing system like that of claim 32.
Regarding claim 34, Nakahara and Saad teach a data processing system like that of claim 20, as is described above, and which comprises a circuit implementing a neural network architecture, the circuit being configured to receive a binary network input and, in dependence on the binary network input, propagate signals in the circuit via a plurality of processing nodes of the circuit in accordance with respective binary weights. Nakahara and Saad, however, do not teach or suggest that the data processing system is configured to adapt weights for a node in dependence on a set of expected node outputs for a set of node inputs by operating a fast Walsh-Hadamard function taking as input the set of expected node outputs, as is required by claim 34.
Like noted above, Bruck generally describes an analogy between neural networks and error-correcting codes in which both error-correcting codes and neural networks can be described by polynomials over the n-cube (see e.g. section I “Introduction” on page 976). Bruck further suggests that the binary weights (i.e. polynomial coefficients) for a first node can be adapted in dependence on a set of expected node outputs expected for a set of node inputs by operating a fast Walsh-Hadamard function taking as input the set of expected node outputs (see e.g. Appendix ii “Generalization to Nonlinear Codes” on pages 985-986).
It would have been obvious to one of ordinary skill in the art, having the teachings of Nakahara, Saad and Bruck before the effective filing date of the claimed invention, to modify the data processing system taught by Nakahara and Saad such that the circuit is configured to adapt binary weights for a node in dependence on a set of expected node outputs expected for a set of node inputs by operating a fast Walsh-Hadamard function taking as input the set of expected node outputs, as is taught by Bruck. It would have been advantageous to one of ordinary skill to utilize such a combination because it would enable such weights to be efficiently identified, as is suggested by Bruck (see e.g. Appendix ii “Generalization to Nonlinear Codes” on pages 985-986). Accordingly, Nakahara, Saad and Bruck are considered to teach, to one of ordinary skill in the art, a data processing system like that of claim 34.
Claim 37 is rejected under 35 U.S.C. 103 as being unpatentable over the combination of Nakahara and Saad, which is described above, and also over U.S. Patent Application Publication No. 2011/0176638 to Davey et al. (“Davey”).
Regarding claim 37, Nakahara and Saad teach a communication terminal like that of claim 35, as is described above, and which comprises a circuit implementing a neural network architecture, the circuit being configured to receive a binary network input and, in dependence on the binary network input, propagate signals in the circuit via a plurality of processing nodes of the circuit in accordance with respective binary weights. Nakahara and Saad, however, do not disclose that the terminal is configured to perform error correction on data received over a communication link using the data processing system, as is required by claim 37.
Performing error correction on received data is nevertheless well-known in the art. Davey for example teaches performing error correction on data received over a communication link using a data processing system (see e.g. paragraphs 0005-0007).
It would have been obvious to one of ordinary skill in the art, having the teachings of Nakahara, Saad and Davey before the effective filing date of the claimed invention, to configure the communication terminal taught by Nakahara and Saad so as to perform error correction on data received over a communication link using the data processing system like taught by Davey. It would have been advantageous to one of ordinary skill to utilize such a combination because would improve the reliability of the data, as is evident from Davey (see e.g. paragraphs 0005-0007). Accordingly, Nakahara, Saad and Davey are considered to teach, to one of ordinary skill in the art, a data processing system like that of claim 37.
Response to Arguments
The Examiner acknowledges the Applicant’s amendments to claims 20, 24, 28, 30-35 and 38. In response to these amendments, the objection presented in the previous Office to claim 30 is respectfully withdrawn. Also withdrawn in response to these amendments is the 35 U.S.C. § 101 rejection presented in the previous Office Action to claim 38 for being directed to non-statutory subject matter. The Examiner respectfully notes, however, that claim 38 remains rejected under 35 U.S.C. § 101 (along with claims 20-37) for being directed to an abstract idea (e.g. a mental process and/or mathematical concept) without significantly more, as is detailed above.
Regarding the 35 U.S.C. § 101 rejections of claims 20-38, the Applicant argues that independent claims 20, 35 and 38 do not recite a mathematical concept under prong 1 of Step 2A of the subject matter eligibility test. The Applicant argues that claim 20 for example recites various elements (e.g. “a circuit implementing a neural network architecture”) that describe a specific hardware configuration and signal propagation mechanism, not an abstract idea. The Applicant further argues that the claims recite various operations (e.g. weight updating) that are performed by a machine and are therefore not a mental process or mathematical concept that can be performed abstractly.
In response, the Examiner submits that even though claims 20, 35 and 38 describe hardware configurations that perform various operations, these claims also recite mental processes and/or mathematical concepts. That is, various operations recited as being performed by the hardware configurations can be considered a mental process and/or mathematical concept. Applying a binary network input to a neural network architecture and forming a binary network output in accordance with respective binary weights is a mathematical concept: the forward-propagation of inputs through a neural network is essentially a mathematical algorithm (see e.g. paragraphs 0006 and 0009 of the instant application). Similarly, training a node of such a neural network by implementing an error correcting function can also be considered a mathematical concept. Training according to the error correcting function entails performing mathematical calculations and other mathematical concepts (see e.g. paragraphs 0063-0073 of the instant application). 1
As noted above, the Applicant argues that the claims also recite a specific hardware configuration and signal propagation mechanism. However, the only hardware actually recited in independent claim 20, for example, is a “circuit” that implements the neural architecture, propagates signals through nodes thereof to form the binary network output, and trains a first node by implementing the error correcting function. The claims describe the circuit at a high level of generality, reciting only what it does, and not any actual components of the circuit. But, like noted in the previous paragraph, most of what the circuit does can be considered a mental process and/or mathematical concept. Accordingly, the inclusion of the circuit in the claims is tantamount to mere instructions to apply the judicial exceptions on a computer, and therefore fails to integrate the judicial exception into a practical application or amount to significantly more than the judicial exception. See MPEP § 2106.05(f). The Examiner thus respectfully maintains that the claims recite mathematical concepts implemented on generic computer hardware (e.g. a “circuit”).
Further regarding the 35 U.S.C. § 101 rejections, the Applicant argues that, even if reciting a mathematic concept, claim 20 integrates the judicial exception into a practical application because the claim is directed to an improvement in the functioning of a computer or other technology. The Applicant argues that the claimed binary neural network training, for example, minimizes power consumption and latency, improves circuit efficiency, and reduces memory access overhead.
However, in response, the Examiner respectfully submits that these improvements are not reflected in the claims. See e.g. Intellectual Ventures I LLC v. Symantec Corp., 838 F.3d 1307, 1316, 120 USPQ2d 1353, 1359 (Fed. Cir. 2016) (patent owner argued that the claimed email filtering system improved technology by shrinking the protection gap and mooting the volume problem, but the court disagreed because the claims themselves did not have any limitations that addressed these issues). The resource-saving benefits of binary neural networks is acknowledged in Applicant’s background prior art (see e.g. paragraphs 0008-0009 and 0022-0023), and so implementing a binary neural network on a circuit like in the first clause of claim 20 does not necessarily reflect any improvement per se. The claimed training (e.g. in the second clause of claim 20) is recited in such generality so as to encompass almost any training of a binary neural network node. As such, it also doesn’t necessarily follow that the training as recited in the independent claims results in any of the purported improvements. Consequently, the Examiner respectfully maintains that the claims do not integrate the judicial exception into a practical application.
Further regarding the 35 U.S.C. § 101 rejections, the Applicant argues that, even if directed to a judicial exception, claim 20 includes additional elements that amount to significantly more than the judicial exception. In particular, the Applicant notes that claim 20 recites elements that operate in an unconventional manner, e.g. the recitation of “wherein the circuit is further configured to train a first node of the plurality of processing nodes by implementing an error correcting function to identify a set of binary weights which minimize, for a given binary input to the first node, any error between a binary output of the first node when formed in accordance with current binary weights of the first node and a preferred output from the first node.”
However, for the reasons previously noted, the Examiner respectfully submits that “train[ing] a first node of the plurality of processing nodes by implementing an error correcting function to identify a set of binary weights which minimize, for the given binary input to the first node, any error between a binary output of the first node when formed in accordance with current binary weights of the first node and a preferred output from the first node” is a mathematical concept. The only additional element in claim 20 (i.e. additional to the judicial exception/mathematical concept) is the recitation that a “circuit” is configured to perform such training. Like noted above, the inclusion of the circuit in the claims is tantamount to mere instructions to apply the judicial exception on a computer, and therefore fails amount to significantly more than the judicial exception. See MPEP § 2106.05(f). Accordingly, the Examiner respectfully maintains the 35 U.S.C. § 101 rejections of claims 20-38.
Regarding the prior art rejections, the Applicant submits that Saad’s neural network has a continuous output. The Applicant argues that Saad consequently does not teach or suggest a circuit that is “further configured to train a first node of the plurality of processing nodes by implementing an error correcting function to identify a set of binary weights which minimize, for a given binary input to the first node, any error between a binary output of the first node when formed in accordance with current binary weights of the first node and a preferred output from the first node, and to update the current binary weights of the first node to be the identified set of binary weights,” as is recited in independent claim 20 and similarly expressed in independent claims 35 and 38.
The Examiner however respectfully disagrees. The broadest reasonable interpretation of the claims does not require actually producing a binary output during the training. Instead, the claims recite an intended result of the training, i.e. “to identify a set of binary weights which minimize, for a given binary input to the first node, any error between a binary output of the first node when formed in accordance with current binary weights of the first node and a preferred output from the first node.”
The training described by Saad identifies such binary weights. The training is intended to minimize, for given binary inputs to the nodes, any error (i.e. “energy”) between a binary (“discrete”) output of the nodes when formed in accordance with current binary weights of the nodes and a preferred output from the nodes:
PNG
media_image1.png
465
490
media_image1.png
Greyscale
(Page 574; emphasis added).
Instead of a binary function, the training described by Saad employs a continuous function
f
to represent the node output (i.e. “neural response”); the continuous function
f
approximates the sign function (i.e. a binary function) and is used instead of the sign function, because the sign function does not have a non-zero derivative like needed to identify the amount to change the weights or internal representations during training:
PNG
media_image2.png
388
516
media_image2.png
Greyscale
(Page 575; emphasis added).
PNG
media_image3.png
372
365
media_image3.png
Greyscale
It is apparent that this approximation,
f
, is used only during training, or before to derive the appropriate rules to change the weights and internal representations. One of ordinary skill would understand that once training is complete, the approximation
f
for the sign function is no longer necessary (and in fact would be non-preferred since binary values output by the nodes would require less storage than continuous values). Because the continuous function
f
is intended to approximate the sign function, the binary weights produced by the training are applicable to a binary neural network that generates binary outputs (i.e. utilizing a sign function) like taught by Saad and Nakahara, and would understandably minimize any error between a binary output of the nodes and a preferred output from the nodes. Accordingly, the Examiner respectfully maintains that Saad teaches a circuit that is “further configured to train a first node of the plurality of processing nodes by implementing an error correcting function to identify a set of binary weights which minimize, for a given binary input to the first node, any error between a binary output of the first node when formed in accordance with current binary weights of the first node and a preferred output from the first node, and to update the current binary weights of the first node to be the identified set of binary weights,” as is recited in independent claim 20.
Moreover, the Examiner respectfully notes that Saad also suggests that the continuous function
f
is not even required during the training, and that instead binary (“discrete”) values can be utilized to represent the node outputs (e.g. internal representations, IR) during training:
PNG
media_image4.png
174
484
media_image4.png
Greyscale
PNG
media_image5.png
409
493
media_image5.png
Greyscale
(Pages 575-576; emphasis added).
Accordingly, the Examiner again respectfully maintains that Saad teaches “an error correcting function to identify a set of binary weights which minimize, for a given binary input to the first node, any error between a binary output of the first node when formed in accordance with current binary weights of the first node and a preferred output from the first node, and to update the current binary weights of the first node to be the identified set of binary weights,” as is recited in independent claim 20.
The Applicants arguments filed on October 20, 2025 have thus been fully considered, but are not persuasive.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BLAINE T BASOM whose telephone number is (571)272-4044. The examiner can normally be reached Monday-Friday, 9:00 am - 5:30 pm, EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matt Ell can be reached at (571)270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BTB/
2/5/2026
/MATTHEW ELL/Supervisory Patent Examiner, Art Unit 2141
1 See also, e.g., claim 2 of example 47 of the Subject Matter Eligibility Examples, in which the following training limitation is considered a recitation of a mathematical concept: “training, by the computer, the ANN based on the input data and a selected training algorithm to generate a trained ANN, wherein the selected training algorithm includes a backpropagation algorithm and a gradient descent algorithm.”