DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Applicant claims the benefit of prior-filed U.S. Provisional Patent Application No. 63/427,653 filed November 23, 2022, which is acknowledged.
Drawings
The drawings were received on 11/21/2023. These drawings are acceptable.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on the following date(s): 02/02/2024 has been considered by the examiner.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are noted below with the generic place holder in bold and functional language in italicized font:
Claim 1
a plurality of inputs; a plurality of outputs; a learning network between the plurality of inputs and the plurality of outputs, the learning network including a plurality of layers, each of the plurality of layers including a weight layer including a plurality of weights coupled with an activation layer including a plurality of neurons configured to apply at least one activation function, a plurality of connections coupling the plurality of layers; and a negative feedback network selectively couplable between the plurality of outputs and the plurality of inputs; wherein the plurality of weights are configured to be trained by providing input signals corresponding to a target output to the plurality of inputs, …
Claim 9:
comprising: a plurality of inputs; a plurality of outputs; a plurality of weight layers; a plurality of activation layers interleaved with the plurality of weight layers, the plurality of weight layers and the plurality of activation layers being between the plurality of inputs and the plurality of outputs, each of the plurality of weights layer including a plurality of weights, each of the plurality of activation layers an activation layer including a plurality of neurons configured to apply at least one activation function; and a negative electrical feedback network selectively couplable between the plurality of outputs and the plurality of inputs; wherein the plurality of weights are configured to be trained by providing electrical input signals corresponding to a target output to the plurality of inputs, ...
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-14, 17-18 and 20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim limitations noted above, in the claim interpretation section, of claims 1 and 9 have been evaluated under the three-prong test set forth in MPEP § 2181, subsection I, but the result is inconclusive. Thus, it is unclear whether this limitation should be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because neurons and weights are part of a neural network model that are not typically recited as generic place holders as claimed. Furthermore, the specification highlights that the claimed components can be implemented as hardware components or computer functions per noted sections:
[0024] In some embodiments, each layer 101 includes one or more weight layers and one or more activation layers. The weights in the weight layers may take the form of an impedance or other analogous electrical property... As such, the weights (e.g. impedances at the crossings of the crossbar) in a weight layer can multiply the signal through the weight layer by a factor (i.e. weight the signal). In some embodiments, a weight may take the form of a programmable resistance, a programmable capacitance, or data stored in a memory cell that is converted into an impedance or admittance and applied to an input signal. Activation layers apply a function to the input signals. Thus, the activation layers may be viewed as including neurons that receive an electrical signal and apply a particular function (the activation function) based on the status of the neuron. For example, the neurons may be hardware neurons formed from a collection of electrical components that provide particular electrical signal(s) out based on the electrical signal(s) received. Alternatively, the neurons may be configured in another manner. For example, the function may be provided digitally using a processor or other technique. In some embodiments, all of the neurons in a layer 101 apply the same activation function. In other embodiments, different neurons in a particular layer 101 apply different activation functions…
The boundaries of this claim limitation are ambiguous; therefore, the claim is indefinite and is rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph.
In response to this rejection, applicant must clarify whether this limitation should be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. Mere assertion regarding applicant’s intent to invoke or not invoke 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph is insufficient. Applicant may:
(a) Amend the claim to clearly invoke 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, by reciting “means” or a generic placeholder for means, or by reciting “step.” The “means,” generic placeholder, or “step” must be modified by functional language, and must not be modified by sufficient structure, material, or acts for performing the claimed function;
(b) Present a sufficient showing that 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, should apply because the claim limitation recites a function to be performed and does not recite sufficient structure, material, or acts to perform that function;
(c) Amend the claim to clearly avoid invoking 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, by deleting the function or by reciting sufficient structure, material or acts to perform the recited function; or
(d) Present a sufficient showing that 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, does not apply because the limitation does not recite a function or does recite a function along with sufficient structure, material or acts to perform that function.
Regarding the claims that depend on claims 1 and 9, the claims fail to resolve the noted deficiency and thus the claims are appropriately rejected.
Regarding claim 3, recites the limitation “wherein the electrical input signals and the electrical output signals have a dual relationship; and wherein, for being connected with the plurality of inputs and the plurality of outputs, the negative feedback network is configured to provide zero output signals for the outputs for duals of the electrical output signals and input electrical perturbations having the dual relationship with the electrical output signals” is unclear as the claim appears to require the signals to be zero and it is unclear how the feedback is determined when “input electrical perturbations having the dual relationship with the electrical output signals” is required by the limitation “wherein the electrical input signals and the electrical output signals have a dual relationship”. If there is always an output signal of zero for “duals of the electrical output signals and input electrical perturbations having the dual relationship with the electrical output signals” as claimed. Is the feedback network always setting all signals to zero or some combination of signals? If the former is the intended interpretation than the claims appear to require dual relationship in the plurality of inputs and outputs that are always set to zero for having a dual relationship, how/when is the feedback computed?
The examiner interprets any unperturbed signal as within the scope of the claim limitations.
Regarding claims 10 and 17, the claims recite similar limitations to those in claim 3 and thus rejected under the same rationale.
Regarding the claims that depend on claims 3, 10 and 17, the claims fail to resolve the noted deficiency and thus the claims are appropriately rejected.
Regarding claim 20, the limitation “further comprising: repeating the providing, measuring, perturbing, measuring, and updating” renders the claim indefinite because the antecedent basis of the claimed operation is unclear and the order/conditions for repeating claimed operations are not clear. Is the repeat task performed on a new set of operations and what measuring operations are repeated as to measuring operations are claimed in claim 16.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e. an abstract idea) without significantly more.
Claim 1: Dose claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
outputs and the plurality of inputs, perturbing the output signals by a plurality of perturbations with the negative feedback network coupled between the plurality of inputs and the plurality of outputs, measuring corresponding perturbations for the plurality of connections, and updating the weights based on the corresponding perturbations, the plurality of perturbations being based on a difference between the plurality of output signals and the target output. (Considered directed to a Mental Process: Making evaluations and judgements of observations for formulating observations, evaluations and judgements as claimed; see MPEP § 2106.04(a)(2), subsection III; And Mathematical concepts – mathematical relationships as claimed; Per MPEP a mathematical relationship is a relationship between variables or numbers. A mathematical relationship may be expressed in words or using mathematical symbol (see MPEP § 2106.04(a)(2), subsection I);
The broadest reasonable interpretation (BRI) includes a mathematical relationship, per specification paragraph 0025: … The perturbations around the operating point of each layer 101 may be linearly approximated (via the Jacobian matrix of each layer 101) using the chain matrix (transmission) parameterization… And the broadest reasonable interpretation includes processes for making determinations, per paragraph [0039] The input perturbations from negative feedback networks 330 propagate forward through learning network 300. These perturbations can be measured within learning network 300, for example at the connections between layers 310 and 320. This provides a measure of the local target, or how the weights in the nearest weight layer 310 preceding the measurement position should be adjusted… The weights in each weight layer 310 may then be adjusted in accordance with the measured perturbation. This process may be iteratively performed to optimize the weights of layers 310. Learning network 300 may then be used for the desired application...)
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
a plurality of inputs; a plurality of outputs; a learning network between the plurality of inputs and the plurality of outputs, the learning network including a plurality of layers, each of the plurality of layers including a weight layer including a plurality of weights coupled with an activation layer including a plurality of neurons configured to apply at least one activation function, a plurality of connections coupling the plurality of layers; and a negative feedback network selectively couplable between the plurality of outputs and the plurality of inputs; (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation simply link the judicial exception to a field of use and/or technology environment, see MPEP 2106.05(h).)
wherein the plurality of weights are configured to be trained by providing input signals corresponding to a target output to the plurality of inputs, (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation merely include instructions to implement an abstract idea on a computer, or merely use a computer as a tool to perform an abstract idea; Thus claim limitations amount to mere instructions to apply the judicial exception using a computer/computing environment as a tool, as discussed in MPEP § 2106.05(f).)
Alternatively, wherein the plurality of weights are configured to be trained by providing input signals corresponding to a target output to the plurality of inputs, … (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to insignificant solution activity, e.g. Receiving or transmitting data over a network)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application.
First, the additional limitations directed to elements that generally link the use of a judicial exception to a particular technological environment or field of use and merely invoke the use of computer technology as a tool for applying the judicial exception.
Secondly, the noted additional limitation elements directed to insignificant solution activity, as noted above, the courts have deemed these types of activity as well-known routine and convectional, see evidences noted below (See MPEP 2106.05(d)(II)):
Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network); but see DDR Holdings, LLC v. Hotels.com, L.P., 773 F.3d 1245, 1258, 113 USPQ2d 1097, 1106 (Fed. Cir. 2014) ("Unlike the claims in Ultramercial, the claims at issue here specify how interactions with the Internet are manipulated to yield a desired result‐‐a result that overrides the routine and conventional sequence of events ordinarily triggered by the click of a hyperlink." (emphasis added));
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 2: Dose claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
the plurality of perturbations is a plurality of electrical perturbations, the corresponding perturbations are corresponding electrical perturbations. (Mathematical concepts – mathematical relationships as claimed; Per MPEP a mathematical relationship is a relationship between variables or numbers. A mathematical relationship may be expressed in words or using mathematical symbol (see MPEP § 2106.04(a)(2), subsection I);)
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
wherein the negative feedback network is an electrical negative feedback network, the input signals are electrical input signals, the output signals are electrical output signals (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation simply link the judicial exception to a field of use and/or technology environment, see MPEP 2106.05(h).)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application. Specifically, first, the additional limitations directed to elements that generally link the use of a judicial exception to a particular technological environment.
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 3: Dose claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
wherein the electrical input signals and the electrical output signals have a dual relationship; and wherein, for being connected with the plurality of inputs and the plurality of outputs, the negative feedback network is configured to provide zero output signals for the outputs for duals of the electrical output signals and input electrical perturbations having the dual relationship with the electrical output signals. (Considered directed to a Mathematical concepts – mathematical relationships as claimed; See MPEP § 2106.04(a)(2), subsection I)
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application. Specifically, the additional limitations are directed to elements that generally link the use of a judicial exception to a particular technological environment or field of use.
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 4: Dose claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
Abstract idea recited in claim 3 as analyzed above.
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
wherein the electrical input signals are selected from voltage input signals and current input signals and wherein the electrical output signals are current output signals for the voltage input signals and voltage output signals for the current input signals.. (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation simply link the judicial exception to a field of use and/or technology environment, see MPEP 2106.05(h).)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application. Specifically, the additional limitations are directed to elements that generally link the use of a judicial exception to a particular technological environment or field of use.
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 5: Dose claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
Abstract idea as analyzed in claim 2.
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
wherein the negative feedback network includes a plurality of operational amplifiers having op-amp inputs configured to be selectively connected with the plurality of outputs and op-amp outputs configured to be selectively coupled with the plurality of inputs. (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation simply link the judicial exception to a field of use and/or technology environment, see MPEP 2106.05(h).)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application. Specifically, the additional limitations are directed to elements that generally link the use of a judicial exception to a particular technological environment or field of use.
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 6: Dose claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
Abstract idea as analyzed in claim 2.
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
wherein each of the plurality of layers has a first width. (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation simply link the judicial exception to a field of use and/or technology environment, see MPEP 2106.05(h).)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application. Specifically, the additional limitations are directed to elements that generally link the use of a judicial exception to a particular technological environment or field of use.
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 7: Dose claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
Abstract idea recited in claims 6 as analyzed above.
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
wherein the system includes: at least one additional layer having a second width different from the first width, the at least one additional layer including at least one of an additional weight layer or an additional activation layer. (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation simply link the judicial exception to a field of use and/or technology environment, see MPEP 2106.05(h).)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application. Specifically, the additional limitations are directed to elements that generally link the use of a judicial exception to a particular technological environment or field of use.
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 8: Dose claim fall within a statutory category? No, but can be amended to fall within a statutory category.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
Abstract idea as analyzed in claim 1.
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
wherein the at least one activation function is at least one invertible activation function. (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation simply link the judicial exception to a field of use and/or technology environment, see MPEP 2106.05(h).)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application. Specifically, first, the additional limitations directed to elements that generally link the use of a judicial exception to a particular technological environment or field of use.
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 9: Dose claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
. (Considered directed to a Mental Process: Making evaluations and judgements of observations for formulating observations, evaluations and judgements as claimed; see MPEP § 2106.04(a)(2), subsection III; And Mathematical concepts – mathematical relationships as claimed; See notes in claim 1 regarding BRI)
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
a plurality of inputs; a plurality of outputs; a plurality of weight layers; a plurality of activation layers interleaved with the plurality of weight layers, the plurality of weight layers and the plurality of activation layers being between the plurality of inputs and the plurality of outputs, each of the plurality of weights layer including a plurality of weights, each of the plurality of activation layers an activation layer including a plurality of neurons configured to apply at least one activation function; and a negative electrical feedback network selectively couplable between the plurality of outputs and the plurality of inputs; (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation simply link the judicial exception to a field of use and/or technology environment, see MPEP 2106.05(h).)
wherein the plurality of weights are configured to be trained by providing electrical input signals corresponding to a target output to the plurality of inputs; (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation merely include instructions to implement an abstract idea on a computer, or merely use a computer as a tool to perform an abstract idea; Thus claim limitations amount to mere instructions to apply the judicial exception using a computer/computing environment as a tool, as discussed in MPEP § 2106.05(f).)
Alternatively, wherein the plurality of weights are configured to be trained by providing electrical input signals corresponding to a target output to the plurality of inputs; … (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to insignificant solution activity, e.g. Receiving or transmitting data over a network)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application.
First, the additional limitations directed to elements that generally link the use of a judicial exception to a particular technological environment or field of use and merely invoke the use of computer technology as a tool for applying the judicial exception.
Secondly, the noted additional limitation elements directed to insignificant solution activity, as noted above, the courts have deemed these types of activity as well-known routine and convectional, see evidences noted below (See MPEP 2106.05(d)(II)):
Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network); but see DDR Holdings, LLC v. Hotels.com, L.P., 773 F.3d 1245, 1258, 113 USPQ2d 1097, 1106 (Fed. Cir. 2014) ("Unlike the claims in Ultramercial, the claims at issue here specify how interactions with the Internet are manipulated to yield a desired result‐‐a result that overrides the routine and conventional sequence of events ordinarily triggered by the click of a hyperlink." (emphasis added));
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Regarding claim 10, the limitations are similar to those in claim 3, and are thus rejected under the same rationale.
Regarding claim 11, the limitations are similar to those in claim 4, and are thus rejected under the same rationale.
Regarding claim 12, the limitations are similar to those in claim 5, and are thus rejected under the same rationale.
Regarding claim 13, the limitations are similar to those in claim 7, and are thus rejected under the same rationale.
Regarding claim 14, the limitations are similar to those in claim 8, and are thus rejected under the same rationale.
Claim 15: Dose claim fall within a statutory category? Yes
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
(Considered directed to a Mental Process: Making evaluations and judgements of observations for formulating observations, evaluations and judgements as claimed; see MPEP § 2106.04(a)(2), subsection III; And Mathematical concepts – mathematical relationships as claimed; Per MPEP a mathematical relationship is a relationship between variables or numbers. A mathematical relationship may be expressed in words or using mathematical symbol (see MPEP § 2106.04(a)(2), subsection I);
The broadest reasonable interpretation (BRI) includes a mathematical relationship, per specification paragraph 0025: … The perturbations around the operating point of each layer 101 may be linearly approximated (via the Jacobian matrix of each layer 101) using the chain matrix (transmission) parameterization… And the broadest reasonable interpretation includes processes for making determinations, per paragraph [0039] The input perturbations from negative feedback networks 330 propagate forward through learning network 300. These perturbations can be measured within learning network 300, for example at the connections between layers 310 and 320. This provides a measure of the local target, or how the weights in the nearest weight layer 310 preceding the measurement position should be adjusted… The weights in each weight layer 310 may then be adjusted in accordance with the measured perturbation. This process may be iteratively performed to optimize the weights of layers 310. Learning network 300 may then be used for the desired application...)
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
providing, to a plurality of inputs of a network, input signals corresponding to a target output, the network… (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to insignificant solution activity, e.g. Receiving or transmitting data over a network)
the network including the plurality of inputs, a plurality of outputs, and a plurality of layers between the plurality of inputs and the plurality of outputs, each of the plurality of layers including a weight layer including a plurality of weights coupled with an activation layer including a plurality of neurons configured to apply at least one activation function, a plurality of connections between the plurality of layers, (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation simply link the judicial exception to a field of use and/or technology environment, see MPEP 2106.05(h).)
the network settling at an operating point based on the input signals, (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation generally recites an effect of the judicial exception or claims every mode of accomplishing that effect. Thus, claim limitations amounts to a claim that is merely adding the words "apply it" to the judicial exception, as discussed in MPEP § 2106.05(f).)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application.
First, the additional limitations directed to elements that generally link the use of a judicial exception to a particular technological environment or field of use and merely invoke the use of computer technology as a tool for applying the judicial exception.
Secondly, the noted additional limitation elements directed to insignificant solution activity, as noted above, the courts have deemed these types of activity as well-known routine and convectional, see evidences noted below (See MPEP 2106.05(d)(II)):
Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network); but see DDR Holdings, LLC v. Hotels.com, L.P., 773 F.3d 1245, 1258, 113 USPQ2d 1097, 1106 (Fed. Cir. 2014) ("Unlike the claims in Ultramercial, the claims at issue here specify how interactions with the Internet are manipulated to yield a desired result‐‐a result that overrides the routine and conventional sequence of events ordinarily triggered by the click of a hyperlink." (emphasis added));
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Regarding claim 16, the limitations are similar to those in claim 2, and are thus rejected under the same rationale.
Regarding claim 17, the limitations are similar to those in claim 3, and are thus rejected under the same rationale.
Regarding claim 18, the limitations are similar to those in claim 5, and are thus rejected under the same rationale.
Regarding claim 19, the limitations are similar to those in claim 8, and are thus rejected under the same rationale.
Claim 20: Dose claim fall within a statutory category? Yes
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
Abstract idea noted in claim 16.
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
further comprising: repeating the providing, measuring, perturbing, measuring, and updating. (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to insignificant solution activity, e.g. Performing repetitive calculations)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application.
First, the additional limitations directed to elements that generally link the use of a judicial exception to a particular technological environment or field of use.
Secondly, the noted additional limitation elements directed to insignificant solution activity, as noted above, the courts have deemed these types of activity as well-known routine and convectional, see evidences noted below (See MPEP 2106.05(d)(II)):
Performing repetitive calculations, Flook, 437 U.S. at 594, 198 USPQ2d at 199 (recomputing or readjusting alarm limit values); Bancorp Services v. Sun Life, 687 F.3d 1266, 1278, 103 USPQ2d 1425, 1433 (Fed. Cir. 2012) ("The computer required by some of Bancorp’s claims is employed only for its most basic function, the performance of repetitive calculations, and as such does not impose meaningful limits on the scope of those claims.");
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
As shown above, claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed a judicial exception and does not recite, when claim elements are examined individually and as a whole, elements that the courts have identified as "significantly more” than the recited judicial exception. The claims are therefore directed to an abstract idea.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1 and 15 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Jabri et al. (US 5640494, hereinafter ‘Jab’).
Regarding independent claim 1, Jab teaches a system, comprising: a plurality of inputs; a plurality of outputs; a learning network between the plurality of inputs and the plurality of outputs, the learning network including a plurality of layers, each of the plurality of layers including a weight layer including a plurality of weights coupled with an activation layer including a plurality of neurons configured to apply at least one activation function, a plurality of connections coupling the plurality of layers; (in 1:18-23: Neural networks [a system, comprising: a plurality of inputs; a plurality of outputs as the plurality of input and outputs of the neurons that make up a neural network system of the input and output layers] are made up of neurons connected to each other by synapses. The network may be layered, in which case only the input and output layers are visible to the outside world, and intermediate layers [a learning network between the plurality of inputs and the plurality of outputs, the learning network including a plurality of layers, each of the plurality of layers including a weight layer including a plurality of weights coupled with an activation layer including a plurality of neurons configured to apply at least one activation function] are "hidden" The function of the synapses (also referred to as weights) [each of the plurality of layers including a weight layer including a plurality of weights coupled with an activation layer including a plurality of neurons] is to amplify the signals passing through them, from one neuron to another, by a strength factor... And in 7:38-62: Neurons with programmable gain may also be trained using the weight perturbation [each of the plurality of layers including a weight layer including a plurality of weights coupled with an activation layer including a plurality of neurons] method by simply treating the programmable gain as an additional weight in the network. The neuron gain may either be applied to the net input of the neuron or the output of the neuron, and may attenuate or amplify the net input to a neuron, or the output value of a neuron. The advantages of such a feature, are … the facilitation of pruning as the gain reflects the extent of the participation of a neuron in the network, The node activation function [the learning network including a plurality of layers, each of the plurality of layers including a weight layer including a plurality of weights coupled with an activation layer including a plurality of neurons configured to apply at least one activation function] is given by, y.sub.i =G.sub.i .multidot.f.sub.i (g.sub.i .multidot.net.sub.i) where net.sub.i is, ##EQU11## f.sub.i () is the transfer function of node i, g.sub.i is the net input gain of node i and G.sub.i is the output gain of node i….
and a negative feedback network selectively couplable between the plurality of outputs and the plurality of inputs; (in 5:5:10: The weight perturbation technique may also be used to train multi-layer recurrent networks, and many artificial neural network models with feedback [and a negative feedback network selectively couplable between the plurality of outputs and the plurality of inputs], including: multi-layer neural networks; simple recurrent networks like Elman networks; and recurrent networks training to recognise temporal sequences (like Williams and Zipser networks) …; And in 1:65-2:15: … The Madaline Rule III has been suggested as a less expensive alternative to back-propagation for analog implementation. This rule evaluates the required derivatives using "node perturbation". This means that each neuron is perturbated by an amount .DELTA.net.sub.i, which produces a corresponding change in the TMSE. The change in value of the required strength factor .DELTA.w.sub.ij is estimated by the following equation: ##EQU1## where DELTA.E=E.sub.pert -E, i.e., the difference between the mean squared errors produced at the output of the network for a given pair of input and training signals [a negative feedback network selectively couplable between the plurality of outputs and the plurality of inputs] when a node is perturbated (E.sub.pert) and when it is not (E);…
wherein the plurality of weights are configured to be trained by providing input signals corresponding to a target output to the plurality of inputs, (in 2:44-67: The neurons of the input port are connected to the neurons of the output port by one or more paths, each of which comprises an alternating series of synapses (weights) and neurons. The weights amplify passing signals by a strength factor. A strength factor perturbating and refresh means applies perturbations to the strength factors of weights in the network, and updates the values of the strength factors [wherein the plurality of weights are configured to be trained by providing input signals corresponding to a target output to the plurality of inputs] depending upon the difference between signals appearing at the output port, for a given pair of input and training patterns [a target output to the plurality of inputs], when the weight is perturbated and when it is not…. The strength factor perturbating and refresh means preferably updates the values of strength factors of each weight in accordance with the signal representing the update value for that strength factor received from the multiplying means. )
measuring output signals at the plurality of outputs with the negative feedback network decoupled between the plurality of outputs and the plurality of inputs, perturbing the output signals by a plurality of perturbations with the negative feedback network coupled between the plurality of inputs and the plurality of outputs, (in 3:26-31: DELTA.E is E.sub.pert -E, i.e., the difference between the mean squared errors produced at the output of the network for a given pair of input and training patterns when a weight is perturbated (E.sub.pert) [perturbing the output signals by a plurality of perturbations with the negative feedback network coupled between the plurality of inputs and the plurality of outputs] and when it is not (E) [measuring output signals at the plurality of outputs with the negative feedback network decoupled between the plurality of outputs and the plurality of inputs when the weight between the input and output is not perturbed];…)
measuring corresponding perturbations for the plurality of connections, and updating the weights based on the corresponding perturbations, the plurality of perturbations being based on a difference between the plurality of output signals and the target output. 3:14-34: During training of the neural network the value of the strength factor of each weight is perturbated, and then updated [updating the weights based on the corresponding perturbations] by an amount depending on the signal representing the update value [measuring corresponding perturbations for the plurality of connections] for that strength factor. This technique, called `weight perturbation`, relies on its ability to approximate the required derivative, that is the change in the value of each strength factor required to update it [measuring corresponding perturbations for the plurality of connections, and updating the weights based on the corresponding perturbations], according to the following equation: ##EQU3## where DELTA.E is E.sub.pert -E, i.e., the difference between the mean squared errors produced at the output of the network for a given pair of input and training patterns [the plurality of perturbations being based on a difference between the plurality of output signals and the target output] when a weight is perturbated (E.sub.pert) and when it is not (E); DELTA..sub.pert w.sub.ij is the perturbation applied to the strength factor at weight w.sub.ij ; and DELTA.w.sub.ij is the update amount for the strength factor of w.sub.ij…)
Regarding independent claim 15, Jab teaches a method, comprising: providing, to a plurality of inputs of a network, input signals corresponding to a target output, the network including the plurality of inputs, a plurality of outputs, and a plurality of layers between the plurality of inputs and the plurality of outputs, each of the plurality of layers including a weight layer including a plurality of weights coupled with an activation layer including a plurality of neurons configured to apply at least one activation function, (in 1:18-23: Neural networks [providing, to a plurality of inputs of a network, input signals corresponding to a target output, the network including the plurality of inputs, a plurality of outputs, as the plurality of input and outputs of the neurons that make up a neural network system of the input and output layers] are made up of neurons connected to each other by synapses. The network may be layered, in which case only the input and output layers are visible to the outside world, and intermediate layers [and a plurality of layers between the plurality of inputs and the plurality of outputs, each of the plurality of layers including a weight layer including a plurality of weights coupled with an activation layer including a plurality of neurons configured to apply at least one activation function] are "hidden" The function of the synapses (also referred to as weights) [each of the plurality of layers including a weight layer including a plurality of weights coupled with an activation layer including a plurality of neurons] is to amplify the signals passing through them, from one neuron to another, by a strength factor... And in 7:38-62: Neurons with programmable gain may also be trained using the weight perturbation [each of the plurality of layers including a weight layer including a plurality of weights coupled with an activation layer including a plurality of neurons] method by simply treating the programmable gain as an additional weight in the network. The neuron gain may either be applied to the net input of the neuron or the output of the neuron, and may attenuate or amplify the net input to a neuron, or the output value of a neuron. The advantages of such a feature, are … the facilitation of pruning as the gain reflects the extent of the participation of a neuron in the network, The node activation function [the learning network including a plurality of layers, each of the plurality of layers including a weight layer including a plurality of weights coupled with an activation layer including a plurality of neurons configured to apply at least one activation function] is given by, y.sub.i =G.sub.i .multidot.f.sub.i (g.sub.i .multidot.net.sub.i) where net.sub.i is, ##EQU11## f.sub.i () is the transfer function of node i, g.sub.i is the net input gain of node i and G.sub.i is the output gain of node i….
a plurality of connections between the plurality of layers, the network settling at an operating point based on the input signals; measuring output signals at the plurality of outputs after the network has settled at the operating point; (in 3:26-31: DELTA.E is E.sub.pert -E, i.e., the difference between the mean squared errors [measuring output signals at the plurality of outputs after the network has settled at the operating point] produced at the output of the network for a given pair of input and training patterns when a weight is perturbated (E.sub.pert) and when it is not (E) [a plurality of connections between the plurality of layers, the network settling at an operating point based on the input signals when connection weight is settled at a unperturbed/perturb operating point];…; And in 2:44-53: The neurons of the input port are connected to the neurons of the output port by one or more paths, each of which comprises an alternating series of synapses (weights) [a plurality of connections between the plurality of layers, the network settling at an operating point based on the input signals] and neurons. The weights amplify passing signals by a strength factor. A strength factor perturbating and refresh means applies perturbations to the strength factors of weights in the network, and updates the values of the strength factors depending upon the difference between signals appearing at the output port, for a given pair of input and training patterns, when the weight is perturbated and when it is not [a plurality of connections between the plurality of layers, the network settling at an operating point based on the input signals]. )
perturbing, at the outputs, the operating point with a plurality of perturbations, a negative feedback network providing feedback, from the plurality of outputs to the plurality of inputs, based on the plurality of perturbations, the plurality of perturbations being based on a difference between the output signals and the target output, (3:8-13: Advantageously the neural network further comprises an output gain perturbating and refresh means to apply perturbations to output gains of neurons [perturbing, at the outputs, the operating point with a plurality of perturbations, a negative feedback network providing feedback, from the plurality of outputs to the plurality of inputs] in the network and no update the value of each output gain depending on the difference between the signals appearing at the output port, for a given pair of input [the plurality of perturbations being based on a difference between the output signals and the target output] and training patterns, when than output gain is perturbated and when it is not… )
measuring corresponding perturbations for the plurality of connections, and updating the weights based on the corresponding perturbations. 3:14-34: During training of the neural network the value of the strength factor of each weight is perturbated, and then updated [updating the weights based on the corresponding perturbations] by an amount depending on the signal representing the update value [measuring corresponding perturbations for the plurality of connections] for that strength factor. This technique, called `weight perturbation`, relies on its ability to approximate the required derivative, that is the change in the value of each strength factor required to update it [measuring corresponding perturbations for the plurality of connections, and updating the weights based on the corresponding perturbations], according to the following equation: ##EQU3## where DELTA.E is E.sub.pert -E, i.e., the difference between the mean squared errors produced at the output of the network for a given pair of input and training patterns when a weight is perturbated (E.sub.pert) and when it is not (E); DELTA..sub.pert w.sub.ij is the perturbation applied to the strength factor at weight w.sub.ij ; and DELTA.w.sub.ij is the update amount for the strength factor of w.sub.ij…)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 2-5, 9-12, 15-18 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Jab in view of Swenson et al. (US 5150450, hereinafter ‘Son’).
Regarding claim 2, the rejection of claim 1 is incorporated and Jab further teaches the system of claim 1, wherein the negative feedback network is an electrical negative feedback network, (in 2:11-14: DELTA.E=E.sub.pert -E, i.e., the difference between the mean squared errors produced at the output of the network for a given pair of input and training signals [wherein the negative feedback network is an electrical negative feedback network] when a node is perturbated (E.sub.pert) and when it is not (E)]; … Examiner notes that the disclosed signals are claimed electrical signals on hardware circuitry performed noted functions; wherein hardware operations is disclosed in 2:21-24: … In addition to the hardware needed for the operation of the network [the negative feedback network is an electrical negative feedback network], the implementation of the Madaline Rule III training for a neural network having N neurons in analog VLSI requires:…)
the input signals are electrical input signals, the output signals are electrical output signals, the plurality of perturbations is a plurality of electrical perturbations, the corresponding perturbations are corresponding electrical perturbations. (in 1:18-24: Neural networks are made up of neurons connected to each other by synapses [ the input signals are electrical input signals, the output signals are electrical output signals, the plurality of perturbations is a plurality of electrical perturbations, the corresponding perturbations are corresponding electrical perturbations]. The network may be layered, in which case only the input and output layers are visible to the outside world, and intermediate layers are "hidden". The function of the synapses (also referred to as weights) is to amplify the signals [wherein the negative feedback network is an electrical negative feedback network, the input signals are electrical input signals, the output signals are electrical output signals, the plurality of perturbations is a plurality of electrical perturbations, the corresponding perturbations are corresponding electrical perturbations] passing through them, from one neuron to another, by a strength factor.)
Additionally, Son teaches that hardware circuits for performing neural network signal processing operations, in 1:30-51: A representative ANN is depicted conceptually and conventionally in FIG. 1, and has two "neurons" 10, each providing an output 11 of the network, and four data inputs 12... It is evident that such complex networks are best practically implemented with very large scale integrated circuits [the negative feedback network is an electrical negative feedback network] and, typically, with complementary metal oxide semiconductor (CMOS) technology. Therefore, methods and circuits described herein, both of the prior art and of the present invention, should be considered as they might be implemented in very large scale integrated circuits in which several connected such layers each have thousands of synapses. And in 6:26-32: Another object is to provide methods and circuits facilitating such perturbation in artificial neural networks embodied in very large scale integrated circuits [wherein the negative feedback network is an electrical negative feedback network, the input signals are electrical input signals, the output signals are electrical output signals, the plurality of perturbations is a plurality of electrical perturbations, the corresponding perturbations are corresponding electrical perturbations]. Still another object is to provide such circuits particularly effective in an artificial neural network having synapse weight values corresponding to charges on floating gates of a metal oxide semiconductor structure… 7:26-58: The FIG. 3 ANN has six inputs or input terminals 110, 111 and 112, corresponding individually to the six synapses 102 of each neuron. Each of the terminals 110-112 corresponds structurally to one of the inputs 12 and is connected to the variable conductance 104 of the corresponding two of the synapses 102 by a conductor 115 which extends through these synapses... In the claims, a signal received at a terminal 112 is sometimes referred to an "input signal" with the associated conductor 115 referred to as an "input conductor" and the associated synapse 102 referred to as an "input synapse" Also in the claims, a signal received at terminal 110 is sometimes referred to as a "perturbation signal" with the associated conductor 115 being referred to as a "perturbation conductor" and the associated synapse 102 being referred to as a "perturbation synapse". The FIG. 3 ANN has a column or neuron select input 120, a neuron select multiplexer 121, neuron select conductors 122, a row or input select input 125, a row select multiplexer 126, row select conductors 127, a weight adjust amount input 135 and conductor 136, and a weight adjust select conductor 138 and input 139 corresponding to FIG. 1,…
Son and Jab are analogous art because both involve developing information retrieval and processing techniques using machine learning systems and algorithms.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of the prior art for retrieving and processing information using artificial neural networks embodied in very large-scale integrated circuits, as disclosed by Son with the method of developing information retrieval and processing techniques using artificial neural networks as disclosed by Jab.
One of ordinary skill in the arts would have been motivated to combine the methods disclosed by Son and Jab as noted above. Doing so allows for developing and implementing adaptive pattern recognizers capable of supervised or unsupervised memory modification or learning, (Son, 1:8-12)
Regarding claim 3, the rejection of claim 2 is incorporated and Jab in combination with Son further teaches the system of claim 2, wherein the electrical input signals and the electrical output signals have a dual relationship; and wherein, for being connected with the plurality of inputs and the plurality of outputs, the negative feedback network is configured to provide zero output signals for the outputs for duals of the electrical output signals and input electrical perturbations having the dual relationship with the electrical output signals. (in 2:44-67: The neurons of the input port are connected to the neurons of the output port by one or more paths, each of which comprises an alternating series of synapses (weights) and neurons. The weights amplify passing signals by a strength factor. A strength factor perturbating and refresh means applies perturbations to the strength factors of weights in the network, and updates the values of the strength factors depending upon the difference between signals [wherein the electrical input signals and the electrical output signals have a dual relationship; and wherein, for being connected with the plurality of inputs and the plurality of outputs] appearing at the output port [the plurality of outputs], for a given pair of input [the plurality of inputs] and training patterns, when the weight is perturbated and when it is not [the negative feedback network is configured to provide zero output signals for the outputs for duals of the electrical output signals and input electrical perturbations having the dual relationship with the electrical output signals]. The output port is preferably connected to a differencing means to provide an error signal, which represents the error produced at the output port, for a given pair of input and training patterns, when the strength factor of a weight is perturbated and when it is not.. The strength factor perturbating and refresh means preferably updates the values of strength factors of each weight in accordance with the signal representing the update value for that strength factor received from the multiplying means.)
Additionally, Son teaches: wherein the electrical input signals and the electrical output signals have a dual relationship; and wherein, for being connected with the plurality of inputs and the plurality of outputs, the negative feedback network is configured to provide zero output signals for the outputs for duals of the electrical output signals and input electrical perturbations having the dual relationship with the electrical output signals. (in 5:48-6:15: The subject invention is used in perturbation of individual output circuits or neurons of an artificial neural network for memory modification or learning by the network. The network has, conventionally, a plurality of the output circuits, a plurality of input terminals, and a plurality of addressable synapses individually connecting each input terminal to each output circuit. Each synapse has a weight which is selectively variable to control the contribution of a signal on the corresponding input terminal to a signal generated by the corresponding output circuit [wherein the electrical input signals and the electrical output signals have a dual relationship; and wherein, for being connected with the plurality of inputs and the plurality of outputs]... An output circuit may be selected for perturbation by loading the synapse connecting this circuit and the perturbation input with an appropriate weight while zeroing the weights of the synapses connecting the perturbation terminal to the other output circuits [the negative feedback network is configured to provide zero output signals for the outputs for duals of the electrical output signals and input electrical perturbations having the dual relationship with the electrical output signals]. Where the weighing is by a device damaged by repeated zeroing and reloading cycles, each synapse connected to the perturbation terminal may have an addressable switch through which the perturbation terminal is connected to the output circuit corresponding to the synapse, this switch being closed to perturb the corresponding output circuit and open at other times.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Son and Jab for the same reasons disclosed above.
Regarding claim 4, the rejection of claim 3 is incorporated and Jab in combination with Son further teaches the system of claim 3, wherein the electrical input signals are selected from voltage input signals and current input signals (in 6:34-40: Line 2 performs a forward pass through the network. This means that the effect of the application of the current input pattern p [wherein the electrical input signals are selected from voltage input signals and current input signals] is propagated through the network. Propagation takes place through all neurodes and synapses whether they are forward connections or recurrent connections (if recurrent connections exist,…)
and wherein the electrical output signals are current output signals for the voltage input signals and voltage output signals for the current input signals. (in 2:40-47: According to the present invention, there is provided a neural network of the type including an input port comprising one or more neurons (or neurodes) and an output port [wherein the electrical output signals are current output signals for the voltage input signals and voltage output signals for the current input signals] comprising one or more neurons. The neurons of the input port are connected to the neurons of the output port by one or more paths, each of which comprises an alternating series of synapses (weights) and neurons…6:34-40: Line 2 performs a forward pass through the network. This means that the effect of the application of the current input pattern p [wherein the electrical input signals are selected from voltage input signals and current input signals] is propagated through the network [wherein the electrical output signals are current output signals for the voltage input signals and voltage output signals for the current input signals]. Propagation takes place through all neurodes and synapses whether they are forward connections or recurrent connections (if recurrent connections exist,…))
Additionally, Son teaches the electrical signals as claimed …voltage … signals and current … signals, in 1:52-2:7: … In FIG. 1 each synapse 13 is depicted as having a variable conductance 15, represented by the conventional symbol for a variable resistance to indicate that a signal [voltage … signals and current … signals], typically a voltage signal [voltage … signals], on each input 12 determines a signal, typically a current signal [and current … signals], to a summing circuit 16 of each neuron 10. The variable conductance may be provided by an active circuit, and the synapse may function as a sink and/or a source of the current determined by the synapse... The output of each circuit 16 corresponds to the sum of the currents determined by the associated synapses and is provided to a function generator 18 which drives the corresponding output 11 to provide a, typically voltage, output signal […voltage … signals and current … signals]. This output signal thus corresponds to a predetermined relation or activation function applied to such sum of the currents […voltage … signals and current … signals]. It is well-known that by suitable adjustments to the weights, that is, modification of the "memory" of the network occurring when the memory "learns", the output signals may be made to assume desired values corresponding to predetermined values of the input signals…
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Son and Jab for the same reasons disclosed above.
Regarding claim 5, the rejection of claim 2 is incorporated and Jab in combination with Son further teaches the system of claim 2, wherein the negative feedback network includes a plurality of operational amplifiers having op-amp inputs configured to be selectively connected with the plurality of outputs and op-amp outputs configured to be selectively coupled with the plurality of inputs. (in 2:44-53: The neurons of the input port are connected to the neurons of the output port by one or more paths, each of which comprises an alternating series of synapses (weights) and neurons. The weights amplify passing signals by a strength factor [wherein the negative feedback network includes a plurality of operational amplifiers having op-amp inputs configured to be selectively connected with the plurality of outputs and op-amp outputs configured to be selectively coupled with the plurality of inputs.]. A strength factor perturbating and refresh means applies perturbations to the strength factors of weights in the network, and updates the values of the strength factors depending upon the difference between signals appearing at the output port, for a given pair of input and training patterns, when the weight is perturbated and when it is not [… having op-amp inputs configured to be selectively connected with the plurality of outputs and op-amp outputs configured to be selectively coupled with the plurality of inputs.].)
Additionally Son teaches wherein the negative feedback network includes a plurality of operational amplifiers having op-amp inputs configured to be selectively connected with the plurality of outputs and op-amp outputs configured to be selectively coupled with the plurality of inputs, in 14:63-15:10: With an ANN having the circuits f FIGS. 4 and 5, the neuron already selected is perturbed, as indicated in block 235 of FIG. 8B, simply by providing a perturbation "on" signal to a terminal such as terminal 165 causing switch 150 or 151 of the perturbation synapse 152 or 153 of the neuron to connect the perturbation signal conductor 157 to nodes 55 and 56 and thus to the summing and output circuits, such as amplifiers [wherein the negative feedback network includes a plurality of operational amplifiers having op-amp inputs] 58 and 59 of FIG. 2, and thus pass the effect of a signal [op-amp inputs configured to be selectively connected with the plurality of outputs and op-amp outputs configured to be selectively coupled with the plurality of inputs] on a conductor 157 on to nodes 55 and 56. However, the switches 150 or 151 of the other synapses, which receive the perturbation signal from conductor 157 but are located in other neurons than the current or selected neuron [op-amp inputs configured to be selectively connected with the plurality of outputs and op-amp outputs configured to be selectively coupled with the plurality of inputs], continue to block the effect of this signal on nodes 55 and 56 so that the output signals of these other neurons are not perturbed.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Son and Jab for the same reasons disclosed above.
Regarding independent claim 9, Jab teaches a learning network, comprising: a plurality of inputs; a plurality of outputs; a plurality of weight layers; a plurality of activation layers interleaved with the plurality of weight layers, the plurality of weight layers and the plurality of activation layers being between the plurality of inputs and the plurality of outputs, each of the plurality of weights layer including a plurality of weights, each of the plurality of activation layers an activation layer including a plurality of neurons configured to apply at least one activation function (in 1:18-23: Neural networks [a system, comprising: a plurality of inputs; a plurality of outputs as the plurality of input and outputs of the neurons that make up a neural network system of the input and output layers] are made up of neurons connected to each other by synapses. The network may be layered, in which case only the input and output layers are visible to the outside world, and intermediate layers [a learning network, comprising: a plurality of inputs; a plurality of outputs; a plurality of weight layers; a plurality of activation layers interleaved with the plurality of weight layers, the plurality of weight layers and the plurality of activation layers being between the plurality of inputs and the plurality of outputs, each of the plurality of weights layer including a plurality of weights, each of the plurality of activation layers an activation layer including a plurality of neurons configured to apply at least one activation function] are "hidden" The function of the synapses (also referred to as weights) [the plurality of weight layers and the plurality of activation layers being between the plurality of inputs and the plurality of outputs, each of the plurality of weights layer including a plurality of weights] is to amplify the signals passing through them, from one neuron to another, by a strength factor... And in 7:38-62: Neurons with programmable gain may also be trained using the weight perturbation [the plurality of weight layers and the plurality of activation layers being between the plurality of inputs and the plurality of outputs, each of the plurality of weights layer including a plurality of weights] method by simply treating the programmable gain as an additional weight in the network. The neuron gain may either be applied to the net input of the neuron or the output of the neuron, and may attenuate or amplify the net input to a neuron, or the output value of a neuron. The advantages of such a feature, are … the facilitation of pruning as the gain reflects the extent of the participation of a neuron in the network, The node activation function [a learning network, comprising: …, the plurality of weight layers and the plurality of activation layers being between the plurality of inputs and the plurality of outputs, each of the plurality of weights layer including a plurality of weights, each of the plurality of activation layers an activation layer including a plurality of neurons configured to apply at least one activation function] is given by, y.sub.i =G.sub.i .multidot.f.sub.i (g.sub.i .multidot.net.sub.i) where net.sub.i is, ##EQU11## f.sub.i () is the transfer function of node i, g.sub.i is the net input gain of node i and G.sub.i is the output gain of node I [a plurality of inputs; a plurality of outputs; a plurality of weight layers; a plurality of activation layers interleaved with the plurality of weight layers]….
and a negative electrical feedback network selectively couplable between the plurality of outputs and the plurality of inputs; (in 5:5:10: The weight perturbation technique may also be used to train multi-layer recurrent networks, and many artificial neural network models with feedback [and a negative electrical feedback network selectively couplable between the plurality of outputs and the plurality of inputs], including: multi-layer neural networks; simple recurrent networks like Elman networks; and recurrent networks training to recognise temporal sequences (like Williams and Zipser networks) …; And in 1:65-2:15: … The Madaline Rule III has been suggested as a less expensive alternative to back-propagation for analog implementation. This rule evaluates the required derivatives using "node perturbation". This means that each neuron is perturbated by an amount .DELTA.net.sub.i, which produces a corresponding change in the TMSE. The change in value of the required strength factor .DELTA.w.sub.ij is estimated by the following equation: ##EQU1## where DELTA.E=E.sub.pert -E, i.e., the difference between the mean squared errors produced at the output of the network for a given pair of input and training signals [a negative feedback network selectively couplable between the plurality of outputs and the plurality of inputs] when a node is perturbated (E.sub.pert) and when it is not (E);…
wherein the plurality of weights are configured to be trained by providing electrical input signals corresponding to a target output to the plurality of inputs, (in 2:44-67: The neurons of the input port are connected to the neurons of the output port by one or more paths, each of which comprises an alternating series of synapses (weights) and neurons. The weights amplify passing signals by a strength factor. A strength factor perturbating and refresh means applies perturbations to the strength factors of weights in the network, and updates the values of the strength factors [wherein the plurality of weights are configured to be trained by providing electrical input signals corresponding to a target output to the plurality of inputs] depending upon the difference between signals appearing at the output port, for a given pair of input and training patterns [a target output to the plurality of inputs], when the weight is perturbated and when it is not…. The strength factor perturbating and refresh means preferably updates the values of strength factors of each weight in accordance with the signal representing the update value for that strength factor received from the multiplying means. )
measuring electrical output signals at the plurality of outputs with the negative electrical feedback network electrically decoupled between the plurality of outputs and the plurality of inputs, perturbing the electrical output signals by a plurality of electrical perturbations with the negative electrical feedback network electrically coupled between the plurality of inputs and the plurality of outputs, (in 3:26-31: DELTA.E is E.sub.pert -E, i.e., the difference between the mean squared errors produced at the output of the network for a given pair of input and training patterns when a weight is perturbated (E.sub.pert) [perturbing the electrical output signals by a plurality of electrical perturbations with the negative electrical feedback network electrically coupled between the plurality of inputs and the plurality of outputs] and when it is not (E) [measuring electrical output signals at the plurality of outputs with the negative electrical feedback network electrically decoupled between the plurality of outputs and the plurality of inputs, when the weight between the input and output is not perturbed];…)
measuring corresponding electrical perturbations between the plurality of weight layers and the plurality of activation layers, and updating the weights based on the corresponding electrical perturbations, the plurality of electrical perturbations being based on a difference between the plurality of electrical output signals and the target output.3:14-34: During training of the neural network the value of the strength factor of each weight is perturbated, and then updated [updating the weights based on the corresponding perturbations] by an amount depending on the signal representing the update value [measuring corresponding perturbations for the plurality of connections] for that strength factor. This technique, called `weight perturbation`, relies on its ability to approximate the required derivative, that is the change in the value of each strength factor required to update it [measuring corresponding electrical perturbations between the plurality of weight layers and the plurality of activation layers, and updating the weights based on the corresponding electrical perturbations], according to the following equation: ##EQU3## where DELTA.E is E.sub.pert -E, i.e., the difference between the mean squared errors produced at the output of the network for a given pair of input and training patterns [the plurality of electrical perturbations being based on a difference between the plurality of electrical output signals and the target output] when a weight is perturbated (E.sub.pert) and when it is not (E); DELTA..sub.pert w.sub.ij is the perturbation applied to the strength factor at weight w.sub.ij ; and DELTA.w.sub.ij is the update amount for the strength factor of w.sub.ij…)
Additionally, Son teaches the electrical signals as claimed …electrical … signals, in 1:52-2:7: … In FIG. 1 each synapse 13 is depicted as having a variable conductance 15, represented by the conventional symbol for a variable resistance to indicate that a signal […electrical … signals], typically a voltage signal […electrical … signals], on each input 12 determines a signal, typically a current signal, to a summing circuit 16 of each neuron 10. The variable conductance may be provided by an active circuit, and the synapse may function as a sink and/or a source of the current determined by the synapse... The output of each circuit 16 corresponds to the sum of the currents determined by the associated synapses and is provided to a function generator 18 which drives the corresponding output 11 to provide a, typically voltage, output signal […electrical … signals]. This output signal thus corresponds to a predetermined relation or activation function applied to such sum of the currents. It is well-known that by suitable adjustments to the weights, that is, modification of the "memory" of the network occurring when the memory "learns", the output signals may be made to assume desired values corresponding to predetermined values of the input signals…
Additionally, Son teaches that hardware circuits for performing neural network signal processing operations, in 1:30-51: A representative ANN is depicted conceptually and conventionally in FIG. 1, and has two "neurons" 10, each providing an output 11 of the network, and four data inputs 12... It is evident that such complex networks are best practically implemented with very large scale integrated circuits [and a negative electrical feedback network selectively couplable between the plurality of outputs and the plurality of inputs] and, typically, with complementary metal oxide semiconductor (CMOS) technology. Therefore, methods and circuits described herein, both of the prior art and of the present invention, should be considered as they might be implemented in very large scale integrated circuits in which several connected such layers each have thousands of synapses. And in 6:26-32: Another object is to provide methods and circuits facilitating such perturbation in artificial neural networks embodied in very large scale integrated circuits. Still another object is to provide such circuits particularly effective in an artificial neural network having synapse weight values corresponding to charges on floating gates of a metal oxide semiconductor structure… 7:26-58: The FIG. 3 ANN has six inputs or input terminals 110, 111 and 112, corresponding individually to the six synapses 102 of each neuron. Each of the terminals 110-112 corresponds structurally to one of the inputs 12 and is connected to the variable conductance 104 of the corresponding two of the synapses 102 by a conductor 115 which extends through these synapses... In the claims, a signal received at a terminal 112 is sometimes referred to an "input signal" with the associated conductor 115 referred to as an "input conductor" and the associated synapse 102 referred to as an "input synapse" Also in the claims, a signal received at terminal 110 is sometimes referred to as a "perturbation signal" with the associated conductor 115 being referred to as a "perturbation conductor" and the associated synapse 102 being referred to as a "perturbation synapse". The FIG. 3 ANN has a column or neuron select input 120, a neuron select multiplexer 121, neuron select conductors 122, a row or input select input 125, a row select multiplexer 126, row select conductors 127, a weight adjust amount input 135 and conductor 136, and a weight adjust select conductor 138 and input 139 corresponding to FIG. 1,…
Son and Jab are analogous art because both involve developing information retrieval and processing techniques using machine learning systems and algorithms.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of the prior art for retrieving and processing information using artificial neural networks embodied in very large-scale integrated circuits, as disclosed by Son with the method of developing information retrieval and processing techniques using artificial neural networks as disclosed by Jab.
One of ordinary skill in the arts would have been motivated to combine the methods disclosed by Son and Jab as noted above. Doing so allows for developing and implementing adaptive pattern recognizers capable of supervised or unsupervised memory modification or learning, (Son, 1:8-12)
Regarding claims 10-12, the limitations are similar to those in claims 3-5 respectively, and are thus rejected under the same rationale.
Regarding claims 16-17, the limitations are similar to those in claims 2-3 respectively, and are thus rejected under the same rationale.
Regarding claim 18, the limitations are similar to those in claim 5, and are thus rejected under the same rationale
Regarding claim 20, the rejection of claim 16 is incorporated and Jab in combination with Son teaches the method of claim 16, further comprising: repeating the providing, measuring, perturbing, measuring, and updating. (in 5:59-65: When the strength factors of the weights are perturbated, the error gradient and magnitude are identified by the differencing means 8, and the value of the strength factor of each weight is then refreshed. The strength factors may be updated repeatedly [further comprising: repeating the providing, measuring, perturbing, measuring, and updating] until the error is within preset limits. At this point training is finished and the network is ready for operation. )
Claims 6-7 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Jab in view of Son and in further view Yamada et al. (US 20220005866, hereinafter ‘Yam’).
Regarding claim 6, the rejection of claim 2 is incorporated and Jab in combination with Son further teaches the system of claim 2, wherein each of the plurality of layers has a first width. (in 1:18-22: Neural networks are made up of neurons connected to each other by synapses. The network may be layered, in which case only the input and output layers are visible to the outside world, and intermediate layers are "hidden" [wherein each of the plurality of layers has a first width as the number of nodes in set for hidden layers as claimed plurality of layers having a first width].)
Additionally, Yam teaches wherein each of the plurality of layers has a first width as claimed plurality of layers having a first width depicted in Fig. 7: Lm2-3:
PNG
media_image1.png
610
634
media_image1.png
Greyscale
[0104] FIG. 7 is a schematic view of a neural network NN. The neural network NN includes an input layer L.sub.in, intermediate layers L.sub.in, and an output layer L.sub.out. In FIG. 7, although an example in which the number of the intermediate layers L.sub.m is three is presented, the number of intermediate layers L.sub.m is arbitrary. Each of the input layer L.sub.in, the intermediate layers L.sub.m, and the output layer L.sub.out has a plurality of chips C, and each of the chips C corresponds to a neuron in a brain [wherein each of the plurality of layers has a first width]. Each of the input layer L.sub.in, the intermediate layers L.sub.m, and the output layer L.sub.out is connected to a transmission means. The transmission means corresponds to a synapse in a brain. In the neural network NN, the transmission means (synapse) performs learning, whereby a correct answer ratio of a problem increases. The learning is finding a knowledge that can be used in the future from information…
Yam, Son and Jab are analogous art because both involve developing information retrieval and processing techniques using machine learning systems and algorithms.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of the prior art for retrieving and processing information using artificial neural provided using neuromorphic device having circuits, as disclosed by Yam with the method of developing information retrieval and processing techniques using artificial neural networks as collectively disclosed by Son and Jab.
One of ordinary skill in the arts would have been motivated to combine the methods disclosed by Yam, Son and Jab as noted above. Doing so allows for developing and implementing more complex machine learning operations and improve power of expression for representing weights using neuromorphic devices, (Yam, 0136 & 0104)
Regarding claim 7, the rejection of claim 6 is incorporated and Jab in combination with Son and Yam further teaches the system of claim 6, wherein the system includes: at least one additional layer having a second width different from the first width, the at least one additional layer including at least one of an additional weight layer or an additional activation layer. (in 1:18-22: Neural networks are made up of neurons connected to each other by synapses. The network may be layered, in which case only the input and output layers are visible to the outside world, and intermediate layers are "hidden" [wherein the system includes: at least one additional layer having a second width different from the first width, the at least one additional layer including at least one of an additional weight layer or an additional activation layer as additional hidden layer claimed with having a second width different from the first width].)
Additionally, Yam teaches wherein the system includes: at least one additional layer having a second width different from the first width, the at least one additional layer including at least one of an additional weight layer or an additional activation layer as claimed plurality of layers having a first width depicted in Fig. 7: Lm1:
PNG
media_image1.png
610
634
media_image1.png
Greyscale
[0104] … The first element group of the first hierarchical layer L1 performs a product-sum operation having a positive weight. The first hierarchical layer L1 is called an activation layer [the at least one additional layer including at least one of an additional weight layer or an additional activation layer]. …
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Yam, Son and Jab for the same reasons disclosed above.
Regarding claim 13, the limitations are similar to those in claims 6 and 7, and are thus rejected under the same rationale
Claims 8, 14 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Jab in view of Son and in further view Ambrogio et al. (US 20230306251, hereinafter ‘Gio’).
Regarding claim 8, the rejection of claim 1 is incorporated and Jab in combination with Son further teaches the system of claim 1, wherein the at least one activation function is at least one . (in 7:51-55: The node activation function [wherein the at least one activation function is at least one ] is given by, y.sub.i =G.sub.i .multidot.f.sub.i (g.sub.i .multidot.net.sub.i)…)
Gio teaches wherein the at least one activation function is at least one invertible activation function, in [0029] In the artificial neural network 124, each artificial neuron 126 implements an activation function which defines an output of the node given an input or set of inputs to the node. For example, depending on the given application and the type of artificial neural network 124, the activation functions implemented by the artificial neurons 126 can include one or more types of non-linear activation functions including, but not limited to, a rectified linear unit (ReLU) activation function, a clamped ReLU activation function, a sigmoid activation function [wherein the at least one activation function is at least one invertible activation function], a hyperbolic tangent (tanh) activation function [wherein the at least one activation function is at least one invertible activation function], a softmax activation function, etc. ...
Gio, Son and Jab are analogous art because both involve developing information retrieval and processing techniques using machine learning systems and algorithms.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of the prior art for retrieving and processing information systems for neuromorphic computing, and techniques for implementing and calibrating hardware activation functions, as disclosed by Gio with the method of developing information retrieval and processing techniques using artificial neural networks as collectively disclosed by Son and Jab.
One of ordinary skill in the arts would have been motivated to combine the methods disclosed by Gio, Son and Jab as noted above. Doing so allows for developing and implementing artificial neuron activation function configured to transform the inputs to the artificial neuron into an output value or “activation” of the given artificial neuron in neuromorphic computing systems, (Gio, Abstract & 0002)
Regarding claim 14, the limitations are similar to those in claim 8, and are thus rejected under the same rationale.
Regarding claim 19, the limitations are similar to those in claim 8, and are thus rejected under the same rationale.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Colbert et al. (US 20240193413): teaches that different hidden layers of the neural network can have different data sizes (different values of the bit width P) of the accumulator registers 250 such that neurons in different hidden layers use different values of the bit width P.
Girka et al. (NPL: Anonymization as homeomorphic data space transformation for privacy-preserving deep learning): teaches invertible activation function includes sigmoid or hyperbolic tangent, in pg. 870)
Chalana et al. (US 20220215052): teaches for a neural network, a maximum number of nodes in each layer, also referred to as a “width” of a neural network
Gokmen et al. (US 20230195832): teaches a system, comprising: a plurality of inputs; a plurality of outputs; a learning network between the plurality of inputs and the plurality of outputs, the learning network including a plurality of layers, each of the plurality of layers including a weight layer including a plurality of weights coupled with an activation layer including a plurality of neurons configured to apply at least one activation function, a plurality of connections coupling the plurality of layers; (in [0019] ... In particular, FIG. 1 schematically illustrates a computing system 100 which comprises a digital processing system 110, and a neuromorphic computing system 120. The digital processing system 110 comprises a plurality of processors 112. The neuromorphic computing system 120 comprises a plurality of neural cores 122. The neural cores 122 are configured to implement an artificial neural network [a system, comprising] 124 which comprises artificial neurons 126, and artificial synaptic device arrays 128. The artificial neural network 124 can be any type of neural network including, but not limited to, a feed-forward neural network (e.g., a Deep Neural Network (DNN), a Convolutional Neural Network (CNN), etc.), a Recurrent Neural Network (RNN) (e.g., a Long Short-Term Memory (LSTM) neural network), etc… [0020] In general, the artificial neural network 124 comprises a plurality of layers [the learning network including a plurality of layers] which comprise the artificial neurons 126, wherein the layers include an input layer, an output layer, and one or more hidden model layers between the input and output layers [a plurality of inputs; a plurality of outputs]. Each layer is connected to another layer using an array of artificial synaptic devices which provide synaptic weights [each of the plurality of layers including a weight layer including a plurality of weights coupled with an activation layer including a plurality of neurons configured to apply at least one activation function] that represent connection strengths [a plurality of connections coupling the plurality of layers] between artificial neurons in one layer with the artificial neurons in another layer…[0021] More specifically, depending on the type of artificial neural network, the layers of the artificial neural network 124 can include functional layers including, but not limited to, fully connected layers, activation layers, convolutional layers, pooling layers, normalization layers, etc... An activation layer in a neural network comprises activation functions [a plurality of neurons configured to apply at least one activation function] which define how a weighted sum of an input is transformed into an output from a node or nodes in a layer of the network [a plurality of neurons configured to apply at least one activation function]. For example, activation functions include, but are not limited to, a rectifier or ReLU activation function, a sigmoid activation function, a hyperbolic tangent (tanH) activation function, a softmax activation function, etc.
and a negative feedback network selectively couplable between the plurality of outputs and the plurality of inputs; (in [0045] As further schematically shown in FIG. 4A for illustrative purposes, the current integrator circuit 426-m comprises an operational amplifier 440 (e.g., operational transconductance amplifier (OTA)), and an integrating capacitor 442. The integrating capacitor 442 is connected in a negative feedback path [and a negative feedback network selectively couplable between the plurality of outputs and the plurality of inputs] between input and output nodes N1 and N2 of the operational amplifier 440…)
wherein the plurality of weights are configured to be trained by providing input signals corresponding to a target output to the plurality of inputs, (in [0054] After the backward pass operation is completed on the given RPU array 405, a weight update process is performed to tune the conductance values of the RPU cells 410 (and thus update the weight values of the given synaptic weight matrix W) based on the forward-propagated digital vector x=[x.sub.1, x.sub.2, . . . , x.sub.n] (FIG. 4A) and the backward-propagated digital error vector x.sub.err=[x.sub.1, x.sub.2, . . . , x.sub.m], which were previously input to the given RPU array 405 during the forward and backward pass operations.… The vector cross product operations for the weight update operation are implemented based on the known concept that coincidence detection (using an AND logic gate operation) of stochastic streams representing real numbers is equivalent to a multiplication operation.)
Any inquiry concerning this communication or earlier communications from the examiner should be directed to OLUWATOSIN ALABI whose telephone number is (571)272-0516. The examiner can normally be reached Monday-Friday, 8:00am-5:00pm EST..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael Huntley can be reached at (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/OLUWATOSIN ALABI/Primary Examiner, Art Unit 2129
Prosecution Insights