DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1, 4-5, 8-10, 13-14, and 17-18 are presented for examination.
Response to Amendment
Applicant’s amendment appears to have obviated the specification and claim objections. Therefore, those objections are withdrawn.
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Claim Rejections - 35 USC § 112
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claim 13 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 13 is rejected for being dependent on canceled claim 12. For purposes of examination, Examiner will presume that claim 13 was intended to be dependent on claim 10.
Claim Rejections - 35 USC § 103
Claims 1, 4-5, 8, 10, 13-14, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Yang et al. (US 20190244088) (“Yang”) in view of Bingham et al. (US 20220051076) (“Bingham”).
Regarding claim 1, Yang discloses “[a] neuromorphic device comprising:
a synaptic array (Yang Figs. 6a-d show a memristive crossbar array with memristors 602a-c [synaptic array]); and
a neuron circuit connected to the synaptic array, configured to accumulate an output current of the synaptic array, and configured to output a spike pulse when the accumulated current exceeds a threshold value (Yang Figs. 6a-d show neurons 604a-c [neuron circuit] connected to the synaptic array; paragraph 14 discloses that a neuron integrates [accumulates] inputs received through synapses [outputs of synaptic array] and generates an output signal if a threshold has been reached within a defined time interval; see also paragraph 28 (disclosing that the memristor device fires a current pulse and that the output is a spike)), wherein the neuron circuit includes a discharge switching element for discharging the accumulated current, the discharge switching element including a synaptic element of a same type as an element included in the synaptic array (Yang paragraph 28 discloses that a current spike across a diffusive memristor coincides with the discharging of a capacitor and the active release of the charge stored in the capacitor [i.e., the memristor triggers the discharge of the current and is therefore a discharge switching element of the same type [i.e., memristor] of the synapses [elements] in the array]),
wherein the discharge switching element compensates1 for a change in synaptic weight stored in the synaptic array according to a change in external environment (spike-timing-dependent-plasticity is a prevalent protocol for synaptic weight update in neural networks [i.e., the synaptic array stores synaptic weights that are changed] – Yang, paragraph 40; spike across a diffusive memristor coincides with the discharging of a capacitor – id. at paragraph 39 [i.e., the spike resulting from the discharge updates the synaptic weights]; paragraph 14 discloses that a neuron integrates inputs received through synapses and generates an output signal [i.e., the spike that updates the weights] if a threshold has been reached within a defined time interval [passage of time = change in external environment]) ….”
Yang appears not to disclose explicitly the further limitations of the claim. However, Bingham discloses that “the discharge switching element provides an effect of adjusting a gradient of an activation function implemented by the neuron circuit according to a change in external environment (as training progresses, gradient descent makes small adjustments to the function parameters alpha, beta, and gamma, resulting in activation functions that change over time [time change = change in external environment] – Bingham, paragraph 56; see also paragraph 13 (indicating that the process is computer-implemented, i.e., it is implemented on a processor that contains elements for discharging current that are used in adjusting the gradient)).”
Bingham and the instant application both relate to activation functions for machine learning and are analogous. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Yang to include activation functions that change with changes in environment, as disclosed by Bingham, and an ordinary artisan could reasonably expect to have done so successfully. Doing so would increase the flexibility of the system by allowing the activation function to change with changed circumstances. See Bingham, paragraph 56.
Claim 10 is a circuit claim corresponding to device claim 1 and is rejected for the same reasons as given in the rejection of that claim.
Regarding claim 4, the rejection of claim 1 is incorporated. Yang further discloses a “discharge switching element,” as shown above in the rejection of claim 1.
Yang appears not to disclose explicitly the further limitations of the claim. However, Bingham discloses “adjust[ing] the gradient of the activation function implemented by the neuron circuit according to the change in external environment (as training progresses, gradient descent makes small adjustments to the function parameters alpha, beta, and gamma, resulting in activation functions that change over time [time change = change in external environment] – Bingham, paragraph 56), reduc[ing] the gradient of the activation function according to an operation … when an environmental change in which a weight stored in a synapse increases occurs, and increas[ing] the gradient of the activation function according to the operation … when the environmental change in which the weight stored in the synapse decreases occurs (as training progresses, gradient descent makes small adjustments to the function parameters alpha, beta, and gamma, resulting in activation functions that change over time [time change = change in external environment] – Bingham, paragraph 56; see also Figs. 5(a)-(e) (showing that the activation function shapes change over time; note that the change could be in either direction, such that in general a forward lapse of time [increasing environmental change] will yield a reduction in gradient and a backward lapse of time [decreasing environmental change] will yield an increase in gradient)).” It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Yang to decrease the activation gradient with an increase in time and vice versa, as disclosed by Bingham, and an ordinary artisan could reasonably expect to have done so successfully. Doing so would increase the flexibility of the system by allowing the activation function to change with changed circumstances. See Bingham, paragraph 56.
Claim 13 is a circuit claim corresponding to device claim 4 and is rejected for the same reasons as given in the rejection of that claim.
Regarding claim 5, Yang/Bingham discloses that “a temperature change or a time change is reflected in an external environment change factor (Yang paragraph 14 discloses that a neuron integrates inputs received through synapses and generates an output signal if a threshold has been reached within a defined time interval [passage of time = external environment change factor]).”
Claim 14 is a circuit claim corresponding to device claim 5 and is rejected for the same reasons as given in the rejection of that claim.
Regarding claim 8, Yang/Bingham discloses that “the neuron circuit includes:
an accumulator configured to be driven according to the output current of the synaptic array and configured to charge a charging element with the output current of the synaptic array (when input pulses from a voltage source are applied to a memristor device of a circuit, the circuit capacitance charges [capacitor = charging element; the fact that the neuron is responsible for charging the capacitor suggests that it contains an accumulator for that purpose] – Yang, paragraph 27; see also paragraphs 32-33 (disclosing that the synapses in the synapse array emit spikes [output currents] that are applied to the neurons));
a spike generator configured to transmit a charging voltage of the charging element to an output terminal thereof (when input pulses from the voltage source are applied to a memristor device of the circuit, the circuit capacitance changes; if the threshold is reached, the memristor is fired [i.e., the memristor functions as a spike generator and the spike/charging voltage is transmitted to an output] and the capacitor [charging element] is discharged –Yang, paragraph 27); and
a discharger configured to discharge the charging element by using the discharge switching element during a spike generating operation of the spike generator (when input pulses from the voltage source are applied to a memristor device of the circuit, the circuit capacitance changes; if the threshold is reached, the memristor is fired [i.e., the memristor functions as a spike generator] and the capacitor [charging element] is discharged –Yang, paragraph 27).”
Claim 17 is a circuit claim corresponding to device claim 8 and is rejected for the same reasons as given in the rejection of that claim.
Claims 9 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Yang in view of Bingham and further in view of Tran et al. (US 20210174185) (“Tran”).
Regarding claim 9, the rejection of claim 8 is incorporated. Yang further discloses a “spike generator”, as shown in the rejection of claim 8.
Yang/Bingham appears not to disclose explicitly the further limitations of the claim. However, Tran discloses that “the discharger includes a current mirror configured with a first switching element and a second switching element, and the discharge switching element (current sample and hold circuit for neuron output comprises a first switch, a second switch, and a capacitor that discharges into the gate of an output transistor – Tran, claim 5; see also paragraph 11 (disclosing the generation of a mirrored current of the neuron circuit)),
the discharge switching element comprises a first terminal connected to a power source and a second terminal connected to the second switching element (capacitor comprises a first terminal and a second terminal, and the capacitor may discharge into a gate of an output transistor [part of second switching element, see below mapping; note also that the fact that the capacitor has charge to discharge indicates that it is connected to a power source] – Tran, claim 5),
the discharge switching element is switched by the output terminal (in a first mode the switches are closed and couple input current to input transistors and input transistors to capacitors and output transistors, and in a second mode the first switch is open and the second switch is open and the capacitor discharges into the gate of the output transistor [i.e., whether the switches connecting the output of the input transistor to the capacitor are open or not determines whether a discharge occurs, so the input transistor’s output terminal plays a role in the switching] – Tran, claim 5) …,
a first terminal of the first switching element is connected to an output terminal of the charging element, and a second terminal of the first switching element is grounded (input transistor comprises a first terminal, a second terminal connected to ground, and a gate; first switch couples an input current to the first terminal of the input transistor and the gate of the input transistor, and the second switch couples the first terminal of the input transistor to the first [output] terminal of the capacitor [charging element] [i.e., the input transistor plus second switch comprise a first switching element] – Tran, claim 5), and
a first terminal of the second switching element is connected to the second terminal of the discharge switching element, and a second terminal of the second switching element is grounded (output transistor comprises a first terminal providing an output current, a second terminal connected to ground, and a gate; first switch couples an input current to the first terminal of the input transistor and the gate of the input transistor; and the second switch couples the first terminal of the input transistor to the first terminal of the capacitor and the gate of the output transistor [i.e., insofar as the second switch couples the first transistor, which functions as a discharge switching element, to the output transistor, which is connected to ground, the second switching element may be regarded as the combination of the second switch and the output transistor] – Tran, claim 5).”
Tran and the instant application both relate to physical instantiations of neural networks and are analogous. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Yang/Bingham to configure the circuit with two switching elements and a discharge switching element that are connected to ground, as disclosed by Tran, and an ordinary artisan could reasonably expect to have done so successfully. Doing so would allow the output current to be kept constant regardless of loading via the current mirror. See Tran, claim 5.
Claim 18 is a circuit claim corresponding to device claim 9 and is rejected for the same reasons as given in the rejection of that claim.
Response to Arguments
Applicant's arguments filed December 23, 2025 (“Remarks”) have been fully considered but they are not persuasive.
Applicant argues that Yang allegedly does not read on the claims as amended because the claims recite a discharge switching element that is a distinct circuit element of a same type as the synaptic devices in the synapse array, whereas Yang discloses a system comprising memristors acting as synapses that trigger the discharge of capacitors. Applicant specifically disagrees with Examiner’s contention that the memristor itself can be a “discharge switching element” because it triggers the discharge of the capacitor. Remarks at 12-13. However, Applicant has not defined “discharge switching element,” and in the absence of a definition, it may be construed to mean any element that triggers a discharge. For the reasons given in the rejection, the memristors of Yang fit that definition. Moreover, there are multiple memristors in the system, all of which perform both synaptic and discharge switching functions. Thus, to the extent that the claims must be read as requiring that the discharge switching element be physically separate from the synapse, one of the memristors may qualify as the claimed “discharge switching element” and another may be the “element included in the synaptic array”. Even assuming arguendo that the memristors cannot read on both discharge switching elements and elements of synaptic arrays, which Examiner does not concede, Applicant is also not defining the term “same type.” Thus, even if one assumes that it is the capacitors of Yang, and not the memristors, that correspond to the claimed “discharge switching elements,” the capacitors and memristors are both of the “same type” insofar as they are both fundamental circuit elements through which charge passes.
Applicant then argues that Yang does not read on compensating for a change in synaptic weight according to a change in external environment because the system of Yang merely passively generates spikes in response to changes in spike timing, whereas the instant claims actively adjust the gradient of an activation function according to such changes. Remarks at 13. However, the claims are silent as to whether the mechanism for changing the synaptic weights is passive or active. Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). Moreover, Examiner does not rely on Yang to teach the claimed adjustment of gradients of an activation function, but rather relies on Bingham for that claim element. One cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
Finally, Applicant argues that Bingham allegedly does not disclose the claimed gradient adjustment because the gradient adjustment of Bingham occurs at the software level, whereas that of the claimed invention is at the hardware level. Remarks at 14. However, here again Applicant is arguing that the references fail to disclose a feature that is not claimed. At most, the claims require that the discharge switching element have the effect of adjusting a gradient of an activation function, not that the gradient adjustment actually occur at the hardware level. Since Bingham discloses that the system is computer-implemented, as shown in the rejection above, it follows that it is implemented by some processor that contains elements through which current flows, i.e., discharge switching elements. Even assuming arguendo that Bingham does not disclose discharge switching elements, which Examiner does not concede, Yang clearly does, and an ordinary artisan could modify Yang so that its discharge switching elements play some role in gradient adjustment, as disclosed by Bingham.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RYAN C VAUGHN whose telephone number is (571)272-4849. The examiner can normally be reached M-R 7:00a-5:00p ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kamran Afshar, can be reached at 571-272-7796. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RYAN C VAUGHN/ Primary Examiner, Art Unit 2125
1 It is unclear what the language “compensates for” means in this context, since there does not appear to be any quantity that would need to be “compensate[d] for” as a result of the change in the synaptic weights. Examiner is interpreting this language as meaning that the element changes the synaptic weight according to a change in the external environment.
Prosecution Insights