Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 8 and 16 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claims 8 and 16, “the pre-synaptic neuron” lacks antecedent basis in the claim and it is unclear if it refers to the claimed “pre-synaptic neuron circuit” of claims 1/9 or if it can be any neuron which can be considered pre-synaptic.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1-2, 8-10, and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nishi (US20210056383A1) in view of Chen (US20190197391A1), further in view of Linares-Barranco (US20190138900A1).
Regarding claim 1, Nishi teaches a neural network apparatus comprising:
a plurality of neuron circuits (Figure 5: 110, ¶53, “Although FIG. 5 illustrates a single unit including one neuron circuit 110 of the spiking neural network device, the actual spiking neural network device includes a huge number of neuron circuits 110, and thus, as many numbers of units illustrated in FIG. 5 are combined to implement the device. The input units 130 in FIG. 5 correspond to presynaptic neuron circuits of the neuron circuit 110.”);
a plurality of synaptic circuits (Figure 5: 1201-n); and
a control circuit (Figure 5: 1401-n), wherein
each neuron circuit in the plurality of neuron circuits includes
a potential holding circuit configured to receive an output signal output from a synaptic circuit in the plurality of synaptic circuits and hold internal potential changing in accordance with a level or a time width of the received output signal (¶55, “The neuron circuit 110 integrates a spike voltage input from an input unit 130 via a synaptic element 120. If the integrated value exceeds a threshold, the neuron circuit 110 fires and releases a spike voltage to a downstream neuron circuit. When the neuron circuit 110 fires and releases a spike voltage,”),
a firing circuit configured to output a firing signal in a case where absolute value of the internal potential is larger than a firing threshold (¶55, “The neuron circuit 110 integrates a spike voltage input from an input unit 130 via a synaptic element 120. If the integrated value exceeds a threshold, the neuron circuit 110 fires and releases a spike voltage to a downstream neuron circuit. When the neuron circuit 110 fires and releases a spike voltage,”), and
each synaptic circuit in the plurality of synaptic circuits includes
a weight storage circuit configured to store a synaptic weight (Figure 5: Wjn),
a transmission circuit configured to receive the firing signal output from a pre-synaptic neuron circuit being a neuron circuit in the plurality of neuron circuits and output the output signal to a post-synaptic neuron circuit being a neuron circuit in the plurality of neuron circuits, the output signal being obtained by adding influence of the synaptic weight to the received firing signal (Figure 5: pre-synaptic neuron 130n, see ¶53, post-synaptic neuron 110, ¶54, “ In the spiking neural network device according to the first embodiment, a spike voltage is released from an input unit 130 and input to the neuron circuit 110 via a synaptic element 120 connected to this input unit 130. The magnitude of the spike voltage input to the neuron circuit 110 increases if the synaptic element 120 has a large weight and decreases if the synaptic element 120 has a small weight wji”), and
a learning circuit configured to change the synaptic weight in accordance with a contrast between a learning threshold and the absolute value of the internal potential held in the post-synaptic neuron circuit (¶56, “The synaptic potentiators 140 perform the potentiating operation to potentiate the weight w.sub.ji of the synaptic elements 120 based on the STDP rules. Whether the weight w.sub.ji of the synaptic elements 120 is actually potentiated when the synaptic potentiators 140 perform the potentiating operations may be probabilistic or deterministic. The probability or rate of potentiation of a weight w.sub.ji is determined by a function f(t.sub.i.sup.pre, t.sub.j.sup.post) where t.sub.i.sup.pre is time at which a spike voltage is input to a synaptic element 120 and t.sub.i.sup.post is time at which the neuron circuit 110 fires. In other words, when the synaptic potentiator 140 performs a potentiating operation, the weight w.sub.ji of the synaptic element 120 is potentiated at a probability or rate in accordance with the function f(t.sub.i.sup.pre, t.sub.j.sup.post) that depends on the firing timing of the neuron circuit 110 and the input timing of the spike voltage.” – This constitutes “accordance with a contrast between a learning threshold and the absolute value of the internal potential” because the “learning threshold” can take the value of the threshold potential and when the absolute value of a positive threshold potential is greater than the learning threshold potential defined as such, the synaptic weight is allowed to be potentiated), the synaptic weight being changed in a case where the firing signal is output from the pre-synaptic neuron circuit (see above citation, pre-synaptic firing causes transmission and potentiation), and
Nishi does not teach a firing threshold adjustment circuit configured to change the firing threshold in accordance with frequency of the firing signal.
Chen teaches a firing threshold adjustment circuit configured to change the firing threshold in accordance with frequency of the firing signal (¶29, “As shown in FIG. 2, the neuron 200 integrates incoming weighted spikes (spike inputs) in its membrane potential 202, and applies a threshold function 204. In an example, when the membrane potential is greater than a threshold, the neuron 200 produces an output spike and resets the membrane potential. The negative feedback loop 206 uses the spiking rate of the neuron 200 and when the spiking activity exceeds a reference firing rate, the feedback loop 206 shifts the threshold function to the right (e.g., increasing the threshold). This may result in a decrease in the firing rate of the neuron 200. In an example, the threshold function may be shifted to the left (e.g., decreasing the threshold) when the firing rate of the neuron 200 is lower than the reference firing rate. In an example, the reference firing rate is a desired firing rate used to determine whether to shift the transfer function.”) in order to regulate firing and implement homeostatic plasticity.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to implement a firing threshold adjustment circuit configured to change the firing threshold in accordance with frequency of the firing signal in order to provide homeostatic plasticity and firing regulation to make a more robust neuromorphic circuit.
Nishi does not teach wherein the control circuit is configured to change the learning threshold in accordance with frequency of the firing signal output from a target neuron circuit being at least one neuron circuit in the plurality of neuron circuits, the learning threshold being used for changing the synaptic weight stored in one or more synaptic circuits each outputting the output signal to the target neuron in the plurality of neuron circuits.
Linares-Barranco teaches wherein the control circuit is configured to
change the learning threshold in accordance with frequency of the firing signal output from a target neuron circuit being at least one neuron circuit in the plurality of neuron circuits (¶122-124– active signals are counted and once a learning threshold is reached, it may be increased)
the learning threshold being used for changing the synaptic weight stored in one or more synaptic circuits each outputting the output signal to the target neuron in the plurality of neuron circuits (¶125-126, when the threshold is reached, synaptic weight adjustment may be performed) which can improve performance and capacity (¶297).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the control circuit of Nishi to be configured to change the learning threshold in accordance with frequency of the firing signal output from a target neuron circuit being at least one neuron circuit in the plurality of neuron circuits, the learning threshold being used for changing the synaptic weight stored in one or more synaptic circuits each outputting the output signal to the target neuron in the plurality of neuron circuits in order to improve performance.
Regarding claim 2, Nishi as modified teaches all of the limitations of claim 1, but does not teach
the firing threshold adjustment circuit is configured to increase the firing threshold when the frequency of the firing signal is equal to or higher than a firing upper limit frequency given in advance, and reduce the firing threshold when the frequency of the firing signal is lower than a firing lower limit frequency given in advance, and the firing lower limit frequency is equal to or smaller than the firing upper limit frequency.
Chen teaches the firing threshold adjustment circuit is configured to increase the firing threshold when the frequency of the firing signal is equal to or higher than a firing upper limit frequency given in advance, and reduce the firing threshold when the frequency of the firing signal is lower than a firing lower limit frequency given in advance, and the firing lower limit frequency is equal to or smaller than the firing upper limit frequency (¶29).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the firing threshold adjustment circuit in Nishi as taught by Chen for the same reasons with respect to claim 1.
Regarding claim 8, Nishi as modified teaches all of the limitations of claim 1, wherein
at least one synaptic circuit in the plurality of synaptic circuits is configured to supply the output signal to the pre-synaptic neuron in the plurality of neuron circuits (Figure 5: 110 can be considered a pre-synaptic circuit to a subsequent synaptic element or neuron).
Regarding claim 9, the device of Nishi as modified according to claim 1 meets the structural requirements of the preamble of claim 9 and performs the method of claim 9 under normal operation.
Regarding claim 10, Nishi as modified according to claim 2 performs the method of claim 10 under normal operation.
Regarding claim 16, Nishi as modified according to claim 8 performs the method of claim 16 under normal operation.
Claim(s) 8 and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nishi (US20210056383A1) in view of Chen (US20190197391A1), further in view of Linares-Barranco (US20190138900A1) as applied to claim 1, further in view of Hopfield (US4660166).
Regarding claim 8, Nishi as modified teaches all of the limitations of claim 1, and under the interpretation that claim 8 recites “the pre-synaptic neuron” in the sense of the pre-synaptic neuron circuit before the instant synaptic circuit, Nishi as modified does not teach at least one synaptic circuit in the plurality of synaptic circuits is configured to supply the output signal to the pre-synaptic neuron in the plurality of neuron circuits.
Hopfield teaches at least one synaptic circuit in the plurality of synaptic circuits is configured to supply the output signal to the pre-synaptic neuron in the plurality of neuron circuits (Figure 2: outputs from A are fed to previous A’s) in order to provide good solutions from the network (“…so a collective neural network constructed in accordance with the invention can be used to find a good solution.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Nishi such that at least one synaptic circuit in the plurality of synaptic circuits is configured to supply the output signal to the pre-synaptic neuron in the plurality of neuron circuits in order to improve network learning and improve network solutions.
Regarding claim 16, Nishi as modified according to claim 8 performs the method of claim 16 under normal operation.
Allowable Subject Matter
Claims 3-7 and 11-15 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Regarding claims 3 and 11:
The claims recite particular functions of the learning circuit wherein the learning circuit is configured to increase or reduce the synaptic weight in accordance with the claimed learning threshold being equal to or above, or below, a first or second threshold, respectively. While the prior art establishes a varying firing threshold, there is no disclosure of a variable learning threshold for the synaptic weight, different from the firing threshold, which is varied according to the methodology of claims 3 and 11.
Claims 4-7 depend from claim 3.
Claims 12-15 depend from claim 11.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Van Der Made (US20200143229A1).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SCHYLER S SANKS whose telephone number is (571)272-6125. The examiner can normally be reached 06:30 - 15:30 Central Time, M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael Huntley can be reached at (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SCHYLER S SANKS/Primary Examiner, Art Unit 2129