DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 07/21/2025 has been entered.
Response to Amendment
The amendment filed 07/21/2025 has been entered. Claims 1-20 remain pending in the application.
Response to Arguments
Applicant’s arguments, filed 07/21/2025, with respect to the rejections of claims 1, 9 and 16 under 103 have been fully considered and are persuasive because of the amendments. Therefore, the rejections have been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Gangopadhyay et al. (NPL: A Coupled Network of Growth Transform Neurons for Spike-Encoded Auditory Feature Extraction) in view of Hiratani et al. (NPL: Interplay between Short- and Long-Term Plasticity in Cell-Assembly Formation) and further in view of Moraitis et al. (US Pub. 2020/0184325).
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 1, 9 and 16 are rejected under 35 U.S.C. 112(b) for reciting the limitation (among others) “minimize network energy consumed by the plurality of neurons to determine the extrinsic energy constraint including balancing, via a min-max optimization technique including a neuronal and synaptic time constant, the short-term dynamics and the long-term dynamics”. It is unclear what does balancing short-term dynamics with long-term dynamics mean.
The Applicant argues that paragraph 0116 of the specification of the current Applicant describes balancing, but paragraph 0116 recites “To achieve this task, a min-max optimization technique is investigated that will balance the short-term dynamics determined by the activity of neurons with the long-term dynamics determined by the activity of the synapses. The formulation is summarized as
PNG
media_image1.png
608
664
media_image1.png
Greyscale
”.
The above reciting of paragraph 0116 does not clearly describe what is short-term dynamics, what is long-term dynamics, and how the short-term dynamics and long-term dynamics are balanced using a selected time constant.
For examination purpose, the limitation of “balancing the short-term dynamics and long-term dynamics” is interpreted as “implementing a function associated with short term dynamic (short-term plasticity/short term depression) using a long-term synaptic plasticity rule”.
Claims 2-8 are rejected for being dependent on a rejected base claim, namely claim 1.
Claims 10-15 are rejected for being dependent on a rejected base claim, namely claim 9.
Claims 17-20 are rejected for being dependent on a rejected base claim, namely claim 16.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3 and 6 are rejected under 35 U.S.C. 103 as being unpatentable over Gangopadhyay et al. (NPL: A Coupled Network of Growth Transform Neurons for Spike-Encoded Auditory Feature Extraction) in view of Hiratani et al. (NPL: Interplay between Short- and Long-Term Plasticity in Cell-Assembly Formation) and further in view of Moraitis et al. (US Pub. 2020/0184325).
As per claim 1, Gangopadhyay teaches a spiking neural network comprising:
a plurality of neurons implemented in respective circuits [page 2, Fig. 1B discloses a coupled network of Growth Transform (GT) neurons], each neuron configured to:
produce a continuous-valued membrane potential according to a Growth Transform bounded by an extrinsic energy constraint [page 3, 1st paragraph, “as illustrated in Figure 1B, we explore a coupled network of GT neurons … We show that as the input stimulus varies over time, the spiking pattern of the growth transform neuron adapts, effectively tracking and transforming the input”; Fig. 2, page 4, 1st paragraph, “the derivation of the GT neuron model based on the energy minimization framework … neuronal spike generation and spike propagation using a multi-compartmental model as shown in Figure 2. Here each compartment could model a neuron which reacts to external stimuli and propagate signals to other compartments (neurons) by producing a rapid change in their transmembrane electrical potential difference (known as an action potential or a spike)”; page 2, last paragraph, “the spike generation process as the direct derivative of an energy functional of continuous-valued neural variables (e.g. membrane potential), that can moreover be used to mimic known neural dynamics”; Fig. 3 shows the membrane potential Vi defined as a function of spiking current received from another neurons and the stimulus bi; Fig. 2, page 4, 3rd and 4th paragraphs disclose in the growth transform (GT) neuron model, the membrane potential to be always bounded as follows
PNG
media_image2.png
31
141
media_image2.png
Greyscale
the neuron membrane potential being bounded by the ionic potentials on either side of the cell membrane that mediate the changes in membrane potential. We can decompose the response Vi(t) of the i-th neuron and the bias term bi(t) into two differential components
as follows
Vi(t) = Vi+(t) – Vi-(t), and
Bi(t) = bi+(t) – bi-(t), i = 1, … N
where Vi+(t) and Vi-(t) satisfy the following constraints:
Vi+(t) + Vi-(t) = 1
Vi+(t), Vi-(t) ≥ 0.],
the continuous- valued membrane potential defined as a function of spiking current received from another neuron in the plurality of neurons, and a received electrical current stimulus [Fig. 2 shows neuron/compartment i receives stimulus bi and Qij as the inputs, where page 4, 3rd paragraph, “Qij ϵ R denotes the contribution of the pre-synaptic potential at neuron j to the net excitation received by the post-synaptic neuron i, bi(t) ϵ R is the instantaneous external current stimulus”, and Fig. 2, page 4, 1st paragraph, “neuronal spike generation and spike propagation using a multi-compartmental model as shown in Figure 2. Here each compartment could model a neuron which reacts to external stimuli and propagate signals to other compartments (neurons) by producing a rapid change in their transmembrane electrical potential difference (known as an action potential or a spike)”; page 3, 1st paragraph, “as the input stimulus varies over time, the spiking pattern of the growth transform neuron adapts, effectively tracking and transforming the input”; Fig. 3 shows the membrane potential Vi defined as a function of spiking current received from another neurons and the stimulus bi]; and
a network energy function representing network energy consumed by all of the plurality of neurons [page 2, 2nd paragraph, “In this paper, we approach the process of spike generation and encoding in a neuron model … by designing an energy functional for a network of neurons which produces an emergent spiking dynamics when minimized under realistic physical constraints”; page 3, 1st paragraph, “In this paper, we first introduce a generic energy (cost) function for spiking neurons by applying a current balance equation at each neural compartment in the network”; page 3, section II, part A, “Energy function for Growth Transform spiking neuron model”; page 6, last paragraph, the energy function D (Vi+, Vi-)]; and
a neuromorphic framework integrating learning and inference [abstract, “the proposed GT neuron model provides a flexible neuromorphic framework to systematically design large-scale spiking neural networks with stable and interpretable dynamics”; page 2, 2nd paragraph, “In this paper, we approach the process of spike generation and encoding in a neuron model from a perspective that is fundamentally different compared to traditional dynamical systems formulations - by designing an energy functional for a network of neurons which produces an emergent spiking dynamics when minimized under realistic physical constraints. This is motivated by the fact that physical processes occurring in nature have an universal tendency to move towards a minimum-energy state - defined by the values taken by the process variables - that is a function of the inputs and the state of the process. Energy-based frameworks are the backbone of most machine learning models, which perform inference by minimizing an energy functional that capture dependencies between the variables”], the neuromorphic network configured to:
minimize network energy consumed by the plurality of neurons to determine the extrinsic energy constraint [abstract, “different forms of coupling between the neurons could produce compact and energy-efficient representation”; page 2, 2nd paragraph, “designing an energy functional for a network of neurons which produces an emergent spiking dynamic when minimized under realistic physical constraints”; page 3, section II. METHODS, 1st paragraph, “we introduce an energy functional for the GT neural network and then derive the basic model of a GT neuron as a result of its minimization”; page 4, 1st paragraph, “the derivation of the GT neuron model based on the energy minimization framework”; page 3, 1st paragraph, “we first introduce a generic energy (cost) function for spiking neurons”; page 6, 2nd paragraph, “we proposed a neuron model that sequentially optimizes the cost function D in Eq. 15 subject to constraints H using a multiplicative update called growth transforms, a fixed-point algorithm for optimizing continuous objective functions over a manifold like H with linear and non-negativity constraints”; page 5, energy minimization problem has the following form
PNG
media_image3.png
169
791
media_image3.png
Greyscale
It can be seen that the energy function is minimized with the constraints determined to satisfy the domain of H] via a min-max optimization technique including a neuronal and synaptic time constant [page 4, 1st paragraph, “the energy minimization framework”; page 5, 2nd and 3rd paragraphs, “The equations in 12 can be viewed as a stochastic variant of the first-order conditions of an energy minimization problem subject to the constraints given by Eqs 7 and 8 … The equivalent energy minimization problem has the following form … we explore how minimizing an energy function of this form can produce single neuron and population dynamics similar to that observed in biological neuronal networks … The cost function D in Eq. 15 comprises two components: (a) a quadratic term that tries to minimize the error between the variables Vi and yi; and (b) a regularization term”; page 6, 2nd paragraph, “to continually minimize the error with which the neural responses track the
time-varying input signal, so that Eq 2 is satisfied on an average”];
model the plurality of synaptic connections among the plurality of neurons as respective transconductances that regulate magnitude of spiking currents received from each of the plurality of neurons by each other of the plurality of neurons [page 3, Fig. 2 discloses the j-th neuron/compartment is coupled to the i-th neuron/compartment by a coupling strength Qij; page 4, 1st paragraph, “We will abstract the physics of neuronal spike generation and spike propagation using a multi-compartmental model as shown in Figure 2. Here each compartment could model a neuron which reacts to external stimuli and propagate signals to other compartments (neurons) by producing a rapid change in their transmembrane electrical potential difference (known as an action potential or a spike) … in this simplified model we assume that the coupling between the compartments i and j is given by a transimpedance variable Qij”; Fig. 3, page 4, 3rd paragraph, “Qij ϵ R denotes the contribution of the pre-synaptic potential at neuron j to the net excitation received by the post-synaptic neuron i];
paragraph 0078 of the specification of the current Application recites “Each post-synaptic neuron i, receives electrical input from at least one pre-synaptic neuron j, through a synapse modeled by a transconductance Qij”.
encode the received electrical current stimulus in corresponding continuous-valued membrane potentials of the plurality of neurons using mean spiking rates, inter-spike intervals, and changes in spiking rates [page 8, 1st paragraph, “We then demonstrate how the GT neuron encodes a time-varying stimulus in its spike pattern using different encoding techniques”; page 15, 2nd paragraph, “the GT neuron can encode external stimuli using firing rates and inter-spike intervals, similar to biological neurons. Moreover, the sub-threshold activity of GT neurons was also seen to encode stimulus information in addition to spike-based statistics”; page 3, 2nd paragraph, “When the stimulus is not strong enough to elicit spiking from a neuron, its subthreshold activity still encodes information about the stimulus that cannot be obtained from its spiking activity only”; Fig. 8 shows the spike statistics including firing rate and spike-times, wherein, “produces spike trains from which different spike statistics are extracted to encode discriminatory features”; It can be seen that different spike statistics are extracted to encode could mean different firing rates (changes in spiking rates) are extracted to encode; And, page 10, section C. Spike-encoding using Growth Transform Neurons, “For a time-varying input stimulus, GT neurons show a number of encoding properties. Rate encoding: Mean firing rate over time is a popular rate coding scheme … Rate-based features were obtained from this spike-train by computing the average spiking frequency of the neuron over a moving window. Inter-spike interval-based encoding ... Mean membrane potential …”].
Gangopadhyay does not explicitly teach
a neuromorphic framework … using (i) short-term dynamics of activity of the plurality of neurons, and (ii) long-term dynamics of activity of a plurality of synaptic connections among the plurality of neurons;
balancing the short-term dynamics and the long-term dynamics;
Hiratani teaches
a neuromorphic framework … using (i) short-term dynamics of activity of the plurality of neurons, and (ii) long-term dynamics of activity of a plurality of synaptic connections among the plurality of neurons [Introduction, “Learning and memory are fundamental brain functions supported by hippocampal neural circuits, and long-term potentiation (LTP) and depression (LTD) of synapses are considered to underlie activity-dependent modifications of hippocampal circuits during memory processes … Along with long-term plasticity, cortical synapses also undergo short-term plasticity [8,9]. Short-term plasticity, especially short-term depression (STD), can induce dramatic changes in the characteristic dynamics of recurrent”; page 2, Col. 1, Results section, 1st paragraph, “We construct a recurrent circuit model consisting of 2500 excitatory neurons and 500 inhibitory neurons that are randomly connected with each other. We introduce short-term plasticity and long-term plasticity into synaptic connections between excitatory neurons”; page 11, Col. 2, Model configuration section, 1st paragraph, “We construct a recurrent circuit model based on the chaotic balance network model [51,52] and extend it to include both short-term and long-term plasticity”; Where, long-term potentiation (LTP) and depression (LTD) of synapses are the two processes associated with the long-term dynamics of synaptic activity, and short-term dynamics of activity of the plurality of neurons are driven by short-term plasticity).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the growth transform (GT) neuron model of Gangopadhyay to include a neuromorphic framework using (i) short-term dynamics of activity of the plurality of neurons, and (ii) long-term dynamics of activity of a plurality of synaptic connections among the plurality of neurons of Hiratani. Doing so would help enriching synaptic weight dynamics in neural networks and causing effects on the cell assembly retention and modulation (page 9, Discussion section, 1st paragraph).
Gangopadhyay and Hiratani do not explicitly teach
balancing the short-term dynamics and the long-term dynamics;
Moraitis teaches
Balancing the short-term dynamics and the long-term dynamics [abstract, “a method of generating spikes by a neuron of a spiking neural network”; paragraphs 0038-0039, “the long-term synaptic plasticity component is configured to determine changes to the respective synaptic weight W(t) depending on the time distance in pairs of presynaptic and postsynaptic spikes of the given pair, wherein the short-term synaptic plasticity component is configured to determine changes to the respective short-term function F(t) depending on the time of arrival of presynaptic spikes … the short-term function F(t) may be a short-term depression or fatigue effect to the weight W(t) such that the values of G(t) are smaller than W(t)”; paragraph 0081, “In case G2(t)=W2(t) and G1(t) is W1(t) with short-term depression, the apparatus 308B may use the STDP learning rule to tune the efficacy G2(t). W1(t) and W2(t) are synaptic weights W(t) which are indexed to indicate that W1(t) and W2(t) are tuned by the long-term components 311 and 315 of the apparatus 308A and 308B respectively. The STDP learning rule is a long-term synaptic plasticity rule”];
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the growth transform (GT) neuron model of Gangopadhyay to include a process of using a long-term synaptic plastivity rule to tune the efficacy associated with the short-term depression of Moraitis. Doing so would help performing a learning to recognize patterns of long timescales by changing the synaptic weight based on the time distance in pairs of presynaptic and postsynaptic spikes (Moraitis, 0079-0081).
As per claim 2, Gangopadhyay, Hiratani and Moraitis teach the spiking neural network of claim 1.
Gangopadhyay further teaches
the Growth Transform ensures that the membrane potentials are always bounded [Fig. 2, page 4, 3rd and 4th paragraphs disclose in the growth transform (GT) neuron model, the membrane potential to be always bounded as follows
PNG
media_image2.png
31
141
media_image2.png
Greyscale
the neuron membrane potential being bounded by the ionic potentials on either side of the cell membrane that mediate the changes in membrane potential.].
As per claim 3, Gangopadhyay, Hiratani and Moraitis teach the spiking neural network of claim 1.
Gangopadhyay further teaches
the extrinsic energy constraint includes power dissipation due to coupling between neurons, power injected to or extracted from the plurality of neurons as a result of external stimulation, and power dissipated due to neural responses [page 5, energy minimization problem has the following form
PNG
media_image3.png
169
791
media_image3.png
Greyscale
The above function comprising the extrinsic energy constraint including power dissipation due to coupling between neurons and power injected to or extracted from the plurality of neurons as a result of external stimulation, and power dissipated due to neural responses, according to paragraph 0078 of the specification of the current Application which recites “the network energy function H() and the minimization objective are expressed in EQ. 2. The network energy function represents the extrinsic, or metabolic, power supplied to the network, including power dissipation due to coupling between neurons, power injected to or extracted from the system as a result of external stimulation, and power dissipated due to neural responses.
PNG
media_image4.png
294
524
media_image4.png
Greyscale
”].
As per claim 6, Gangopadhyay, Hiratani and Moraitis teach the spiking neural network of claim 1.
Gangopadhyay further teaches
at least one of a barrier and penalty function to enable finer control over spiking responses of the plurality of neurons [page 8, 2nd paragraph, “The ion-channel function ψ(.) used in this paper is shown in Figure 4. The gradient discontinuity in this case was set to V+ = V- = 0:5. In resting state, Vi+ < (0.5 - ϵ) and Vi- > (0.5 + ϵ), where ϵ is the width of the very narrow transition zone that sets the minimum input current needed for a spike as well as its height. When a depolarizing current pulse comes, both Vi+ and Vi- try to move towards 0.5. If the input current is strong enough, the variables cross over the threshold, producing a spike. This adds a large penalty to the gradient term for Vi+ and the updates force the variables to come back to a hyperpolarized state (a more negative Vi) than where it started from. The resulting membrane potential trace Vi is also given in Figure 4”].
Claims 4-5 are rejected under 35 U.S.C. 103 as being unpatentable over Gangopadhyay et al. in view of Hiratani et al. in view of Moraitis et al. and further in view of Gangopadhyay et al. (NPL: Spiking, Bursting, and Population Dynamics in a Network of Growth Transform Neurons, hereinafter Gangopadhyay-2).
As per claim 4, Gangopadhyay, Hiratani and Moraitis teach the spiking neural network of claim 1.
Gangopadhyay, Hiratani and Moraitis do not explicitly teach
each neuron of the plurality of neurons includes a modulation function that modulates response trajectories of the plurality of neurons without affecting the minimum network energy and a steady-state solution.
Gangopadhya-2 teaches
each neuron of the plurality of neurons includes a modulation function that modulates response trajectories of the plurality of neurons without affecting the minimum network energy and a steady-state solution [Fig. 2, abstract, “we present a geometric approach for visualizing the dynamics of the network where each of the neurons traverses a trajectory in a dual optimization space, whereas the network itself traverses a trajectory in an equivalent primal optimization space”; page 2, Col. 1, 2nd paragraph, “In this paper, we demonstrate the use of this approach to design SVMs that exhibit ⅀∆ modulation type limit cycles, spiking behavior, and bursting responses”; Fig. 12, page 10, Col. 1, 1st paragraph, “Illustration showing that by changing the parameters of the potential function, the spiking rate and the magnitudes of the spikes can be adapted while keeping the classification performance (or boundaries) similar … The spiking potential function can be modulated to change the spiking dynamics across the population. For a particular support vector neuron, the spiking rate decreases with an increase in the slope of the transition region at the classification boundary, i.e., with W (refer to Table I). This is illustrated in Fig. 12 with three different values of W and their corresponding steady-state spiking behaviors and classification plots. It is seen that all three learn the same classification boundary but with different spiking dynamics”; page 4, Col. 2, 1st paragraph, “The dynamic properties of the limit cycle will be determined by the shape of the potential function Φ(.) which we later show produces ⅀∆ modulation, spiking, and bursting as the network converges to a steady-state solution””].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the growth transform (GT) neuron model of Gangopadhyay to include each neuron of the plurality of neurons includes a modulation function that modulates response trajectories of the plurality of neurons of Gangopadhya-2. Doing so would help changing the spiking dynamics across the population (Gangopadhya-2, page 10, Col. 1, 1st paragraph).
As per claim 5, Gangopadhyay, Hiratani, Moraitis and Gangopadhya-2 teach the spiking neural network of claim 4.
Gangopadhya-2 teaches
the modulation function is varied to tune transient firing statistics to model cell excitability of the plurality of neurons [Fig. 12, page 10, Col. 1, 1st paragraph, “Illustration showing that by changing the parameters of the potential function, the spiking rate and the magnitudes of the spikes can be adapted while keeping the classification performance (or boundaries) similar … The spiking potential function can be modulated to change the spiking dynamics across the population. This is illustrated in Fig. 12 with three different values of W and their corresponding steady-state spiking behaviors and classification plots. It is seen that all three learn the same classification boundary but with different spiking dynamics”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the growth transform (GT) neuron model of Gangopadhyay to include the modulation function is varied to tune transient firing statistics of the plurality of neurons of Gangopadhya-2. Doing so would help changing the spiking dynamics across the population (Gangopadhya-2, page 10, Col. 1, 1st paragraph).
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Gangopadhyay et al. in view of Hiratani et al. in view of Moraitis et al. and further in view of Daily et al. (US Patent 9,020,870).
As per claim 7, Gangopadhyay, Hiratani and Moraitis teach the spiking neural network of claim 1.
Gangopadhyay teaches in abstract and page 3, 1st paragraph, “the proposed GT neuron model provides a flexible neuromorphic framework to systematically design large-scale spiking neural networks with stable and interpretable dynamics … as illustrated in Figure 1B, we explore a coupled network of GT neurons … We show that as the input stimulus varies over time, the spiking pattern of the growth transform neuron adapts, effectively tracking and transforming the input”.
Gangopadhyay, Hiratani and Moraitis do not teach
the spiking neural network is an associative memory network that uses the Growth Transform to store and recall memory patterns for the plurality of neurons.
Daily teaches
the spiking neural network is an associative memory network that uses the Growth Transform to store and recall memory patterns for the plurality of neurons [abstract, “system that uses spiking neuron networks to identify an unknown external stimulus”; Col. 5, lines 15-24, “a method and system capable of learning input patterns, memorizing, recalling and recognizing previously learned patterns. The system uses a spiking neural network coupled with a readout network, along with algorithmic procedures for conditioning and training the networks”; Since Gangopadhyay in abstract teaches the proposed growth transform (GT) neuron model provides a flexible neuromorphic framework to systematically design large-scale spiking neural networks with stable and interpretable dynamics, while Daily teaches the spiking neural network associated with a memory network for memorizing, recalling and recognizing previously learned patterns, therefore, the combination of Gangopadhyay and Daily teaches the above claim limitation].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the growth transform (GT) neuron model of Gangopadhyay to include the spiking neural network is an associative memory network to store and recall memory patterns for the plurality of neurons of Daily. Doing so would help designing a high-capacity memory device and a recall circuitry using a SNN model that can be implemented in hardware (such as VLSI chips) directly, as opposed to algorithms running on general-purpose computers (Daily, Col. 5, lines 61-64).
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Gangopadhyay et al. in view of Hiratani et al. in view of Moraitis et al. and further in view of Coenen et al. (US Pub. 2014/0081895).
As per claim 8, Gangopadhyay, Hiratani and Moraitis teach the spiking neural network of claim 1.
Gangopadhyay, Hiratani and Moraitis do not teach
the spiking neural network is implemented on fully continuous-time analog architecture.
Coenen teaches
the spiking neural network is implemented on fully continuous-time analog architecture [abstract, “The controller may comprise spiking neuron network operable according to reinforcement learning process”; paragraph 0078, “methods for facilitating adaptive controller implementation using spiking neuron networks”; paragraphs 0268-0277, “Referring now to FIG. 10A one exemplary implementation of the reinforcement learning method … At step 1004, spiking neuron network of the controller may be adapted using reinforcement learning process … FIG. 10B illustrates some implementations of a controller operation method comprising reinforcement learning … At step 1026, spiking neuron network of the controller (e.g., the controller 620 of FIG. 6) may be trained using reinforcement learning … methods of FIGS. 10A-10C may be implemented in one or more processing devices (e.g., an analog circuit designed to process information)”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the growth transform (GT) neuron model of Gangopadhyay to include the spiking neural network is implemented on fully continuous-time analog architecture of Coenen. Doing so would help processing information using the analog network (Coenen, 0268).
Claims 9-13 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Gangopadhyay et al. in view of Hiratani et al. in view of Moraitis et al. in view of Gangopadhyay et al. (NPL: Spiking, Bursting, and Population Dynamics in a Network of Growth Transform Neurons, hereinafter Gangopadhyay-2) and further in view of Laukien et al. (US Pub. 2019/0294980).
As per claim 9, Gangopadhyay teaches a method of operating a neural network, the method comprising:
implementing a plurality of neurons in respective circuits [page 2, Fig. 1B discloses a coupled network of Growth Transform (GT) neurons];
producing a continuous-valued membrane potential according to a Growth Transform bounded by an extrinsic energy constraint [page 3, 1st paragraph, “as illustrated in Figure 1B, we explore a coupled network of GT neurons … We show that as the input stimulus varies over time, the spiking pattern of the growth transform neuron adapts, effectively tracking and transforming the input”; Fig. 2, page 4, 1st paragraph, “the derivation of the GT neuron model based on the energy minimization framework … neuronal spike generation and spike propagation using a multi-compartmental model as shown in Figure 2. Here each compartment could model a neuron which reacts to external stimuli and propagate signals to other compartments (neurons) by producing a rapid change in their transmembrane electrical potential difference (known as an action potential or a spike)”; page 2, last paragraph, “the spike generation process as the direct derivative of an energy functional of continuous-valued neural variables (e.g. membrane potential), that can moreover be used to mimic known neural dynamics”; Fig. 3 shows the membrane potential Vi defined as a function of spiking current received from another neurons and the stimulus bi; Fig. 2, page 4, 3rd and 4th paragraphs disclose in the growth transform (GT) neuron model, the membrane potential to be always bounded as follows
PNG
media_image2.png
31
141
media_image2.png
Greyscale
the neuron membrane potential being bounded by the ionic potentials on either side of the cell membrane that mediate the changes in membrane potential. We can decompose the response Vi(t) of the i-th neuron and the bias term bi(t) into two differential components
as follows
Vi(t) = Vi+(t) – Vi-(t), and
Bi(t) = bi+(t) – bi-(t), i = 1, … N
where Vi+(t) and Vi-(t) satisfy the following constraints:
Vi+(t) + Vi-(t) = 1
Vi+(t), Vi-(t) ≥ 0.];
defining a function of spiking current received from another neuron in the plurality of neurons, and a received electrical current stimulus, as a continuous-valued membrane potential [Fig. 2 shows neuron/compartment i receives stimulus bi and Qij as the inputs, where page 4, 3rd paragraph, “Qij ϵ R denotes the contribution of the pre-synaptic potential at neuron j to the net excitation received by the post-synaptic neuron i, bi(t) ϵ R is the instantaneous external current stimulus”, and Fig. 2, page 4, 1st paragraph, “neuronal spike generation and spike propagation using a multi-compartmental model as shown in Figure 2. Here each compartment could model a neuron which reacts to external stimuli and propagate signals to other compartments (neurons) by producing a rapid change in their transmembrane electrical potential difference (known as an action potential or a spike)”; page 3, 1st paragraph, “as the input stimulus varies over time, the spiking pattern of the growth transform neuron adapts, effectively tracking and transforming the input”; Fig. 3 shows the membrane potential Vi defined as a function of spiking current received from another neurons and the stimulus bi];
representing network energy consumed by all of the plurality of neurons as a network energy function [page 2, 2nd paragraph, “In this paper, we approach the process of spike generation and encoding in a neuron model … by designing an energy functional for a network of neurons which produces an emergent spiking dynamics when minimized under realistic physical constraints”; page 3, 1st paragraph, “In this paper, we first introduce a generic energy (cost) function for spiking neurons by applying a current balance equation at each neural compartment in the network”; page 3, section II, part A, “Energy function for Growth Transform spiking neuron model”; page 6, last paragraph, the energy function D (Vi+, Vi-)];
implementing a neuromorphic framework integrating learning and inference [abstract, “the proposed GT neuron model provides a flexible neuromorphic framework to systematically design large-scale spiking neural networks with stable and interpretable dynamics”; page 2, 2nd paragraph, “In this paper, we approach the process of spike generation and encoding in a neuron model from a perspective that is fundamentally different compared to traditional dynamical systems formulations - by designing an energy functional for a network of neurons which produces an emergent spiking dynamics when minimized under realistic physical constraints. This is motivated by the fact that physical processes occurring in nature have an universal tendency to move towards a minimum-energy state - defined by the values taken by the process variables - that is a function of the inputs and the state of the process. Energy-based frameworks are the backbone of most machine learning models, which perform inference by minimizing an energy functional that capture dependencies between the variables”]
minimize network energy consumed by the plurality of neurons to determine the extrinsic energy constraint [abstract, “different forms of coupling between the neurons could produce compact and energy-efficient representation”; page 2, 2nd paragraph, “designing an energy functional for a network of neurons which produces an emergent spiking dynamic when minimized under realistic physical constraints”; page 3, section II. METHODS, 1st paragraph, “we introduce an energy functional for the GT neural network and then derive the basic model of a GT neuron as a result of its minimization”; page 4, 1st paragraph, “the derivation of the GT neuron model based on the energy minimization framework”; page 3, 1st paragraph, “we first introduce a generic energy (cost) function for spiking neurons”; page 6, 2nd paragraph, “we proposed a neuron model that sequentially optimizes the cost function D in Eq. 15 subject to constraints H using a multiplicative update called growth transforms, a fixed-point algorithm for optimizing continuous objective functions over a manifold like H with linear and non-negativity constraints”; page 5, energy minimization problem has the following form
PNG
media_image3.png
169
791
media_image3.png
Greyscale
It can be seen that the energy function is minimized with the constraints determined to satisfy the domain of H] via a min-max optimization technique including a neuronal and synaptic time constant [page 4, 1st paragraph, “the energy minimization framework”; page 5, 2nd and 3rd paragraphs, “The equations in 12 can be viewed as a stochastic variant of the first-order conditions of an energy minimization problem subject to the constraints given by Eqs 7 and 8 … The equivalent energy minimization problem has the following form … we explore how minimizing an energy function of this form can produce single neuron and population dynamics similar to that observed in biological neuronal networks … The cost function D in Eq. 15 comprises two components: (a) a quadratic term that tries to minimize the error between the variables Vi and yi; and (b) a regularization term”; page 6, 2nd paragraph, “to continually minimize the error with which the neural responses track the time-varying input signal, so that Eq 2 is satisfied on an average”];
modeling the plurality of synaptic connections among the plurality of neurons as respective transconductances that regulate magnitude of spiking currents received from each of the plurality of neurons by each other of the plurality of neurons [page 3, Fig. 2 discloses the j-th neuron/compartment is coupled to the i-th neuron/compartment by a coupling strength Qij; page 4, 1st paragraph, “We will abstract the physics of neuronal spike generation and spike propagation using a multi-compartmental model as shown in Figure 2. Here each compartment could model a neuron which reacts to external stimuli and propagate signals to other compartments (neurons) by producing a rapid change in their transmembrane electrical potential difference (known as an action potential or a spike) … in this simplified model we assume that the coupling between the compartments i and j is given by a transimpedance variable Qij”; Fig. 3, page 4, 3rd paragraph, “Qij ϵ R denotes the contribution of the pre-synaptic potential at neuron j to the net excitation received by the post-synaptic neuron i], using gradient discontinuities to modulate the spiking currents [page 4, 1st paragraph, “We will abstract the physics of neuronal spike generation and spike propagation using a multi-compartmental model as shown in Figure 2. Here each compartment could model a neuron which reacts to external stimuli and propagate signals to other compartments (neurons) by producing a rapid change in their transmembrane electrical potential difference (known as an action potential or a spike). Each compartment in the model comprises of ion channels”; page 8, 2nd paragraph, “The ion-channel function ѱ(.) used in this paper is shown in Figure 4. The gradient discontinuity in this case was set to V+ = V- = 0.5. In resting state, Vi+ < (0.5 - ϵ) and Vi- > (0.5 + ϵ), where ϵ is the width of the very narrow transition zone that sets the minimum input current needed for a spike as well as its height. When a depolarizing current pulse comes, both Vi+ and Vi- try to move towards 0.5. If the input current is strong enough, the variables cross over the threshold, producing a spike”];
encoding the received electrical current stimulus in corresponding continuous-valued membrane potentials of the plurality of neurons [page 8, 1st paragraph, “We then demonstrate how the GT neuron encodes a time-varying stimulus in its spike pattern using different encoding techniques”; page 15, 2nd paragraph, “the GT neuron can encode external stimuli using firing rates and inter-spike intervals, similar to biological neurons. Moreover, the sub-threshold activity of GT neurons was also seen to encode stimulus information in addition to spike-based statistics”; page 3, 2nd paragraph, “When the stimulus is not strong enough to elicit spiking from a neuron, its subthreshold activity still encodes information about the stimulus that cannot be obtained from its spiking activity only”].
Gangopadhyay does not explicitly teach
implementing a neuromorphic framework … using (i) short-term dynamics of activity of the plurality of neurons, and (ii) long-term dynamics of activity of a plurality of synaptic connections among the plurality of neurons;
balancing the short-term dynamics and the long-term dynamics;
the modeling includes using gradient discontinuities;
encoding the received electrical current stimulus in corresponding continuous-valued membrane potentials of the plurality of neurons using an average input of the membrane potentials and a sum of the membrane potentials (emphasis added).
Hiratani teaches
implementing a neuromorphic framework … using (i) short-term dynamics of activity of the plurality of neurons, and (ii) long-term dynamics of activity of a plurality of synaptic connections among the plurality of neurons [Introduction, “Learning and memory are fundamental brain functions supported by hippocampal neural circuits, and long-term potentiation (LTP) and depression (LTD) of synapses are considered to underlie activity-dependent modifications of hippocampal circuits during memory processes … Along with long-term plasticity, cortical synapses also undergo short-term plasticity [8,9]. Short-term plasticity, especially short-term depression (STD), can induce dramatic changes in the characteristic dynamics of recurrent”; page 2, Col. 1, Results section, 1st paragraph, “We construct a recurrent circuit model consisting of 2500 excitatory neurons and 500 inhibitory neurons that are randomly connected with each other. We introduce short-term plasticity and long-term plasticity into synaptic connections between excitatory neurons”; page 11, Col. 2, Model configuration section, 1st paragraph, “We construct a recurrent circuit model based on the chaotic balance network model [51,52] and extend it to include both short-term and long-term plasticity”; Where, long-term potentiation (LTP) and depression (LTD) of synapses are the two processes associated with the long-term dynamics of synaptic activity, and short-term dynamics of activity of the plurality of neurons are driven by short-term plasticity).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the growth transform (GT) neuron model of Gangopadhyay to include a process of implementing a neuromorphic framework using (i) short-term dynamics of activity of the plurality of neurons, and (ii) long-term dynamics of activity of a plurality of synaptic connections among the plurality of neurons of Hiratani. Doing so would help enriching synaptic weight dynamics in neural networks and causing effects on the cell assembly retention and modulation (page 9, Discussion section, 1st paragraph).
Gangopadhyay and Hiratani do not explicitly teach
balancing the short-term dynamics and the long-term dynamics;
the modeling includes using gradient discontinuities;
encoding the received electrical current stimulus in corresponding continuous-valued membrane potentials of the plurality of neurons using an average input of the membrane potentials and a sum of the membrane potentials (emphasis added).
Moraitis teaches
balancing the short-term dynamics and the long-term dynamics [abstract, “a method of generating spikes by a neuron of a spiking neural network”; paragraphs 0038-0039, “the long-term synaptic plasticity component is configured to determine changes to the respective synaptic weight W(t) depending on the time distance in pairs of presynaptic and postsynaptic spikes of the given pair, wherein the short-term synaptic plasticity component is configured to determine changes to the respective short-term function F(t) depending on the time of arrival of presynaptic spikes … the short-term function F(t) may be a short-term depression or fatigue effect to the weight W(t) such that the values of G(t) are smaller than W(t)”; paragraph 0081, “In case G2(t)=W2(t) and G1(t) is W1(t) with short-term depression, the apparatus 308B may use the STDP learning rule to tune the efficacy G2(t). W1(t) and W2(t) are synaptic weights W(t) which are indexed to indicate that W1(t) and W2(t) are tuned by the long-term components 311 and 315 of the apparatus 308A and 308B respectively. The STDP learning rule is a long-term synaptic plasticity rule”];
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the growth transform (GT) neuron model of Gangopadhyay to include a process of using a long-term synaptic plastivity rule to tune the efficacy associated with the short-term depression of Moraitis. Doing so would help performing a learning to recognize patterns of long timescales by changing the synaptic weight based on the time distance in pairs of presynaptic and postsynaptic spikes (Moraitis, 0079-0081).
Gangopadhyay, Hiratani and Moraitis do not explicitly teach
the modeling includes using gradient discontinuities;
encoding the received electrical current stimulus in corresponding continuous-valued membrane potentials of the plurality of neurons using an average input of the membrane potentials and a sum of the membrane potentials (emphasis added).
Gangopadhyay-2 teaches
the modeling includes using gradient discontinuities [page 2, 2nd paragraph, “Our approach will be to first estimate an equivalent dual optimization function based on the mapping given by (1). Each neuron will then implement a continuous mapping based on a polynomial growth transform update that will dynamically optimize the cost function. Because the growth transform mapping will be designed to evolve over a constrained manifold, the neuronal responses and the network will always be stable. The switching, spiking, and bursting dynamics of the neurons will then emerge by choosing different types of potential functions”; page 4, section III, Consider the potential function given by Φ(pik ) = |pik – (1/2)| and shown in Fig. 3(a). The gradient of the function Φ(.) now has a discontinuity at pik = 1/2, k = 1, 2, ∀i … The primal loss function exhibits a piecewise linear response where the slope of the loss function changes at classification margins (or errors) that are symmetric about the separating hyperplane; page 10. Col. 1, last paragraph, “The spiking potential function can be modulated to change the spiking dynamics across the population … the spiking rate decreases with an increase in the slope of the transition region at the classification boundary, i.e., with W (refer to Table I). This is illustrated in Fig. 12 with three different values of W and their corresponding steady-state spiking behaviors”];
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the growth transform (GT) neuron model of Gangopadhyay to include the modeling includes using gradient discontinuities of Gangopadhyay-2. Doing so would help generating spiking and bursting as the network converges to a steady-state solution (Gangopadhyay-2, page 4).
Gangopadhyay, Hiratani, Moraitis and Gangopadhyay-2 do not teach
encoding the received electrical current stimulus in corresponding continuous-valued membrane potentials of the plurality of neurons using an average input of the membrane potentials and a sum of the membrane potentials (emphasis added).
Laukien teaches
encoding the received electrical current stimulus in corresponding continuous-valued membrane potentials of the plurality of neurons using an average input of the membrane potentials and a sum of the membrane potentials [paragraph 0173, “activateEncoder () and its kernels pass the hierarchy's inputs up from layer to layer. The input is first combined with its historical values to generate a derived input, and then converted into a stimulus which is the size and shape of the output encoding
PNG
media_image5.png
242
498
media_image5.png
Greyscale
”;
Wherein, paragraph 0174, “deriveinputKernel produces a pair of inputs for the encoder, one a copy of the input and the other a moving average of recent inputs (moving average is a calculation that determines an average of a set of data (set of inputs in this case) over a period of time)”, and paragraph 0175, “encodeKemel produces the encoder's stimulus, which is a weighted sum of each unit's derived inputs”; paragraph 0163, “for example, the derived input D stores a copy of the immediate input Xij in dij1, along with a moving average of past inputs in dij2”; Since Gangopadhyay in Fig. 3, pages 3 and 8 teaches the input stimulus associated with the membrane potential Vi, and the process of encoding the electrical current stimulus in corresponding continuous-valued membrane potentials, while Laukien teaches encoding the stimulus is based in part on the average of the inputs and weighted sum of the inputs, therefore, the combination of Gangopadhyay and Laukien read on the above claim limitation].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the growth transform (GT) neuron model of Gangopadhyay to include the process of encoding the received stimulus using an average input and a sum of the inputs of Laukien. Doing so would help processing streams of time-varying data using at least an encoder which transforming the derived input into a stimulus (Laukien, abstract).
As per claim 10, Gangopadhyay, Hiratani, Moraitis, Gangopadhyay-2 and Laukien teach the method of claim 9.
Gangopadhyay further teaches
bounding the membrane potentials by the Growth Transform always [Fig. 2, page 4, 3rd and 4th paragraphs disclose in the growth transform (GT) neuron model, the membrane potential to be always bounded as follows
PNG
media_image2.png
31
141
media_image2.png
Greyscale
the neuron membrane potential being bounded by the ionic potentials on either side of the cell membrane that mediate the changes in membrane potential].
As per claim 11, Gangopadhyay, Hiratani, Moraitis, Gangopadhyay-2 and Laukien teach the method of claim 10.
Gangopadhyay further teaches
power dissipation due to coupling between neurons, injecting power to or extracted from the plurality of neurons as a result of external stimulation, and dissipating power due to neural responses in the extrinsic energy constraint [page 5, energy minimization problem has the following form
PNG
media_image3.png
169
791
media_image3.png
Greyscale
The above function comprising the extrinsic energy constraint including power dissipation due to coupling between neurons and power injected to or extracted from the plurality of neurons as a result of external stimulation, and power dissipated due to neural responses, according to paragraph 0078 of the specification of the current Application which recites “the network energy function H() and the minimization objective are expressed in EQ. 2. The network energy function represents the extrinsic, or metabolic, power supplied to the network, including power dissipation due to coupling between neurons, power injected to or extracted from the system as a result of external stimulation, and power dissipated due to neural responses.
PNG
media_image4.png
294
524
media_image4.png
Greyscale
”].
As per claim 12, Gangopadhyay, Hiratani, Moraitis, Gangopadhyay-2 and Laukien teach the method of claim 10.
Gangopadhya-2 further teaches
a modulation function for each neuron of the plurality of neurons that modulates response trajectories of the plurality of neurons without affecting the minimum network energy and a steady-state solution [Fig. 2, abstract, “we present a geometric approach for visualizing the dynamics of the network where each of the neurons traverses a trajectory in a dual optimization space, whereas the network itself traverses a trajectory in an equivalent primal optimization space”; page 2, Col. 1, 2nd paragraph, “In this paper, we demonstrate the use of this approach to design SVMs that exhibit ⅀∆ modulation type limit cycles, spiking behavior, and bursting responses”; Fig. 12, page 10, Col. 1, 1st paragraph, “Illustration showing that by changing the parameters of the potential function, the spiking rate and the magnitudes of the spikes can be adapted while keeping the classification performance (or boundaries) similar … The spiking potential function can be modulated to change the spiking dynamics across the population. For a particular support vector neuron, the spiking rate decreases with an increase in the slope of the transition region at the classification boundary, i.e., with W (refer to Table I). This is illustrated in Fig. 12 with three different values of W and their corresponding steady-state spiking behaviors and classification plots. It is seen that all three learn the same classification boundary but with different spiking dynamics”; page 4, Col. 2, 1st paragraph, “The dynamic properties of the limit cycle will be determined by the shape of the potential function Φ(.) which we later show produces ⅀∆ modulation, spiking, and bursting as the network converges to a steady-state solution””], wherein varying the modulation function tunes transient firing statistics to model cell excitability of the plurality of neurons [Fig. 12, page 10, Col. 1, 1st paragraph, “Illustration showing that by changing the parameters of the potential function, the spiking rate and the magnitudes of the spikes can be adapted while keeping the classification performance (or boundaries) similar … The spiking potential function can be modulated to change the spiking dynamics across the population. This is illustrated in Fig. 12 with three different values of W and their corresponding steady-state spiking behaviors and classification plots. It is seen that all three learn the same classification boundary but with different spiking dynamics”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the growth transform (GT) neuron model of Gangopadhyay to include each neuron of the plurality of neurons includes a modulation function that modulates response trajectories of the plurality of neurons, wherein varying the modulation function tunes transient firing statistics of Gangopadhya-2. Doing so would help changing the spiking dynamics across the population (Gangopadhya-2, page 10, Col. 1, 1st paragraph).
As per claim 13, Gangopadhyay, Hiratani, Moraitis, Gangopadhyay-2 and Laukien teach the method of claim 10.
Gangopadhyay further teaches
enabling finer control over spiking response of the plurality of neurons with a barrier or penalty function [page 8, 2nd paragraph, “The ion-channel function ψ(.) used in this paper is shown in Figure 4. The gradient discontinuity in this case was set to V+ = V- = 0:5. In resting state, Vi+ < (0.5 - ϵ) and Vi- > (0.5 + ϵ), where ϵ is the width of the very narrow transition zone that sets the minimum input current needed for a spike as well as its height. When a depolarizing current pulse comes, both Vi+ and Vi- try to move towards 0.5. If the input current is strong enough, the variables cross over the threshold, producing a spike. This adds a large penalty to the gradient term for Vi+ and the updates force the variables to come back to a hyperpolarized state (a more negative Vi) than where it started from. The resulting membrane potential trace Vi is also given in Figure 4”].
As per claim 19, Gangopadhyay, Hiratani, Moraitis and Laukien teach the method of claim 16.
Gangopadhyay, Hiratani, Moraitis and Laukien do not teach
a modulation function for each neuron of the plurality of neurons that modulates response trajectories of the plurality of neurons without affecting the minimum network energy and a steady-state solution, wherein varying the modulation function tunes transient firing statistics to model cell excitability of the plurality of neurons.
Gangopadhya-2 teaches
a modulation function for each neuron of the plurality of neurons that modulates response trajectories of the plurality of neurons without affecting the minimum network energy and a steady-state solution [Fig. 2, abstract, “we present a geometric approach for visualizing the dynamics of the network where each of the neurons traverses a trajectory in a dual optimization space, whereas the network itself traverses a trajectory in an equivalent primal optimization space”; page 2, Col. 1, 2nd paragraph, “In this paper, we demonstrate the use of this approach to design SVMs that exhibit ⅀∆ modulation type limit cycles, spiking behavior, and bursting responses”; Fig. 12, page 10, Col. 1, 1st paragraph, “Illustration showing that by changing the parameters of the potential function, the spiking rate and the magnitudes of the spikes can be adapted while keeping the classification performance (or boundaries) similar … The spiking potential function can be modulated to change the spiking dynamics across the population. For a particular support vector neuron, the spiking rate decreases with an increase in the slope of the transition region at the classification boundary, i.e., with W (refer to Table I). This is illustrated in Fig. 12 with three different values of W and their corresponding steady-state spiking behaviors and classification plots. It is seen that all three learn the same classification boundary but with different spiking dynamics”; page 4, Col. 2, 1st paragraph, “The dynamic properties of the limit cycle will be determined by the shape of the potential function Φ(.) which we later show produces ⅀∆ modulation, spiking, and bursting as the network converges to a steady-state solution””], wherein varying the modulation function tunes transient firing statistics to model cell excitability of the plurality of neurons [Fig. 12, page 10, Col. 1, 1st paragraph, “Illustration showing that by changing the parameters of the potential function, the spiking rate and the magnitudes of the spikes can be adapted while keeping the classification performance (or boundaries) similar … The spiking potential function can be modulated to change the spiking dynamics across the population. This is illustrated in Fig. 12 with three different values of W and their corresponding steady-state spiking behaviors and classification plots. It is seen that all three learn the same classification boundary but with different spiking dynamics”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the growth transform (GT) neuron model of Gangopadhyay to include each neuron of the plurality of neurons includes a modulation function that modulates response trajectories of the plurality of neurons, wherein varying the modulation function tunes transient firing statistics of Gangopadhya-2. Doing so would help changing the spiking dynamics across the population (Gangopadhya-2, page 10, Col. 1, 1st paragraph).
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Gangopadhyay et al. in view of Hiratani et al. in view of Moraitis et al. in view of Gangopadhyay et al. in view of Laukien et al. and further in view of Daily et al. (US Patent 9,020,870).
As per claim 14, Gangopadhyay, Hiratani, Moraitis, Gangopadhyay-2 and Laukien teach the method of claim 10.
Gangopadhyay teaches in abstract and page 3, 1st paragraph, “the proposed GT neuron model provides a flexible neuromorphic framework to systematically design large-scale spiking neural networks with stable and interpretable dynamics … as illustrated in Figure 1B, we explore a coupled network of GT neurons … We show that as the input stimulus varies over time, the spiking pattern of the growth transform neuron adapts, effectively tracking and transforming the input”.
Gangopadhyay, Hiratani, Moraitis, Gangopadhyay-2 and Laukien do not teach
using the Growth Transform to store and recall memory patterns for the plurality of neurons to enable the neural network to be an associative memory network.
Daily teaches
using the Growth Transform to store and recall memory patterns for the plurality of neurons to enable the neural network to be an associative memory network [abstract, “system that uses spiking neuron networks to identify an unknown external stimulus”; Col. 5, lines 15-24, “a method and system capable of learning input patterns, memorizing, recalling and recognizing previously learned patterns. The system uses a spiking neural network coupled with a readout network, along with algorithmic procedures for conditioning and training the networks”; Since Gangopadhyay in abstract teaches the proposed growth transform (GT) neuron model provides a flexible neuromorphic framework to systematically design large-scale spiking neural networks with stable and interpretable dynamics, while Daily teaches the spiking neural network associated with a memory network for memorizing, recalling and recognizing previously learned patterns, therefore, the combination of Gangopadhyay and Daily teaches the above claim limitation].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the growth transform (GT) neuron model of Gangopadhyay to include the process of using the Growth Transform to store and recall memory patterns for the plurality of neurons of Daily. Doing so would help designing a high-capacity memory device and a recall circuitry using a SNN model that can be implemented in hardware (such as VLSI chips) directly, as opposed to algorithms running on general-purpose computers (Daily, Col. 5, lines 61-64).
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Gangopadhyay et al. in view of Hiratani et al. in view of Moraitis et al. in view of Gangopadhyay et al. in view of Laukien et al. and further in view of Coenen et al. (US Pub. 2014/0081895).
As per claim 15, Gangopadhyay, Hiratani, Moraitis, Gangopadhyay-2 and Laukien teach the method of claim 10.
Gangopadhyay, Hiratani, Moraitis, Gangopadhyay-2 and Laukien do not teach
implementing the neural network on fully continuous-time analog architecture.
Coenen teaches
implementing the neural network on fully continuous-time analog architecture [abstract, “The controller may comprise spiking neuron network operable according to reinforcement learning process”; paragraph 0078, “methods for facilitating adaptive controller implementation using spiking neuron networks”; paragraphs 0268-0277, “Referring now to FIG. 10A one exemplary implementation of the reinforcement learning method … At step 1004, spiking neuron network of the controller may be adapted using reinforcement learning process … FIG. 10B illustrates some implementations of a controller operation method comprising reinforcement learning … At step 1026, spiking neuron network of the controller (e.g., the controller 620 of FIG. 6) may be trained using reinforcement learning … methods of FIGS. 10A-10C may be implemented in one or more processing devices (e.g., an analog circuit designed to process information)”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the growth transform (GT) neuron model of Gangopadhyay to include the spiking neural network is implemented on fully continuous-time analog architecture of Coenen. Doing so would help processing information using the analog network (Coenen, 0268).
Claims 16-18 are rejected under 35 U.S.C. 103 as being unpatentable over Gangopadhyay et al. in view of Hiratani et al. in view of Moraitis et al. and further in view of Laukien et al. (US Pub. 2019/0294980).
As per claim 16, Gangopadhyay teaches
implement a plurality of neurons in respective circuits [page 2, Fig. 1B discloses a coupled network of Growth Transform (GT) neurons];
produce a continuous-valued membrane potential according to a Growth Transform bounded by an extrinsic energy constraint [page 3, 1st paragraph, “as illustrated in Figure 1B, we explore a coupled network of GT neurons … We show that as the input stimulus varies over time, the spiking pattern of the growth transform neuron adapts, effectively tracking and transforming the input”; Fig. 2, page 4, 1st paragraph, “the derivation of the GT neuron model based on the energy minimization framework … neuronal spike generation and spike propagation using a multi-compartmental model as shown in Figure 2. Here each compartment could model a neuron which reacts to external stimuli and propagate signals to other compartments (neurons) by producing a rapid change in their transmembrane electrical potential difference (known as an action potential or a spike)”; page 2, last paragraph, “the spike generation process as the direct derivative of an energy functional of continuous-valued neural variables (e.g. membrane potential), that can moreover be used to mimic known neural dynamics”; Fig. 3 shows the membrane potential Vi defined as a function of spiking current received from another neurons and the stimulus bi; Fig. 2, page 4, 3rd and 4th paragraphs disclose in the growth transform (GT) neuron model, the membrane potential to be always bounded as follows
PNG
media_image2.png
31
141
media_image2.png
Greyscale
the neuron membrane potential being bounded by the ionic potentials on either side of the cell membrane that mediate the changes in membrane potential. We can decompose the response Vi(t) of the i-th neuron and the bias term bi(t) into two differential components
as follows
Vi(t) = Vi+(t) – Vi-(t), and
Bi(t) = bi+(t) – bi-(t), i = 1, … N
where Vi+(t) and Vi-(t) satisfy the following constraints:
Vi+(t) + Vi-(t) = 1
Vi+(t), Vi-(t) ≥ 0.];
define a function of spiking current received from another neuron in the plurality of neurons, and a received electrical current stimulus, as a continuous-valued membrane potential [Fig. 2 shows neuron/compartment i receives stimulus bi and Qij as the inputs, where page 4, 3rd paragraph, “Qij ϵ R denotes the contribution of the pre-synaptic potential at neuron j to the net excitation received by the post-synaptic neuron i, bi(t) ϵ R is the instantaneous external current stimulus”, and Fig. 2, page 4, 1st paragraph, “neuronal spike generation and spike propagation using a multi-compartmental model as shown in Figure 2. Here each compartment could model a neuron which reacts to external stimuli and propagate signals to other compartments (neurons) by producing a rapid change in their transmembrane electrical potential difference (known as an action potential or a spike)”; page 3, 1st paragraph, “as the input stimulus varies over time, the spiking pattern of the growth transform neuron adapts, effectively tracking and transforming the input”; Fig. 3 shows the membrane potential Vi defined as a function of spiking current received from another neurons and the stimulus bi];
represent network energy consumed by all of the plurality of neurons as a network energy function [page 2, 2nd paragraph, “In this paper, we approach the process of spike generation and encoding in a neuron model … by designing an energy functional for a network of neurons which produces an emergent spiking dynamics when minimized under realistic physical constraints”; page 3, 1st paragraph, “In this paper, we first introduce a generic energy (cost) function for spiking neurons by applying a current balance equation at each neural compartment in the network”; page 3, section II, part A, “Energy function for Growth Transform spiking neuron model”; page 6, last paragraph, the energy function D (Vi+, Vi-)];
implement a neuromorphic framework that includes integrating learning and inference [abstract, “the proposed GT neuron model provides a flexible neuromorphic framework to systematically design large-scale spiking neural networks with stable and interpretable dynamics”; page 2, 2nd paragraph, “In this paper, we approach the process of spike generation and encoding in a neuron model from a perspective that is fundamentally different compared to traditional dynamical systems formulations - by designing an energy functional for a network of neurons which produces an emergent spiking dynamics when minimized under realistic physical constraints. This is motivated by the fact that physical processes occurring in nature have an universal tendency to move towards a minimum-energy state - defined by the values taken by the process variables - that is a function of the inputs and the state of the process. Energy-based frameworks are the backbone of most machine learning models, which perform inference by minimizing an energy functional that capture dependencies between the variables”], the neuromorphic network configured to:
minimize network energy consumed by the plurality of neurons to determine the extrinsic energy constraint [abstract, “different forms of coupling between the neurons could produce compact and energy-efficient representation”; page 2, 2nd paragraph, “designing an energy functional for a network of neurons which produces an emergent spiking dynamic when minimized under realistic physical constraints”; page 3, section II. METHODS, 1st paragraph, “we introduce an energy functional for the GT neural network and then derive the basic model of a GT neuron as a result of its minimization”; page 4, 1st paragraph, “the derivation of the GT neuron model based on the energy minimization framework”; page 3, 1st paragraph, “we first introduce a generic energy (cost) function for spiking neurons”; page 6, 2nd paragraph, “we proposed a neuron model that sequentially optimizes the cost function D in Eq. 15 subject to constraints H using a multiplicative update called growth transforms, a fixed-point algorithm for optimizing continuous objective functions over a manifold like H with linear and non-negativity constraints”; page 5, energy minimization problem has the following form
PNG
media_image3.png
169
791
media_image3.png
Greyscale
It can be seen that the energy function is minimized with the constraints determined to satisfy the domain of H] via a min-max optimization technique including a neuronal and synaptic time constant [page 4, 1st paragraph, “the energy minimization framework”; page 5, 2nd and 3rd paragraphs, “The equations in 12 can be viewed as a stochastic variant of the first-order conditions of an energy minimization problem subject to the constraints given by Eqs 7 and 8 … The equivalent energy minimization problem has the following form … we explore how minimizing an energy function of this form can produce single neuron and population dynamics similar to that observed in biological neuronal networks … The cost function D in Eq. 15 comprises two components: (a) a quadratic term that tries to minimize the error between the variables Vi and yi; and (b) a regularization term”; page 6, 2nd paragraph, “to continually minimize the error with which the neural responses track the
time-varying input signal, so that Eq 2 is satisfied on an average”];
model the plurality of synaptic connections among the plurality of neurons as respective transconductances that regulate magnitude of spiking currents received from each of the plurality of neurons by each other of the plurality of neurons [page 3, Fig. 2 discloses the j-th neuron/compartment is coupled to the i-th neuron/compartment by a coupling strength Qij; page 4, 1st paragraph, “We will abstract the physics of neuronal spike generation and spike propagation using a multi-compartmental model as shown in Figure 2. Here each compartment could model a neuron which reacts to external stimuli and propagate signals to other compartments (neurons) by producing a rapid change in their transmembrane electrical potential difference (known as an action potential or a spike) … in this simplified model we assume that the coupling between the compartments i and j is given by a transimpedance variable Qij”; Fig. 3, page 4, 3rd paragraph, “Qij ϵ R denotes the contribution of the pre-synaptic potential at neuron j to the net excitation received by the post-synaptic neuron i], wherein the modeling includes modeling fixed points of network energy, neural responses, and at least one of spiking statistics and transient neural dynamics [page 6, 2nd – 3rd paragraphs, “we proposed a neuron model that sequentially optimizes the cost function D in Eq. 15 subject to constraints H using a multiplicative update called growth transforms, a fixed-point algorithm for optimizing continuous objective functions … we extend the GT neuron model described previously to incorporate different axonal propagation delays for individual neurons in a way that keeps the neural responses tied to the system objective, and produces a stable output”; And, Fig. 8 discloses a growth transform neural network that produces spike trains from which different spike statistics are extracted to encode discriminatory features];
paragraph 0078 of the specification of the current Application recites “Each post-synaptic neuron i, receives electrical input from at least one pre-synaptic neuron j, through a synapse modeled by a transconductance Qij”.
encode the received electrical current stimulus in corresponding continuous-valued membrane potentials of the plurality of neurons [page 8, 1st paragraph, “We then demonstrate how the GT neuron encodes a time-varying stimulus in its spike pattern using different encoding techniques”; page 15, 2nd paragraph, “the GT neuron can encode external stimuli using firing rates and inter-spike intervals, similar to biological neurons. Moreover, the sub-threshold activity of GT neurons was also seen to encode stimulus information in addition to spike-based statistics”; page 3, 2nd paragraph, “When the stimulus is not strong enough to elicit spiking from a neuron, its subthreshold activity still encodes information about the stimulus that cannot be obtained from its spiking activity only”].
Gangopadhyay does not explicitly teach
At least one non-transitory computer-readable storage medium having computer-executable instructions embodied thereon for operating a neural network, wherein when executed by at least one processor, the computer-executable instructions cause the processor to:
implement a neuromorphic framework … using (i) short-term dynamics of activity of the plurality of neurons, and (ii) long-term dynamics of activity of a plurality of synaptic connections among the plurality of neurons;
balancing the short-term dynamics and the long-term dynamics;
encode the received electrical current stimulus in corresponding continuous-valued membrane potentials of the plurality of neurons using an average input of the membrane potentials and a sum of the membrane potentials (emphasis added).
Hiratani teaches
Implement a neuromorphic framework … using (i) short-term dynamics of activity of the plurality of neurons, and (ii) long-term dynamics of activity of a plurality of synaptic connections among the plurality of neurons [Introduction, “Learning and memory are fundamental brain functions supported by hippocampal neural circuits, and long-term potentiation (LTP) and depression (LTD) of synapses are considered to underlie activity-dependent modifications of hippocampal circuits during memory processes … Along with long-term plasticity, cortical synapses also undergo short-term plasticity [8,9]. Short-term plasticity, especially short-term depression (STD), can induce dramatic changes in the characteristic dynamics of recurrent”; page 2, Col. 1, Results section, 1st paragraph, “We construct a recurrent circuit model consisting of 2500 excitatory neurons and 500 inhibitory neurons that are randomly connected with each other. We introduce short-term plasticity and long-term plasticity into synaptic connections between excitatory neurons”; page 11, Col. 2, Model configuration section, 1st paragraph, “We construct a recurrent circuit model based on the chaotic balance network model [51,52] and extend it to include both short-term and long-term plasticity”; Where, long-term potentiation (LTP) and depression (LTD) of synapses are the two processes associated with the long-term dynamics of synaptic activity, and short-term dynamics of activity of the plurality of neurons are driven by short-term plasticity).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the growth transform (GT) neuron model of Gangopadhyay to include a neuromorphic framework using (i) short-term dynamics of activity of the plurality of neurons, and (ii) long-term dynamics of activity of a plurality of synaptic connections among the plurality of neurons of Hiratani. Doing so would help enriching synaptic weight dynamics in neural networks and causing effects on the cell assembly retention and modulation (page 9, Discussion section, 1st paragraph).
Gangopadhyay and Hiratani do not explicitly teach
At least one non-transitory computer-readable storage medium having computer-executable instructions embodied thereon for operating a neural network, wherein when executed by at least one processor, the computer-executable instructions cause the processor to:
balancing the short-term dynamics and the long-term dynamics;
encode the received electrical current stimulus in corresponding continuous-valued membrane potentials of the plurality of neurons using an average input of the membrane potentials and a sum of the membrane potentials (emphasis added).
Moraitis teaches
At least one non-transitory computer-readable storage medium having computer-executable instructions embodied thereon for operating a neural network, wherein when executed by at least one processor, the computer-executable instructions cause the processor to [paragraph 0145, “The computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present invention”]:
balancing the short-term dynamics and the long-term dynamics [abstract, “a method of generating spikes by a neuron of a spiking neural network”; paragraphs 0038-0039, “the long-term synaptic plasticity component is configured to determine changes to the respective synaptic weight W(t) depending on the time distance in pairs of presynaptic and postsynaptic spikes of the given pair, wherein the short-term synaptic plasticity component is configured to determine changes to the respective short-term function F(t) depending on the time of arrival of presynaptic spikes … the short-term function F(t) may be a short-term depression or fatigue effect to the weight W(t) such that the values of G(t) are smaller than W(t)”; paragraph 0081, “In case G2(t)=W2(t) and G1(t) is W1(t) with short-term depression, the apparatus 308B may use the STDP learning rule to tune the efficacy G2(t). W1(t) and W2(t) are synaptic weights W(t) which are indexed to indicate that W1(t) and W2(t) are tuned by the long-term components 311 and 315 of the apparatus 308A and 308B respectively. The STDP learning rule is a long-term synaptic plasticity rule”];
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the growth transform (GT) neuron model of Gangopadhyay to include a process of using a long-term synaptic plastivity rule to tune the efficacy associated with the short-term depression of Moraitis. Doing so would help performing a learning to recognize patterns of long timescales by changing the synaptic weight based on the time distance in pairs of presynaptic and postsynaptic spikes (Moraitis, 0079-0081).
Gangopadhyay, Hiratani and Moraitis do not explicitly teach
encode the received electrical current stimulus in corresponding continuous-valued membrane potentials of the plurality of neurons using an average input of the membrane potentials and a sum of the membrane potentials (emphasis added).
Laukien teaches
encode the received electrical current stimulus in corresponding continuous-valued membrane potentials of the plurality of neurons using an average input of the membrane potentials and a sum of the membrane potentials [paragraph 0173, “activateEncoder () and its kernels pass the hierarchy's inputs up from layer to layer. The input is first combined with its historical values to generate a derived input, and then converted into a stimulus which is the size and shape of the output encoding
PNG
media_image5.png
242
498
media_image5.png
Greyscale
”;
Wherein, paragraph 0174, “deriveinputKernel produces a pair of inputs for the encoder, one a copy of the input and the other a moving average of recent inputs (moving average is a calculation that determines an average of a set of data (set of inputs in this case) over a period of time)”, and paragraph 0175, “encodeKemel produces the encoder's stimulus, which is a weighted sum of each unit's derived inputs”; paragraph 0163, “for example, the derived input D stores a copy of the immediate input Xij in dij1, along with a moving average of past inputs in dij2”; Since Gangopadhyay in Fig. 3, pages 3 and 8 teaches the input stimulus associated with the membrane potential Vi, and the process of encoding the electrical current stimulus in corresponding continuous-valued membrane potentials, while Laukien teaches encoding the stimulus is based in part on the average of the inputs and weighted sum of the inputs, therefore, the combination of Gangopadhyay and Laukien read on the above claim limitation].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the growth transform (GT) neuron model of Gangopadhyay to include the process of encoding the received stimulus using an average input and a sum of the inputs of Laukien. Doing so would help processing streams of time-varying data using at least an encoder which transforming the derived input into a stimulus (Laukien, abstract).
As per claim 17, Gangopadhyay, Hiratani, Moraitis and Laukien teach the computer-readable storage medium of claim 16.
Gangopadhyay further teaches
always bound the membrane potentials by the Growth Transform [Fig. 2, page 4, 3rd and 4th paragraphs disclose in the growth transform (GT) neuron model, the membrane potential to be always bounded as follows
PNG
media_image2.png
31
141
media_image2.png
Greyscale
the neuron membrane potential being bounded by the ionic potentials on either side of the cell membrane that mediate the changes in membrane potential].
As per claim 18, Gangopadhyay, Hiratani, Moraitis and Laukien teach the computer-readable storage medium of claim 16.
Gangopadhyay further teaches
include power dissipation due to coupling between neurons, inject power to or extracted from the plurality of neurons as a result of external stimulation, and dissipate power due to neural responses in the extrinsic energy constraint [page 5, energy minimization problem has the following form
PNG
media_image3.png
169
791
media_image3.png
Greyscale
The above function comprising the extrinsic energy constraint including power dissipation due to coupling between neurons and power injected to or extracted from the plurality of neurons as a result of external stimulation, and power dissipated due to neural responses, according to paragraph 0078 of the specification of the current Application which recites “the network energy function H() and the minimization objective are expressed in EQ. 2. The network energy function represents the extrinsic, or metabolic, power supplied to the network, including power dissipation due to coupling between neurons, power injected to or extracted from the system as a result of external stimulation, and power dissipated due to neural responses.
PNG
media_image4.png
294
524
media_image4.png
Greyscale
”].
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Gangopadhyay et al. in view of Hiratani et al. in view of Moraitis et al. and further in view of Laukien et al. in view of Daily et al. (US Patent 9,020,870) and further in view of Coenen et al. (US Pub. 2014/0081895).
As per claim 20, Gangopadhyay, Hiratani, Moraitis and Laukien teach the computer-readable storage medium of claim 16.
Gangopadhyay teaches
enable finer control over spiking response of the plurality of neurons with a barrier or penalty function [page 8, 2nd paragraph, “The ion-channel function ψ(.) used in this paper is shown in Figure 4. The gradient discontinuity in this case was set to V+ = V- = 0:5. In resting state, Vi+ < (0.5 - ϵ) and Vi- > (0.5 + ϵ), where ϵ is the width of the very narrow transition zone that sets the minimum input current needed for a spike as well as its height. When a depolarizing current pulse comes, both Vi+ and Vi- try to move towards 0.5. If the input current is strong enough, the variables cross over the threshold, producing a spike. This adds a large penalty to the gradient term for Vi+ and the updates force the variables to come back to a hyperpolarized state (a more negative Vi) than where it started from. The resulting membrane potential trace Vi is also given in Figure 4”].
Gangopadhyay further teaches in abstract and page 3, 1st paragraph, “the proposed GT neuron model provides a flexible neuromorphic framework to systematically design large-scale spiking neural networks with stable and interpretable dynamics … as illustrated in Figure 1B, we explore a coupled network of GT neurons … We show that as the input stimulus varies over time, the spiking pattern of the growth transform neuron adapts, effectively tracking and transforming the input”.
Gangopadhyay, Hiratani, Moraitis and Laukien do not teach
using the Growth Transform to store and recall memory patterns for the plurality of neurons to enable the neural network to be an associative memory network, and implementing the neural network on fully continuous-time analog architecture.
Daily teaches
using the Growth Transform to store and recall memory patterns for the plurality of neurons to enable the neural network to be an associative memory network [abstract, “system that uses spiking neuron networks to identify an unknown external stimulus”; Col. 5, lines 15-24, “a method and system capable of learning input patterns, memorizing, recalling and recognizing previously learned patterns. The system uses a spiking neural network coupled with a readout network, along with algorithmic procedures for conditioning and training the networks”; Since Gangopadhyay in abstract teaches the proposed growth transform (GT) neuron model provides a flexible neuromorphic framework to systematically design large-scale spiking neural networks with stable and interpretable dynamics, while Daily teaches the spiking neural network associated with a memory network for memorizing, recalling and recognizing previously learned patterns, therefore, the combination of Gangopadhyay and Daily teaches the above claim limitation].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the growth transform (GT) neuron model of Gangopadhyay to include the process of using the Growth Transform to store and recall memory patterns for the plurality of neurons of Daily. Doing so would help designing a high-capacity memory device and a recall circuitry using a SNN model that can be implemented in hardware (such as VLSI chips) directly, as opposed to algorithms running on general-purpose computers (Daily, Col. 5, lines 61-64).
Gangopadhyay, Hiratani, Moraitis, Laukien and Daily do not teach
implementing the neural network on fully continuous-time analog architecture.
Coenen teaches
implementing the neural network on fully continuous-time analog architecture [abstract, “The controller may comprise spiking neuron network operable according to reinforcement learning process”; paragraph 0078, “methods for facilitating adaptive controller implementation using spiking neuron networks”; paragraphs 0268-0277, “Referring now to FIG. 10A one exemplary implementation of the reinforcement learning method … At step 1004, spiking neuron network of the controller may be adapted using reinforcement learning process … FIG. 10B illustrates some implementations of a controller operation method comprising reinforcement learning … At step 1026, spiking neuron network of the controller (e.g., the controller 620 of FIG. 6) may be trained using reinforcement learning … methods of FIGS. 10A-10C may be implemented in one or more processing devices (e.g., an analog circuit designed to process information)”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the growth transform (GT) neuron model of Gangopadhyay to include the spiking neural network is implemented on fully continuous-time analog architecture of Coenen. Doing so would help processing information using the analog network (Coenen, 0268).
Prior Art
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
Koelmans et al. (US Pub. 2018/0330228) describes spike-timing-dependent plasticity (STDP) in neurons.
Bazhenov et al. (US Pub. 2014/0012789) describes problem solving capabilities of spiking neuron model networks using rewarded STDP.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TRI T NGUYEN whose telephone number is 571-272-0103. The examiner can normally be reached M-F, 8 AM-5 PM, (CT).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, OMAR FERNANDEZ can be reached at 571-272-2589. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TRI T NGUYEN/Examiner, Art Unit 2128
/OMAR F FERNANDEZ RIVAS/Supervisory Patent Examiner, Art Unit 2128