DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in response to the application filed on March 23rd, 2023.
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. KR10-2022-0065968, filed on May 30th, 2022.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-6, 10, 12-14, and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Morie et al., (Morie et al., “ARITHMETIC LOGIC UNIT, MULTIPLY-ACCUMULATE OPERATION DEVICE, MULTIPLY-ACCUMULATE OPERATION CIRCUIT, AND MULTIPLY-ACCUMULATE OPERATION SYSTEM”, US 11782680 B2, Filed. Jul. 5th, 2019, hereinafter “Morie”) in view of Oh et al., (Oh et al., “SPIKING NEURAL NETWORK CIRCUIT”, US2022/0156556 A1, Filed Sep. 1st, 2021, hereinafter “Oh”).
Regarding claim 1, Morie discloses, “A spike neural network circuit, comprising:” (Figure 14, pg. 11; The system proposed in this article contains a neural circuit. This figure shows a neuron circuit which feed signals into the synapse circuits. The information at these circuits is processed and signals are sent to more post-neuron circuits for further processing.)
“a pulse generator configured to receive an input spike signal, and to generate a first modulation pulse and a second modulation pulse based on the input spike signal;” (First embodiment, pp. 30, Col 7, Ln 33-43; “As the input data 4, for example, arbitrary data such as image data, audio data, and statistical data to be processed by the arithmetic logic unit 100 is used. For example, in the 35 case where image data is used as the input data 4, an electrical signal having a signal value corresponding to the pixel value (RGB value, luminance value, etc.) of each of the pixels of the image data is generated. In addition, an electrical signal corresponding to the input data 4 may be 40 appropriately generated in accordance with the type of the input data 4 and the content of the processing by the arithmetic logic unit 100.” This system will receive an input signal and evaluate it. This system is able to handle many types of data and will introduce spiking signals into the system depending on the data type.)
Morie fails to explicitly disclose, “first and second current source arrays controlled based on a weight memory;”, “a membrane capacitor;”, “a first switch configured to deliver a first calculation signal generated from the first current source array to the membrane capacitor, in response to the first modulation pulse; and”, and “a second switch configured to deliver a second calculation signal generated from the second current source array to the membrane capacitor, in response to the second modulation pulse.”.
However, Oh discloses, “first and second current source arrays controlled based on a weight memory;” (Brief description of the figures, pp. 11, [0042]; “The synapse 121 may include a current source I1, a transistor M1, and a weight memory WM1. The weight memory WM1 may store a weight bit corresponding to a weight W1. In some embodiments, the weight memory WM1 may include a register or a memory cell (e.g., a static random access memory (SRAM) cell, a dynamic random access memory (DRAM) cell, a latch, a NANO flash memory cell, a NOR flash memory cell, a resistive random access memory (RRAM) cell, a ferroelectric random access memory (FRAM) cell, a phase change random access memory (PRAM) cell, a magnetic random access memory (MRAM) cell, etc.).” This system contains an array of synapse circuits. The axon circuit will send a signal to the synapse circuit for processing. Each of the synapse circuits contain a weight module where it is able to evaluate and weight incoming signals.)
“a membrane capacitor;” (Brief Description of the figures, pp. 4, [0049]; “The accumulation unit 11 accumulates the charges output to the pair of output lines 7 by the plurality of synapse circuits 8. The accumulation unit 11 includes two capacitors 13a and 13b. The capacitor 13a is connected between the positive output line 7a and a GND. Further, the capacitor 13b is connected between the negative output line 7b and a GND. Therefore, charges flowing in from the output lines 7a and 7b are respectively accumulated in the capacitors 13a and 13b.” The system contains a capacitor which is located after the synapse circuit. The results of the synapse circuit are sent to the neuron circuit which contains the capacitors.)
“a first switch configured to deliver a first calculation signal generated from the first current source array to the membrane capacitor, in response to the first modulation pulse; and” (Brief description of the figures, pp. 4, [0047]; “The synapses 122 and 123 may be implemented in a similar manner to the synapse 121, and may operate in a similar manner to the synapse 121. For example, the synapse 122 may receive a second input spike signal (a second input spike) from the second axon of the axon circuit 110. The synapse 122 may generate a second operation signal based on the weight of the synapse 122 and the second input spike signal. The synapse 123 may receive a third input spike signal (a third input spike) from the third axon of the axon circuit 110.” The input signal will be handled by the synapse circuit. After this the signal is altered depending on the circuit and the signal is then sent to the neuron circuit. This circuit contains a capacitor which is able hold onto signals until a threshold is met and the capacitor will release its charge.)
“a second switch configured to deliver a second calculation signal generated from the second current source array to the membrane capacitor, in response to the second modulation pulse.” (Brief description of the figures, pp. 4, [0047]; “The synapses 122 and 123 may be implemented in a similar manner to the synapse 121, and may operate in a similar manner to the synapse 121. For example, the synapse 122 may receive a second input spike signal (a second input spike) from the second axon of the axon circuit 110. The synapse 122 may generate a second operation signal based on the weight of the synapse 122 and the second input spike signal. The synapse 123 may receive a third input spike signal (a third input spike) from the third axon of the axon circuit 110. The input signal will be handled by the synapse circuit. After this the signal is altered depending on the circuit and the signal is then sent to the neuron circuit. This circuit contains a capacitor which is able hold onto signals until a threshold is met and the capacitor will release its charge.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Morie and Oh. Morie teaches a spiking neural network which include weights and synapse array to perform machine learning functions similar to neural connection and communication in animals. Oh teaches a spiking neural network circuit which uses similar morphology to neural communication and communication in animals. One of ordinary skill would have motivation to combine Two different forms of neural circuitry to develop a neural network using pulses to perform machine learning function, “The present disclosure relates to a circuit implemented in a semiconductor device to perform an operation of a neural network. The neural network of the present disclosure may be an artificial neural network (ANN) capable of processing data or information in a manner similar to a biological neural network. The neural network may include a plurality of layers including artificial neurons similar to biological neurons and synapses connecting the plurality of layers. Hereinafter, a spiking neural network that processes a spike signal having a toggling pulse shape for a short time will be representatively described.”, (Oh, Detailed description, [0032].
Regarding claim 2, Morie discloses, “wherein the first modulation pulse is activated for a shorter time than the second modulation pulse.” (First Embodiments, Col. 10, Ln. 46-55; “Further, the electrical signal is input to the analog circuit 3 during a predetermined input period T. More specifically, the respective electrical signals are input to the analog circuits 3 so that the pulse waveforms of the electrical 50 signals fall within the input period T. Therefore, the maximum value of the pulse width of the electrical signal is similar to the input period T. Note that the timing at which the respective pulse waveforms (electrical signals) are input, and the like are not limited as long as the pulse waveforms 55 fall within the input period T.” This system is able to interpret different forms of data types. This system ill intake the data type and produce a pulse or spike in accordance with that data type. The timing of the pulses is important ant are based on the respected waveforms.)
Regarding claim 3, Morie discloses, “wherein the first modulation pulse and the second modulation pulse are activated from the same time point.” (First Embodiment, Col. 10, ln. 46-55; “Further, the electrical signal is input to the analog circuit 3 during a predetermined input period T. More specifically, the respective electrical signals are input to the analog circuits 3 so that the pulse waveforms of the electrical 50 signals fall within the input period T. Therefore, the maximum value of the pulse width of the electrical signal is similar to the input period T. Note that the timing at which the respective pulse waveforms (electrical signals) are input, and the like are not limited as long as the pulse waveforms 55 fall within the input period T.” The spiking data in input into this system and the signals are sent at a specific time. The signals are sent within a given timeframe.)
Regarding claim 4, Oh discloses, “accumulate the first calculation signal during a time interval when the first modulation pulse is activated; and” (Brief Description of the figures, pp. 4, [0056] “At time t2, in a manner similar to that at time t1, in response to the input spike signal, a transistor of any one of the synapses connected to the capacitor Cmem may be turned on. Accordingly, an operation signal based on input spike signal and the weight of the synapse may be output from the synapse to the capacitor Cmem. Charges corresponding to the output operation signal may be charged in the capacitor Cmem. As a result, a level of voltage Vmem may rise to a voltage V2.” The signals are processed at the synapse circuits. After processing, a signal is sent to a capacitor called Cmem. This is similar to a membrane capacitor and it will collect and contain charges sent to from the synapse circuits. After a threshold is met the capacitor may discharge and return to a initial voltage state.
“accumulate the second calculation signal during a time interval when the second modulation pulse is activated.” (Brief Description of the figures, pp. 4, [0056] “At time t2, in a manner similar to that at time t1, in response to the input spike signal, a transistor of any one of the synapses connected to the capacitor Cmem may be turned on. Accordingly, an operation signal based on input spike signal and the weight of the synapse may be output from the synapse to the capacitor Cmem. Charges corresponding to the output operation signal may be charged in the capacitor Cmem. As a result, a level of voltage Vmem may rise to a voltage V2.” This system is set up in a grid where many of the synapse circuits output to a set of capacitors. As stated above the capacitor will store charges output from the synapse circuits. After a threshold is met the capacitor may discharge and return to an initial voltage state.)
Regarding claim 5, Oh discloses, “wherein a first current source for generating a current of a smallest magnitude among current sources of the first current source array generates a current of the same magnitude as a second current source for generating a current of a smallest magnitude among current sources of the second current source array.” (Brief Description of the figures, pp. 4, [0048]; The first to third input spike signals may have a relatively low voltage level during a relatively short period and a relatively high voltage level during the remaining period. While the first to third input spike signals are not activated (that is, during a period in which the first to third input spike signals have a relatively high voltage level), transistors (e.g., M1) of the synapses 121 to 123 may be in a turned-off state. The first to third input spike signals may be the same as or different from one another.” The first module in this system is able to produce variable magnitudes of spikes. It depends on the input spike and the timing and location of that spike and synapse. This teaches the varied spike magnitudes.)
Regarding claim 6, Morie discloses, “wherein the first current source and the second current source include a first transistor and a second transistor, respectively, and” (First Embodiment, Col. 14, ln. 9-10; “The synapse circuit 8 includes a first MOS transistor 20a, a second MOS transistor 20b, and a flip-flop circuit 30.” The synapse circuits in this mode also contains transistors.)
“wherein a channel width of the first transistor is configured to be same as a channel width of the second transistor.” (First Embodiment, Col. 14, Ln 34-39; “As the first MOS transistor 20a and the second MOS transistor 20b, for example, similar p-MOS transistors prepared on the basis of the same design parameters (gate width, gate length, etc.) are used. In this embodiment, the first MOS transistor 20a and the second MOS transistor 20b correspond to the weight unit.” The parameters of this system requires that the synapse circuits and transistors are similar, this includes channel widths.)
Regarding claim 10, Oh discloses, “a neuron circuit configured to generate an output spike signal, when a voltage level of the membrane capacitor is higher than a threshold voltage level.” (Brief description of the figures, pp. 3, [0039]; “For example, the neuron circuit 130 may compare an accumulated sum of the output results of the synapse circuit 120 with the reference value (or a threshold value). When the accumulated sum exceeds the reference value, the neuron circuit 130 may generate output spike signals (i.e., fire of the neuron). The output spike signals of the neuron circuit 130 may be provided back to the axon circuit 110, may output to the outside of the spiking neural network circuit 100, or may output to other components of the spiking neural network circuit 100.” The capacitor will store the outputs of the synapse circuits. After a certain threshold is met, the capacitor may release its charge to the output line and send a signal to the outgoing system for further processing. The capacitor will then return to an initial voltage state.)
Regarding claim 12, Morie discloses, “A spike neural network circuit, comprising:” (Figure 14, pg. 11; The system proposed in this article contains a neural circuit. This figure shows a neuron circuit which feed signals into the synapse circuits. The information at these circuits is processed and signals are sent to more post-neuron circuits for further processing.)
“a pulse generator configured to receive an input spike signal and generate first and second modulation pulses activated during times of different lengths based on the input spike signal;” (First embodiment, pp. 30, Col 7, Ln 33-43; “As the input data 4, for example, arbitrary data such as image data, audio data, and statistical data to be processed by the arithmetic logic unit 100 is used. For example, in the 35 case where image data is used as the input data 4, an electrical signal having a signal value corresponding to the pixel value (RGB value, luminance value, etc.) of each of the pixels of the image data is generated. In addition, an electrical signal corresponding to the input data 4 may be 40 appropriately generated in accordance with the type of the input data 4 and the content of the processing by the arithmetic logic unit 100.” This system will receive an input signal and evaluate it. This system is able to handle many types of data and will introduce spiking signals into the system depending on the data type.)
Morie fails to explicitly disclose, “a first current source and a second current source;”, “a first weight switch connected between the first current source and a first node and a second weight switch connected between the second current source and a second node;”, “a first switch connected between the first node and an output line and configured to operate in response to the first modulation pulse;”, “a second switch connected between the second node and the output line and configured to operate in response to the second modulation pulse; and”, and “a membrane capacitor connected with the output line.”.
However, Oh discloses, “a first current source and a second current source;” (Brief Description of the figures, pp. 3, [0040]; FIG. 2 is a block diagram illustrating synapses connected to one transmission line in more detail, according to some embodiments of the present disclosure. For convenience of description, illustration of the axon circuit 110 is omitted, only some synapses 121, 122, and 123 of the synapse circuit 120 are illustrated, and one neuron 131 of the neuron circuit 130 is illustrated in FIG. 2. FIG. 2 may be a block diagram for describing that input spike signals output from a plurality of axons are transferred to the synapses 121 to 123 for the operation of the neuron 131.” This model contains an array of synapse circuits. These circuits will calculate an incoming spike and send a resulting signal to the output lines. These outputs are accumulated by a capacitor.)
“a first weight switch connected between the first current source and a first node and a second weight switch connected between the second current source and a second node;” (Brief description of the figures, pp. 11, [0042]; “The synapse 121 may include a current source I1, a transistor M1, and a weight memory WM1. The weight memory WM1 may store a weight bit corresponding to a weight W1. In some embodiments, the weight memory WM1 may include a register or a memory cell (e.g., a static random-access memory (SRAM) cell, a dynamic random-access memory (DRAM) cell, a latch, a NANO flash memory cell, a NOR flash memory cell, a resistive random access memory (RRAM) cell, a ferroelectric random access memory (FRAM) cell, a phase change random access memory (PRAM) cell, a magnetic random access memory (MRAM) cell, etc.).” This system contains an array of synapse circuits. The axon circuit will send a signal to the synapse circuit for processing. Each of the synapse circuits contain a weight module where it is able to evaluate and weight incoming signals.)
“a first switch connected between the first node and an output line and configured to operate in response to the first modulation pulse;” (Brief description of the figures, pp. 3, [0046]; “The transistor M1 may receive the first input spike signal (a first input spike; for example, a negative pulse signal) through a gate terminal. The first terminal of the transistor M1 may be connected to the current source I1. A second terminal (e.g., a drain) of the transistor M1 may be connected to a transmission line. The transistor M1 may be a switch that is turned on or turned off depending on the first input spike signal. When the transistor M1 is turned on depending on the first input spike signal, the transistor M1 may output the current output, that is, the operation signal, from the current source I1 depending on the first input spike signal to the transmission line.” This system will process the spikes in the synapse circuit. The spikes are input into the system and the result of synapse circuit is sent to the output line. This will operate in response to the pulse or spike input in to the synapse circuit.)
“a second switch connected between the second node and the output line and configured to operate in response to the second modulation pulse; and” (Brief description of the figures, pp. 3, [0046]; “The transistor M1 may receive the first input spike signal (a first input spike; for example, a negative pulse signal) through a gate terminal. The first terminal of the transistor M1 may be connected to the current source II. A second terminal (e.g., a drain) of the transistor M1 may be connected to a transmission line. The transistor M1 may be a switch that is turned on or turned off depending on the first input spike signal. When the transistor M1 is turned on depending on the first input spike signal, the transistor M1 may output the current output, that is, the operation signal, from the current source I1 depending on the first input spike signal to the transmission line.” This system will process the spikes in the synapse circuit. The spikes are input into the system and the result of synapse circuit is sent to the output line. This will operate in response to the pulse or spike input in to the synapse circuit. The system in this article is designed with multiple synapse units all of which perform similar actions.)
“a membrane capacitor connected with the output line.” (Brief Description of the figures, pp. 4, [0049]; “The accumulation unit 11 accumulates the charges output to the pair of output lines 7 by the plurality of synapse circuits 8. The accumulation unit 11 includes two capacitors 13a and 13b. The capacitor 13a is connected between the positive output line 7a and a GND. Further, the capacitor 13b is connected between the negative output line 7b and a GND. Therefore, charges flowing in from the output lines 7a and 7b are respectively accumulated in the capacitors 13a and 13b.” The system contains a capacitor which is located after the synapse circuit. The results of the synapse circuit are sent to the neuron circuit which contains the capacitors. As seen in figure 2, the Cmem is connected to the output line)
Regarding claim 13, Morie discloses, “wherein the first modulation pulse is activated for a shorter time than the second modulation pulse.” (First Embodiments, Col. 10, Ln. 46-55; “Further, the electrical signal is input to the analog circuit 3 during a predetermined input period T. More specifically, the respective electrical signals are input to the analog circuits 3 so that the pulse waveforms of the electrical 50 signals fall within the input period T. Therefore, the maximum value of the pulse width of the electrical signal is similar to the input period T. Note that the timing at which the respective pulse waveforms (electrical signals) are input, and the like are not limited as long as the pulse waveforms 55 fall within the input period T.” This system is able to interpret different forms of data types. This system ill intake the data type and produce a pulse or spike in accordance with that data type. The timing of the pulses is important ant are based on the respected waveforms.)
Regarding claim 14, Morie discloses, “wherein the first modulation pulse and the second modulation pulse are activated from the same time point.” (First Embodiment, Col. 10, ln. 46-55; “Further, the electrical signal is input to the analog circuit 3 during a predetermined input period T. More specifically, the respective electrical signals are input to the analog circuits 3 so that the pulse waveforms of the electrical 50 signals fall within the input period T. Therefore, the maximum value of the pulse width of the electrical signal is similar to the input period T. Note that the timing at which the respective pulse waveforms (electrical signals) are input, and the like are not limited as long as the pulse waveforms 55 fall within the input period T.” The spiking data in input into this system and the signals are sent at a specific time. The signals are sent within a given timeframe.)
Regarding claim 16, Morie discloses, “a weight memory storing a binary weight,” (Brief description of the figures, pp. 3, [0042]; “The synapse 121 may include a current source I1, a transistor M1, and a weight memory WM1. The weight memory WM1 may store a weight bit corresponding to a weight W1.” The weights in this system store bit values. Using the broadest reasonable interpretation, a bit is considered a binary form of a number.)
“wherein the first weight switch operates based on a first bit of the binary weight, and the second weight switch operates based on a second bit of the binary weight.” (Figure 6, pp. 7; This figure shows the different weights in the synapse zone. As interpreted above, this figure shows a sequential order of the weights in the zone starting with WMk through WMm.)
Claims 7-9 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Morie and Oh in view of Yajima, (Yajima, “Ultra‑low‑power switching circuits based on a binary pattern generator with spiking neurons”, Jan. 2022, hereinafter “Yajima”).
Regarding claim 7, Oh discloses, “wherein the weight memory is configured to store a weight in form of a binary number,” (Brief description of the figures, pp. 3, [0042]; “The synapse 121 may include a current source I1, a transistor M1, and a weight memory WM1. The weight memory WM1 may store a weight bit corresponding to a weight W1.” The weights in this system store bit values. Using the broadest reasonable interpretation, a bit is considered a binary form of a number.)
“wherein current sources of the first and second current source arrays correspond to weight bits in the form of the binary number, respectively, and” (Brief description of the figures, pp/ 3, [0042]; “The synapse 121 may include a current source I1, a transistor M1, and a weight memory WM1. The weight memory WM1 may store a weight bit corresponding to a weight W1. Each of the synapse circuits contain a weight memory. The output of the synapse is determined using this weight memory.)
Oh and Morie fail to explicitly disclose, “wherein a time length ratio of the first modulation pulse being activated and the second modulation pulse being activated is determined based on a positional value of bits respectively corresponding to the current sources of the first current source array and a positional value of bits respectively corresponding to the current sources of the second current source array.”.
However, Yajima discloses, “wherein a time length ratio of the first modulation pulse being activated and the second modulation pulse being activated is determined based on a positional value of bits respectively corresponding to the current sources of the first current source array and a positional value of bits respectively corresponding to the current sources of the second current source array.” (Simulation of spiking neuron circuit, pp. 5; “In this study, we fabricated a spiking neuron circuit, which is specially designed for waiting time generation in ultra-low power consumption. For the purpose of waiting time generation, the spiking neuron circuit implements the integrate-fire function while all the other biological functions were excluded intentionally. It is also designed to generate a nanosecond-width square pulse wave as the output spike for seamless connection with CMOS logic circuits with a common 1 V supply.”
This system is able to intake a spike and determine a delay for outputting that spike. This system allows for varying delay times which can include sending a spike based on the weight or voltage of the incoming spike.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Morie, Oh and Yajima. Morie teaches a spiking neural network which include weights and synapse array to perform machine learning functions similar to neural connection and communication in animals. Oh teaches a spiking neural network circuit which uses similar morphology to neural communication and communication in animals. Yajima teaches a system that is able to use low power circuits for spiking neural networks. One of ordinary skill would have motivation to combine Two different forms of neural circuitry to develop a neural network using pulses to perform machine learning function with a system that is able provide a low power solution for spiking neural networks, “A comparison in Table 1 clearly shows the fabricated spiking neuron circuits have a unique feature of digital-circuit compatibility in the sense of the spike width and the supply voltage, and at the same time, a reasonably low energy consumption and a wide control range of the waiting time. It should be emphasized that all of these features are optimized for the waiting time generation inside a digital circuit and asynchronously controlling the switching circuits, rather than for the implementation of the biological functions or for the simple reduction of energy consumption in a single neuron circuit as in the case of previous studies.” (Yajima, Experiments on spiking neuron circuits, pp. 7.)
Regarding claim 8, Oh discloses, “wherein the weight in the form of the binary number includes first to N+Mth bits (where N is a natural number of 1 or more) corresponding to a sequential size,” (Figure 6, pp. 7; This figure shows the synapse zone Z1 and the circuits connected to the different weights. As seen in the figure, the weights are in sequential order starting with WMk then proceeding to WMk+1 all the way to WMm.)
“wherein the current sources of the first current source array correspond to the first to Nth bits, respectively,” (Figure 5, pp. 6; This figure shows the different synapse zones, Z1-Z3. As seen in figure 6, each synapse zone contains its own weights of WMk through WMm. The zones are in sequential order as well.)
“wherein the current sources of the second current source array correspond to the N+ 1st to N+Mth bits, respectively, and” (Figure 5, pp. 6; This figure shows the different synapse zones, Z1-Z3. As seen in figure 6, each synapse zone contains its own weights of WMk through WMm. The zones are in sequential order as well.)
Morie and Oh fail to disclose, “wherein time length of the second modulation pulse being activated is 2N times of time length of the first modulation pulse being activated.”.
However, Yajima discloses, “wherein time length of the second modulation pulse being activated is 2N times of time length of the first modulation pulse being activated.” (Simulation of spiking neuron circuit, pp. 5; “The fabricated spiking neuron circuit consists of two parts: one part generates waiting time, and the other part generates a spike (Fig. 3b). In the former part, the input current is created by the ON current or subthreshold current of the transistor under the application of 1 V, and charges the capacitor with the approximately constant current. Then, after a waiting time that is determined by the ratio of the capacitance to the current, the capacitor potential (V1) reaches the threshold voltage of the inverter (around 0.5 V) and activates the spike generation part as shown in Fig. 3c.” This system will generate a spike after a predetermined amount of time. This will intake a spike and wait before sending another spike.)
Regarding claim 9, Oh discloses, “wherein the first bit is a least significant bit for the weight, and” (Figure 6, pp. 7; This figure shows that the Synapse zone 1. Each of the synapse zones contain weight modules. The first bit values are seen on the left which corresponds to WMk. This would signify a lower bit value compared to the other circuits in the synapse zone.)
“wherein the N+Mth bit is a most significant bit for the weight.” (Figure 6, pp. 7; This figure shows that the Synapse zone 1. Each of the synapse zones contain weight modules. The first bit values are seen on the left which corresponds to WMk. This would signify a lower bit value compared to the other circuits in the synapse zone. Further the weights appear to sequential, meaning that the circuits to the right, in the figure, would be WMk+1, this signifies a higher bit by a value of 1.)
Regarding claim 15, Yajima discloses, “wherein a length of a time when the second modulation pulse is activated is 2N times (wherein N is a natural number of 1 or more) a length of a time when the first modulation pulse is activated.” (Simulation of spiking neuron circuit, pp. 5; “The fabricated spiking neuron circuit consists of two parts: one part generates waiting time, and the other part generates a spike (Fig. 3b). In the former part, the input current is created by the ON current or subthreshold current of the transistor under the application of 1 V, and charges the capacitor with the approximately constant current. Then, after a waiting time that is determined by the ratio of the capacitance to the current, the capacitor potential (V1) reaches the threshold voltage of the inverter (around 0.5 V) and activates the spike generation part as shown in Fig. 3c.” This system is able to generate a spike or pulse after a determined amount of time. This time can be programmed to be any set length of time.)
Claims 11, 17, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Morie, Oh and Yajima in view of Garg et al., (Garg et al., “Spiking Neuron Computation With the Time Machine”, Apr. 2012, hereinafter “Garg”).
Regarding claim 11, Morie discloses, “a first pulse output unit configured to output the first modulation pulse activated before a time point when the counted value is greater than a first reference value from a time point when the input spike signal is received; and” (First embodiment, pp. 30, Col 7, Ln 33-43; “As the input data 4, for example, arbitrary data such as image data, audio data, and statistical data to be processed by the arithmetic logic unit 100 is used. For example, in the 35 case where image data is used as the input data 4, an electrical signal having a signal value corresponding to the pixel value (RGB value, luminance value, etc.) of each of the pixels of the image data is generated. In addition, an electrical signal corresponding to the input data 4 may be 40 appropriately generated in accordance with the type of the input data 4 and the content of the processing by the arithmetic logic unit 100.” This system will receive an input signal and evaluate it. This system is able to handle many types of data and will introduce spiking signals into the system depending on the data type.)
Morie and Oh fail to explicitly disclose, “a second pulse output unit configured to output the second modulation pulse activated before a time point when the counted value is greater than a second reference value from the time point when the input spike signal is received.”
However, Yajima discloses, “a second pulse output unit configured to output the second modulation pulse activated before a time point when the counted value is greater than a second reference value from the time point when the input spike signal is received.” (Simulation of spiking neuron circuit, pp. 5; “The fabricated spiking neuron circuit consists of two parts: one part generates waiting time, and the other part generates a spike (Fig. 3b). In the former part, the input current is created by the ON current or subthreshold current of the transistor under the application of 1 V, and charges the capacitor with the approximately constant current. Then, after a waiting time that is determined by the ratio of the capacitance to the current, the capacitor potential (V1) reaches the threshold voltage of the inverter (around 0.5 V) and activates the spike generation part as shown in Fig. 3c.” This system will generate a spike after a predetermined amount of time. This will intake a spike and wait before sending another spike.)
Morie, Oh and Yajima fail to disclose, “a clock generator configured to generate a clock signal in response to the input spike signal;” and “a counter configured to count the number of times the clock signal toggles;”.
However, Garg discloses, “a clock generator configured to generate a clock signal in response to the input spike signal;” (Asynchronous Counter Design, pp. 145; “The transistor count grows linearly with n. Each counter block shares a global count direction signal and a global clock.” This neural network circuit contains a clock and counter. This will generate bit based on the clock and counter value)
“a counter configured to count the number of times the clock signal toggles;” (Asynchronous Counter Design, pp. 145; “The transistor count grows linearly with n. Each counter block shares a global count direction signal and a global clock. The central idea of the counter is that each bit tells the higher bit when to toggle and the higher bit tells the lower bit when its saturated and cannot toggle anymore.” This neural network circuit contains a clock and counter. This will generate bit based on the clock and counter value)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Morie, Oh, Yajima and Garg. Morie teaches a spiking neural network which include weights and synapse array to perform machine learning functions similar to neural connection and communication in animals. Oh teaches a spiking neural network circuit which uses similar morphology to neural communication and communication in animals. Yajima teaches a system that is able to use low power circuits for spiking neural networks. Garg teaches a spiking neural network system that is able to represent weights in time instead of voltage. One of ordinary skill would have motivation to combine Two different forms of neural circuitry to develop a neural network using pulses to perform machine learning function and a system that is able provide a low power solution for spiking neural networks with a system that is able to represent data in different formats and use time as a parameter when handling and processing spikes or pulses in a spiking neural network, “A major advantage of the TM is that of allowing weights to be set independently and digitally for each connection. This solves the problem of storing weights at dedicated synapses in a neuron array. Since these weights are stored digitally, they are easier to compute and store, as compared to setting analog biases as weights. The TM provides flexibility for routing spikes by allowing arbitrary configurable and recurrent connections between neurons that can be set at the beginning of the computation or changed on the fly as compared to dedicated hardware synapses. The hardware design of the architecture allows for swapping of the low-power integrate-and-fire neuron with any other neuron model which accepts a synaptic injection current directly on to the membrane capacitor.” (Garg, Conclusion, pp. 153).
Regarding claim 17, Yajima discloses, “a first latch circuit including a first set terminal for receiving the input spike signal, a first reset terminal, and a first output terminal;” (Figure 1c, pp. 2; This system contains multiple latch gates. This is seen in this figure. The input lint is connected to the S terminal, as well as the output of the gate connects the rest terminal.)
“a second latch circuit including a second set terminal for receiving the first control signal, a second reset terminal for receiving the second control signal, and a second output terminal for outputting the first modulation pulse; and” (figure 1c, pp. 2; This figure shows the use of multiple later gates. These gates are used to send a pulse after a designated amount of time. As seen in the image, the second latch gate inputs the output of the previous in the S terminal and the output of the gate is connected to the Reset terminal.)
“a third latch circuit including a third set terminal for receiving the third control signal, a fourth reset terminal for receiving the fourth control signal, and a third output terminal for outputting the second modulation pulse,” (Figure 1c, pp. 2; “A binary pattern generator, a specially designed pattern generator for purpose of controlling switching circuits. It consists of a chain of waiting time generators with a wide range of preprogrammed waiting times.” The description of this figure discloses the use of a chain of set-reset latch gates. These can be connected in sequence to each other and are portable to other parts of the circuit.)
“wherein the first reset terminal is configured to receive the fourth control signal.” (Figure 1c, pp. 2; This system contains multiple latch gates. This is seen in this figure. The input lint is connected to the S terminal, as well as the output of the gate connects the rest terminal.)
Morie, Oh and Yajima fail to explicitly disclose, “a clock generator connected with the first output terminal and configured to generate a clock signal;”, “a counter configured to receive the input spike signal and count the number of times the clock signal toggles;”, “a first comparator configured to compare a value counted by the counter with a first reference value to generate first and second control signals;”, and “a second comparator configured to compare the counted value with a second reference value to generate third and fourth control signals;”.
However, Garg discloses, “a clock generator connected with the first output terminal and configured to generate a clock signal;” (Asynchronous Counter Design, pp. 145; “The transistor count grows linearly with n. Each counter block shares a global count direction signal and a global clock.” This neural network circuit contains a clock and counter. This will generate bit based on the clock and counter value)
“a counter configured to receive the input spike signal and count the number of times the clock signal toggles;” (Asynchronous Counter Design, pp. 145; “The transistor count grows linearly with n. Each counter block shares a global count direction signal and a global clock. The central idea of the counter is that each bit tells the higher bit when to toggle and the higher bit tells the lower bit when its saturated and cannot toggle anymore.” This neural network circuit contains a clock and counter. This will generate bit based on the clock and counter value)
“a first comparator configured to compare a value counted by the counter with a first reference value to generate first and second control signals;” (Asynchronous counter design “The central idea of the counter is that each bit tells the higher bit when to toggle and the higher bit tells the lower bit when its saturated and cannot toggle any more. When the higher bit is saturated, the lower bit does not ask the higher bit to toggle. Each bit has two input-output pairs to propagate the toggle and saturation information up and down the counter. For each nth counter the ith bit has an input from
T
i
n
,
i
connected to previous bit’s
T
o
u
t
,
i
, an input
S
i
n
,
i
connected to the next bit’s
S
o
u
t
,
i
and a state
Q
i
.” The timed spikes and the counters are used to generate a spike of a given bit. This teaches that the clock and the counter are compared and used to send a specific signal.)
“a second comparator configured to compare the counted value with a second reference value to generate third and fourth control signals;” (Asynchronous counter design “The central idea of the counter is that each bit tells the higher bit when to toggle and the higher bit tells the lower bit when its saturated and cannot toggle any more. When the higher bit is saturated, the lower bit does not ask the higher bit to toggle. Each bit has two input-output pairs to propagate the toggle and saturation information up and down the counter. For each nth counter the ith bit has an input from
T
i
n
,
i
connected to previous bit’s
T
o
u
t
,
i
, an input
S
i
n
,
i
connected to the next bit’s
S
o
u
t
,
i
and a state
Q
i
.” The timed spikes and the counters are used to generate a spike of a given bit. This teaches that the clock and the counter are compared and used to send a specific signal. This model, like many other models, have this circuits in succession, this model can have multiple counters to generate spikes based on the counter and clock.)
Regarding claim 18, Yajima discloses, “wherein the first control signal and the third control signal are activated at a time point when the input spike signal fires,” (Introduction, pp. 2; “In order to generate the arbitrary waiting time and an output spike signal, a technique of spiking neuron circuits with integrate-and-fire function was adopted. The spiking neuron circuits were optimized solely for the purpose of waiting time generation based on the complementary metal oxide semiconductor (CMOS) technology, and any other biological function was not implemented intentionally.” This system is able use circuits to intake a pulse or spike and then generate a new spike or pulse. This will send a spike or pulse after a designated amount of time.)
“wherein the second control signal is activated at a first time point when the counted value becomes greater than the first reference value, and” (Simulation of spiking neuron circuit, pp. 5; “It is also designed to generate a nanosecond-width square pulse wave as the output spike for seamless connection with CMOS logic circuits with a common 1 V supply. The fabricated spiking neuron circuit consists of two parts: one part generates waiting time, and the other part generates a spike (Fig. 3b). In the former part, the input current is created by the ON current or subthreshold current of the transistor under the application of 1 V, and charges the capacitor with the approximately constant current.” A spike can be generated after a given amount of time. This output can be sent to an output line. This output line carries the spike to other circuits in the system. The broadest reasonable interpretation would teach that this circuit can take in a spike and generate a spike after baked in threshold.) And (Figure 1c, pp. 2; As seen in the figure, the latch gates can be placed in sequential circuits to get a determined amount of time. This teaches a portable unit able to be placed in different places of a neural circuit.)
“wherein the fourth control signal is activated at a second time point when the counted value becomes greater than the second reference value.” (Simulation of spiking neuron circuit, pp. 5; “It is also designed to generate a nanosecond-width square pulse wave as the output spike for seamless connection with CMOS logic circuits with a common 1 V supply. The fabricated spiking neuron circuit consists of two parts: one part generates waiting time, and the other part generates a spike (Fig. 3b). In the former part, the input current is created by the ON current or subthreshold current of the transistor under the application of 1 V, and charges the capacitor with the approximately constant current.” A spike can be generated after a given amount of time. This output can be sent to an output line. This output line carries the spike to other circuits in the system. The broadest reasonable interpretation would teach that this circuit can take in a spike and generate a spike after baked in threshold.) And (Figure 1c, pp. 2; As seen in the figure, the latch gates can be placed in sequential circuits to get a determined amount of time. This teaches a portable unit able to be placed in different places of a neural circuit.)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PAUL MICHAEL GALVIN-SIEBENALER whose telephone number is (571)272-1257. The examiner can normally be reached Monday - Friday 8AM to 5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Viker Lamardo can be reached at (571) 270-5871. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PAUL M GALVIN-SIEBENALER/Examiner, Art Unit 2147
/VIKER A LAMARDO/Supervisory Patent Examiner, Art Unit 2147