DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Applicant(s) Response to Official Action
The response filed on 06/15/2026 has been entered and made of record.
Response to Arguments/Amendments
Presented arguments have been fully considered but are held unpersuasive. Examiner’s response to the presented arguments follows below.
Claim Rejections - 35 USC § 103
Summary of Arguments:
Regarding claims 1, 14, 21, the Applicant argues:
“Paragraph 0049 of Kataeva recites, in part: “ the conductance values G1 and G2 of the memristors 53”. Based on above disclosure, Kataeva merely teaches that 53 have different conductance values G1 and G2. Kataeva fails to disclose that 53 includes reference resistors, and each of the reference resistors has the same reference value for subtracting conductance values of other ones of 53. Accordingly, the Kataeva fails to teach the limitations of “the plurality of resistors comprises a plurality of reference resistors, and each of the plurality of reference resistors is coupled to one of the plurality of second wires and has the same reference value for subtracting conductance values of other ones of the plurality of resistors” as explicitly recited in claim 1... As Kara was cited for other reasons different from what was discussed above, Applicant respectfully submits that Kara fails to remedy what Kataeva fails to teach as discussed above.” [Remarks: Pages 9-10]
“Paragraph 0047 of Kataeva recites, in part: “ The input bar 50 and the output bar 51 are connected to each other via the memristor 53”. Based on above disclosure, Kataeva merely teaches that 50 is coupled to each of 51/52. Kataeva fails to disclose that 50 is electrically isolated from one of 51/52 and is coupled to the rest of 51/52. Accordingly, the Kataeva fails to teach the limitations of “the third wire is electrically isolated from one of the plurality of second wires, and is coupled to the rest of the plurality of second wires through a plurality of first resistors” as explicitly recited in Claim 14. Without teachings of “the third wire is electrically isolated from one of the plurality of second wires, and is coupled to the rest of the plurality of second wires through a plurality of first resistors”, Kataeva certainly fails to teach the limitations of “ a third wire configured to receive a bias signal...” ... As Kara was cited for other reasons different from what was discussed above, Applicant respectfully submits that Kara fails to remedy what Kataeva fails to teach as discussed above.” [Remarks: Pages 10-11]
“Paragraph 0059 of Strukov recites, in part: “ applying the activation function to the summation using operational amplifier. f” is the activation function, that can be any nonlinear function such as “tanh”“. Based on above disclosure, Strukov merely teaches that the activation function is applied to the summation. Strukov fails to disclose that the summation is proportional to the activation function and a backpropagation value is proportional to a derivative of the activation function. Accordingly, the Strukov fails to teach the limitations of “the sum value is proportional to the nonlinear activation function, and the backpropagation value is proportional to a derivative of the nonlinear activation function” as explicitly recited in Claim 21. As Kara was cited for other reasons different from what was discussed above, Applicant respectfully submits that Kara fails to remedy what Strukov fails to teach as discussed above.” [Remarks: Page 12]
Regarding claims 2-7, 15-20, 22-26, the Applicant argues:
The claims are allowable “by reason of depending from the allowable independent claims” [Remarks: Page 12]
Examiner’s Response:
Regarding claims 1, 14, 21, the Examiner contends:
The argument fails because it overlooks the inherent design of differential pairs in crossbar arrays to achieve mathematical subtraction, which is explicitly taught in the prior art. Kataeva discloses subtracting conductance values to represent negative weights by connecting output bars to the non-inverting and inverting input terminals of a differential amplifier (Kataeva Para. [0051]). While Kataeva describes this as setting a positive or negative weight across an individual pair of memristors, implementing this exact subtraction mechanism by dedicating one of these memristors (or an entire shared column of memristors) to a fixed baseline reference value is a standard, obvious hardware configuration in the art to achieve the disclosed subtraction. Furthermore, Kara teaches calculating weight corrections and updating stored weights to minimize error, which axiomatically relies on handling signal offsets. Therefore, modifying Kataeva’s differential subtraction mechanism to use a uniform reference resistor value to simplify the array architecture would have been obvious to a person of ordinary skill in the art.
The argument fails because Kara explicitly discloses the use of a bias signal and bias neurons, fully remedying the alleged deficiency in Kataeva. The Examiner's rejection specifically relies on Kara (Paragraphs [0031], [0045]) for teaching “using bias neurons to offset/shift the signals”. Specifically, Kara teaches that the network “may include one or more bias neurons … which do not receive input signals but transmit predetermined bias signals to the next neuron layer” (Kara Para. [0031]). In the physical hardware implementation of a crossbar array, a bias neuron inherently requires an independent row line (a third wire) that provides a constant bias voltage, crosses the output column lines without direct electrical contact (is electrically isolated), and connects to the column lines solely through its respective memristive synapses (first resistors). Therefore, the combination of Kataeva and Kara explicitly renders obvious a third wire receiving a bias signal that is electrically isolated from, but coupled to, the second wires via resistors.
The argument fails because Kara explicitly discloses the mathematical limitation that the backpropagation value is proportional to the derivative of the activation function, directly remedying the alleged deficiency in Strukov. Kara details the backpropagation operation for calculating error values, stating that the error values (
δ
2
j
) for the hidden layer are calculated using a function
g
, where “
g
represents a non-linear function different to the activation function
f
and comprises the derivative of
f
” (Kara Para. [0048]). Thus, the backpropagation value (
δ
2
j
) is explicitly taught to be derived from, and proportional to, the derivative of the nonlinear activation function. Additionally, the rejection accurately relies on Kara (Para. [0048]) to teach calculating backpropagation error signals based on the target value and shifted sum. Applicant's assertion that Kara fails to remedy the deficiency is factually incorrect given the explicit mathematical teachings in Kara’s specification regarding the use of the derivative function.
Examiner’s Response:
Regarding claims 2-7, 15-20, 22-26, the Examiner contends:
The argument is moot based on the response above regarding claims 1, 14 and 21.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-7, 14-20 are rejected under 35 U.S.C. 103 as being unpatentable over Kataeva et al., hereinafter referred to as Kataeva (US 2020/0110991 A1) in view of Kara et al., hereinafter referred to as Kara (US 2019/0122105 A1).
As per claim 1, Kataeva discloses a memory-based device (Kataeva: Abstract; [0044]-[0045]), comprising:
a plurality of first wires (Kataeva: Para. [0046] discloses “a plurality of input bars 50”);
a plurality of second wires arranged across the plurality of first wires (Kataeva: Para. [0046] discloses “a plurality of output bars 51 and 52 are provided in such a manner as to intersect with the plurality of input bars 50”);
a plurality of resistors (memristors 53), wherein each of the plurality of resistors is coupled to one of the plurality of first wires (50) and one of the plurality of second wires (51/52), wherein a plurality of input signals are transmitted (provided) from the plurality of first wires through the plurality of resistors to the plurality of second wires (Kataeva: Para. [0046] discloses “a plurality of memristors 53 for weighting input signals”; Kataeva: Para. [0047] discloses “The memristor 53 for weighting an input signal is provided at an intersection of the input bar 50 and the output bar 51. The input bar 50 and the output bar 51 are connected to each other via the memristor 53”; Kataeva: Para. [0049] discloses “voltage signals V1 and V2 each input from the input neuron 55 to the input bar 50 are multiplied by conductance values G1 and G2, respectively, by the memristors 53”); and
(Kataeva: Paras. [0022], [0024], [0067] disclose applying a non-linear activation function like ReLU and configuring a predetermined bias input, which mathematically shifts the sum value processed by the activation function.),
wherein the plurality of resistors comprises a plurality of reference resistors (Kataeva: Para. [0051] discloses “the memristor 53 [claimed wherein the plurality of resistors comprises a plurality of reference resistors]”), and
each of the plurality of reference resistors is coupled to one of the plurality of second wires and has the same reference value for subtracting conductance values of other ones of the plurality of resistors (Kataeva: Para. [0051] discloses “provided between the input bar 50 and the output bar 52 [claimed and each of the plurality of reference resistors is coupled to one of the plurality of second wires] … is set to a value [claimed and has the same reference value] higher by a positive weight to be set than conductance of the memristor 53 provided between the input bar 50 and the output bar 52 connected to the inverting input terminal [claimed for subtracting conductance values of other ones of the plurality of resistors]” [therefore, by utilizing a differential setup (see figure 3) where the inverting memristor is configured to hold a fixed baseline value, it acts as a uniform reference to subtract against the conductance of the corresponding primary memristors]”).
However, Kataeva does not explicitly disclose “… a processor configured to receive a sum value of the input signals from one of the plurality of second wires, and shift the sum value by a nonlinear activation function to generate a shifted sum value, wherein the processor is configured to calculate a backpropagation value based on the shifted sum value and a target value related to a corresponding input signal of the plurality of input signals, and generate a pulse number based on the corresponding input signal of the plurality of input signals and the backpropagation value, wherein each of a value of the corresponding input signal and the backpropagation value is higher than or equal to a threshold value, wherein the processor is configured to apply a voltage pulse to one of the plurality of resistors related to the corresponding input signal based on the pulse number, in order to modify the value of the corresponding input signal in the plurality of input signals to be the same as the target value.”
Further, Kara is in the same field of endeavor and teaches a processor configured to receive a sum value of the input signals from one of the plurality of second wires (column lines), and shift the sum value by a nonlinear activation function to generate a shifted sum value (Kara: Paras. [0031], [0045] disclose a digital processing unit (DPU 4) receiving the digital signal values corresponding to the weighted sum of the input signals (∑ Wᵢⱼ x₁ᵢ) from the column lines, applying a non-linear activation function (e.g., sigmoid) to the sum and using bias neurons to offset/shift the signals.), wherein the processor is configured to calculate a backpropagation value based on the shifted sum value and a target value related to a corresponding input signal of the plurality of input signals (Kara: Para. [0048] discloses the processor calculating a backpropagation value based on the activated sum and a target value, as the the DPU calculates error values δ3k by subtracting the activation output from the expected network output [target value].), and generate a pulse number (nij) based on the corresponding input signal (x1i) of the plurality of input signals and the backpropagation value (Kara: Paras. [0050], [0062] disclose generating a pulse number based on the input signal and the backpropagation value (calculating weight correction ΔWij using input x1i and error δ2j, and rounding this value to generate a discrete number of programming pulses nij.), wherein each of a value of the corresponding input signal and the backpropagation value is higher than or equal to a threshold value (Kara: Paras. [0048], [0051] disclose executing these calculations specifically for devices “for which x1i and/or δ2j exceed a threshold level,” meeting the requirement that the input and backpropagation values are higher than or equal to a threshold.), wherein the processor is configured to apply a voltage pulse (applied potentiation) to one of the plurality of resistors related to the corresponding input signal based on the pulse number, in order to modify the value of the corresponding input signal in the plurality of input signals to be the same as the target value (Kara: Paras. [0004], [0062], [0067] disclose controller 7 applies |nij| potentiation or depression programming pulses to the device to update the stored weight, repeatedly reducing the error to reach the expected network output [i.e., applying a voltage pulse to the resistor based on the pulse number to modify the network signal outputs to match the target].).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, and having the teachings of Kataeva and Kara before him or her, to modify the memory-based crossbar array of Kataeva to include the processor backpropagation threshold and voltage pulse generation features as described in Kara. The motivation for doing so would have been to improve the efficiency and accuracy of neural network training algorithms by enabling dynamic and precise adjustments to the resistance weights of the memristors.
As per claim 2, Kataeva-Kara disclose the device of claim 1, wherein the plurality of first wires and the plurality of second wires are arranged to form an array, wherein each of the plurality of resistors located at the same column of the array has a first terminal and a second terminal, the first terminals of the plurality of resistors are coupled to different wires of the plurality of first wires, and the second terminals of the plurality of resistor are coupled to the same wire of the plurality of second wires (Kara: Paras. [0032]-[0033] disclose crossbar arrays of memristive devices, connected between row and column lines. The devices 10 are arranged in logical rows and columns with each device connected been a particular row line and column line, indicating devices in the same column are connected to different row lines [first wires] and the same column line [second wire]. Additionally, Kataeva: Paras. [0047]-[0048] disclose the crossbar circuit 44 includes a plurality of input bars 50, a plurality of output bars 51 and 52, a plurality of memristors 53. The memristor 53 for weighting an input signal is provided at an intersection of the input bar 50 and the output bar 51. The input bar 50 and the output bar 51 are connected to each other via the memristor 53.).
As per claim 3, Kataeva-Kara disclose the device of claim 2, wherein the processor is configured to respectively shift sum values from the plurality of second wires located in columns of the array by the nonlinear activation function to generate shifted sum values (Kara: Para. [0042] discloses the activation function typically comprises a non-linear function; Kara: Para. [0045] discloses DPU 4 [the processor] is configured to calculate digital signal values by applying the activation function to the weighted sum signals from the column lines [second wires] of the array to generate output values.).
As per claim 4, Kataeva-Kara disclose the device of claim 3, wherein the processor is configured to calculate backpropagation values based on the shifted sum values and a plurality of target values, wherein each of the plurality of target values corresponds to one of the plurality of input signals (Kataeva: Paras. [0036], [0093]-[0094] disclose calculating a difference “e = target - y” for “back propagation” using actual output voltage y generated at each “shift position” and a “target analog output voltage target” corresponding to the input “Teaching data”, teaching calculating backpropagation values based on shifted sum values and a plurality of target values, wherein each of the plurality of target values corresponds to one of the plurality of input signals.).
As per claim 5, Kataeva-Kara disclose the device of claim 4, wherein the processor is configured to compare each of the values of the plurality of input signals with the threshold value, and compare each of the backpropagation values with the threshold value, wherein the processor is configured to generate pulse numbers based on the plurality of input signals and the backpropagation values if each of the plurality of input signals and the backpropagation values is higher than or equal to the threshold value (Kara: Paras. [0048]-[0051] disclose the processor comparing the input signals and backpropagation values (error signals) with a threshold level, and computing a weight-correction value based on the input signals and backpropagation values if they exceed the threshold level (ΔWij may be computed for only a subset of devices “for which x1i and/or δ2j exceed a threshold level,” where x1i represents input signals and δ2j represents backpropagation error signals); generating pulse numbers based on this computation (The rounded result gives a number n of these programming pulses to be applied to the device).).
As per claim 6, Kataeva-Kara disclose the device of claim 5, wherein the processor is configured to correspondingly apply voltage pulses to the plurality of resistors, through one of the plurality of first wires and one of the plurality of second wires, based on the pulse numbers, so as to change resistance of the plurality of resistors (Kara: Para. [0041] discloses that by application of appropriate programming pulses to a cell via the corresponding row line [first wire] and column line [second wire], the cell can be programmed to a high-resistance or low-resistance state; Kara: Paras. [0058], [0062] disclose the DPU [processor] determining a number of programming pulses to be applied to the device, and applying the corresponding number of potentiation or depression programming pulses to the device to change its state/resistance.).
As per claim 7, Kataeva-Kara disclose the device of claim 1, wherein the nonlinear activation function comprises at least one of a sigmoid function and a rectified linear unit (ReLU) function (Kataeva: Para. [0039] discloses “According to the present embodiment, a ReLU is used as the activation function”.).
As per claim 14, Kataeva discloses a device (Kataeva: Abstract; [0044]-[0045]), comprising:
a plurality of first wires configured to receive a plurality of input signals (Kataeva: Para. [0046] discloses “a plurality of input bars 50”);
a plurality of second wires (Kataeva: Para. [0046] discloses “a plurality of output bars 51 and 52”.);
a plurality of resistors (memristors 53) configured to transmit the plurality of input signals to one of the plurality of second wires to generate a sum value (Kataeva: Paras. [0022], [0047], [0049] disclose each of the neurons calculates a sum of values provided at each intersection of the input bars 50 and the output bars 51; Para. [0046] discloses “a plurality of memristors 53 for weighting input signals”; Kataeva: Para. [0047] discloses “The memristor 53 for weighting an input signal is provided at an intersection of the input bar 50 and the output bar 51. The input bar 50 and the output bar 51 are connected to each other via the memristor 53”; Kataeva: Para. [0049] discloses “voltage signals V1 and V2 each input from the input neuron 55 to the input bar 50 are multiplied by conductance values G1 and G2, respectively, by the memristors 53”.); and
a processor configured to:
shift the sum value by a nonlinear activation function to generate a shifted sum value (Kataeva: Paras. [0022], [0024], [0067] disclose applying a non-linear activation function like ReLU and configuring a predetermined bias input, which mathematically shifts the sum value processed by the activation function.),
a third wire
wherein the third wire is electrically isolated from one of the plurality of second wires (Kataeva: Para. [0046] discloses “a plurality of output bars 51 and 52 [claimed one of the plurality of second wires] are provided in such a manner as to intersect with the plurality of input bars 50 [claimed wherein the third wire is electrically isolated from]”),
and is coupled to the rest of the plurality of second wires through a plurality of first resistors (Kataeva: Para. [0047] discloses “The input bar 50 and the output bar 51 are connected to each other via the memristor 53 [claimed and is coupled to the rest of the plurality of second wires through a plurality of first resistors]”).
However, Kataeva does not explicitly disclose “… calculate a feedback based on the shifted sum value and a target value related to a corresponding input signal of the plurality of input signals, generate a pulse number based on the corresponding input signal and the feedback, apply a voltage pulse to one of the plurality of resistors related to the corresponding input signal based on the pulse number, and modify a value of the corresponding input signal to be the same as the target value; and a third wire configured to receive a bias signal …”.
Further, Kara is in the same field of endeavor and teaches calculate a feedback based on the shifted sum value and a target value related to a corresponding input signal of the plurality of input signals (Kara: Para. [0048] discloses the processor calculating a backpropagation value based on the activated sum and a target value, as the the DPU calculates error values δ3k by subtracting the activation output from the expected network output [target value].),
generate a pulse number (nij) based on the corresponding input signal (x1i) and the feedback (Kara: Paras. [0050], [0062] disclose generating a pulse number based on the input signal and the backpropagation value (calculating weight correction ΔWij using input x1i and error δ2j, and rounding this value to generate a discrete number of programming pulses nij.),
apply a voltage pulse (applied potentiation) to one of the plurality of resistors related to the corresponding input signal based on the pulse number, in order to modify the value of the corresponding input signal in the plurality of input signals to be the same as the target value (Kara: Paras. [0004], [0062], [0067] disclose controller 7 applies |nij| potentiation or depression programming pulses to the device to update the stored weight, repeatedly reducing the error to reach the expected network output [i.e., applying a voltage pulse to the resistor based on the pulse number to modify the network signal outputs to match the target].); and
a third wire configured to receive a bias signal (Kara: Para. [0031] discloses “bias neurons [claimed a third wire] which do not receive input signals but transmit predetermined bias signals [claimed configured to receive a bias signal]”).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, and having the teachings of Kataeva and Kara before him or her, to modify the memory-based crossbar array of Kataeva to include the feedback voltage pulse generation and receiving bias signal feature as described in Kara. The motivation for doing so would have been to improve the efficiency and accuracy of neural network training algorithms by enabling dynamic and precise adjustments to the resistance weights of the memristors.
As per claim 15, the claim(s) recites analogous limitations to claim(s) 3 above, and is/are therefore rejected on the same premise.
As per claim 16, the claim(s) recites analogous limitations to claim(s) 4 above, and is/are therefore rejected on the same premise.
As per claim 17, Kataeva-Kara disclose the device of claim 16, wherein the processor is further configured to:
compare each of the plurality of input signals with a threshold value (Kara: Paras. [0048]-[0051] disclose the processor comparing the input signals and backpropagation values [feedbacks] with a threshold level, and computing a weight-correction value based on the input signals and backpropagation values if they exceed the threshold level (ΔWij may be computed for only a subset of devices “for which x1i and/or δ2j exceed a threshold level,” where x1i represents input signals and δ2j represents backpropagation error signals); generating pulse numbers based on this computation (The rounded result gives a number n of these programming pulses to be applied to the device).),
compare each of the plurality of feedbacks with the threshold value (Kara: Paras. [0048]-[0051] disclose the processor comparing the input signals and backpropagation values [feedbacks] with a threshold level.), and
adopt the plurality of input signals and the plurality of feedbacks when the plurality of input signals and the plurality of feedbacks are higher than the threshold value (Kara: Paras. [0048]-[0051] disclose computing a weight-correction value based on the input signals and backpropagation values if they exceed the threshold level (ΔWij may be computed for only a subset of devices “for which x1i and/or δ2j exceed a threshold level,” where x1i represents input signals and δ2j represents backpropagation error signals); generating pulse numbers based on this computation (The rounded result gives a number n of these programming pulses to be applied to the device).).
As per claim 18, Kataeva-Kara disclose the device of claim 17, wherein the plurality of feedbacks comprise backpropagation values (Kara: Paras. [0048]-[0051] disclose computing a weight-correction value based on the input signals and backpropagation values [feedbacks].).
As per claim 19, Kataeva-Kara disclose the device of claim 18, wherein the processor is further configured to:
generate a plurality of pulse numbers based on the plurality of input signals and the plurality of feedbacks, and correspondingly apply voltage pulses to the plurality of resistors, through one of the plurality of first wires and one of the plurality of second wires, based on the plurality of pulse numbers, so as to change resistances of the plurality of resistors (Kara: Para. [0041] discloses that by application of appropriate programming pulses to a cell via the corresponding row line [first wire] and column line [second wire], the cell can be programmed to a high-resistance or low-resistance state; Kara: Paras. [0058], [0062] disclose the DPU [processor] determining a number of programming pulses to be applied to the device, and applying the corresponding number of potentiation or depression programming pulses to the device to change its state/resistance.).
As per claim 20, the claim(s) recites analogous limitations to claim(s) 7 above, and is/are therefore rejected on the same premise.
Claims 21-26 are rejected under 35 U.S.C. 103 as being unpatentable over Strukov et al., hereinafter referred to as Strukov (US 2021/0019609 A1) in view of Kara et al., hereinafter referred to as Kara (US 2019/0122105 A1).
As per claim 21, Strukov discloses a device (Strukov: Abstract.), comprising:
a first processor (Strukov: Para. [0136] discloses one or more processors 500.);
a plurality of resistors (variable resistors), wherein a plurality of input signals are transmitted from the first processor through the plurality of resistors (Strukov: Para. [0084] discloses each synaptic weight is implemented with one or several non-volatile memory elements acting as a variable resistor whose resistance can be tuned. If the input is applied as a voltage signal to rows of the crossbar, current on each column will be the dot product.); and
a second processor configured to receive a sum value of the input signals from one of the plurality of resistors, and shift the sum value by a nonlinear activation function to generate a shifted sum value (Strukov: Paras. [0059], [0136] disclose circuits/processors performing a summation and applying the activation function to the summation using operational amplifier. f' is the activation function, that can be any nonlinear function such as 'tanh'.),
However, Strukov does not explicitly disclose “… wherein the second processor is configured to calculate a backpropagation value based on the shifted sum value and a target value related to a corresponding input signal of the plurality of input signals, and generate a pulse number based on the corresponding input signal of the plurality of input signals and the backpropagation value, wherein each of a value of the corresponding input signal and the backpropagation value is higher than or equal to a threshold value, wherein the second processor is configured to apply a voltage pulse to one of the plurality of resistors related to the corresponding input signal based on the pulse number, in order to modify the value of the corresponding input signal in the plurality of input signals to be the same as the target value, wherein the sum value is proportional to the nonlinear activation function and the backpropagation value is proportional to a derivative of the nonlinear activation function.”.
Further, Kara is in the same field of endeavor and teaches wherein the second processor is configured to calculate a backpropagation value based on the shifted sum value and a target value related to a corresponding input signal of the plurality of input signals (Kara: Para. [0048] discloses the processor calculating a backpropagation value based on the activated sum and a target value, as the DPU calculates error values δ3k by subtracting the activation output from the expected network output [target value].), and generate a pulse number (nij) based on the corresponding input signal (x1i) of the plurality of input signals and the backpropagation value (Kara: Paras. [0050], [0062] disclose generating a pulse number based on the input signal and the backpropagation value (calculating weight correction ΔWij using input x1i and error δ2j), and rounding this value to generate a discrete number of programming pulses nij.), wherein each of a value of the corresponding input signal and the backpropagation value is higher than or equal to a threshold value (Kara: Paras. [0048], [0051] disclose executing these calculations specifically for devices “for which x1i and/or δ2j exceed a threshold level,” meeting the requirement that the input and backpropagation values are higher than or equal to a threshold.), wherein the second processor is configured to apply a voltage pulse (applied potentiation) to one of the plurality of resistors related to the corresponding input signal based on the pulse number, in order to modify the value of the corresponding input signal in the plurality of input signals to be the same as the target value (Kara: Paras. [0004], [0062], [0067] disclose controller 7 applies |nij| potentiation or depression programming pulses to the device to update the stored weight, repeatedly reducing the error to reach the expected network output [i.e., applying a voltage pulse to the resistor based on the pulse number to modify the network signal outputs to match the target].),
wherein the sum value is proportional to the nonlinear activation function and the backpropagation value is proportional to a derivative of the nonlinear activation function (Kara: Paras. [0045]-[0046] disclose applying the activation function f [claimed is proportional to the nonlinear activation function] to the digital signal values corresponding to the array output signals (i.e., ΣiŴijx1i) [claimed the sum value] and Kara: Para. [0048] discloses “error values [claimed the backpropagation value] … where g represents a non-linear function different to the activation function f and comprises the derivative of f [claimed is proportional to a derivative of the nonlinear activation function]” [i.e., the backpropagated error value is calculated by applying a function g, which is explicitly defined as the exact derivative of the activation function f used in the forward pass].).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, and having the teachings of Strukov and Kara before him or her, to modify the neuromorphic computing circuitry of Strukov to include the processor backpropagation threshold and voltage pulse generation features as described in Kara. The motivation for doing so would have been to improve the efficiency and accuracy of neural network training algorithms by enabling dynamic and precise adjustments to the resistance weights of the memristors.
As per claim 22, Strukov-Kara disclose the device of claim 21, wherein the second processor is configured to respectively shift sum values by the nonlinear activation function to generate shifted sum values (Strukov: Para. [0136] discloses one or more processors 500; Kara: Para. [0042] discloses the activation function typically comprises a non-linear function; Kara: Para. [0045] discloses DPU 4 is configured to calculate digital signal values by applying the activation function to the weighted sum signals from the column lines of the array to generate output values.).
As per claim 23, Strukov-Kara disclose the device of claim 22, wherein the second processor is configured to calculate backpropagation values based on the shifted sum values and a plurality of target values, wherein each of the plurality of target values corresponds to one of the plurality of input signals (Strukov: Para. [0136] discloses one or more processors 500; Kara: Para. [0048] discloses calculating error values for backpropagation based on an “expected output” and “digital signal values”, which teaches calculating backpropagation values based on sum values and target values.).
As per claim 24, Strukov-Kara disclose the device of claim 23, wherein the second processor is configured to compare each of the values of the plurality of input signals with the threshold value, and compare each of the backpropagation values with the threshold value, wherein the second processor is configured to generate pulse numbers based on the plurality of input signals and the backpropagation values if each of the plurality of input signals and the backpropagation values is higher than or equal to the threshold value (Strukov: Para. [0136] discloses one or more processors 500; Kara: Paras. [0048]-[0051] disclose the processor comparing the input signals and backpropagation values (error signals) with a threshold level, and computing a weight-correction value based on the input signals and backpropagation values if they exceed the threshold level (ΔWij may be computed for only a subset of devices “for which x1i and/or δ2j exceed a threshold level,” where x1i represents input signals and δ2j represents backpropagation error signals); generating pulse numbers based on this computation (The rounded result gives a number n of these programming pulses to be applied to the device).).
As per claim 25, Strukov-Kara disclose the device of claim 24, wherein the second processor is configured to correspondingly apply voltage pulses to the plurality of resistors based on the pulse numbers, so as to change resistance of the plurality of resistors (Strukov: Para. [0136] discloses one or more processors 500; Kara: Para. [0041] discloses that by application of appropriate programming pulses to a cell via the corresponding row line [first wire] and column line [second wire], the cell can be programmed to a high-resistance or low-resistance state; Kara: Paras. [0058], [0062] disclose the DPU [processor] determining a number of programming pulses to be applied to the device, and applying the corresponding number of potentiation or depression programming pulses to the device to change its state/resistance.).
As per claim 26, Strukov-Kara disclose the device of claim 21, wherein the nonlinear activation function comprises at least one of a sigmoid function and a rectified linear unit (ReLU) function (Strukov: Para. [0059] discloses the activation function can be any nonlinear function.).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure and can be viewed in the list of references.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action. Any inquiry concerning this communication or earlier communications from the examiner should be directed to PEET DHILLON whose telephone number is (571)270-5647. The examiner can normally be reached M-F: 5am-1:30pm. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sath V. Perungavoor can be reached at 571-272-7455. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PEET DHILLON/Primary Examiner
Art Unit: 2488
Date: 07-31-2026
Prosecution Insights