5Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 5/26/2026 have been fully considered but they are not persuasive.
Regarding applicant’s arguments for 35 U.S.C 103 rejection, applicant argues in page 7-9 “
New Claims
Claims 16-33 have been added into the application through this Response. Applicant
respectfully submits that these new claims describe inventions novel and unobvious in view
of the prior art of record and, therefore, respectfully requests that the new claims be held to
be allowable.
In addition, the prior art of record at least does not teach or suggest "resuming
inference operations for the rows for which inference operations were suspended once a
synaptic weight update signal has been applied to each row for which the calculated error
exceeded the error threshold."
For at least the above reasons, Applicant respectfully submits that independent claim
25 and its dependents are also allowable over the prior art of record, including Gokmen.” Applicant argues new claim set 16-33 and how it is not obvious over the cited art. The applicant argues new claim set and the new claim set has not been examined, thus the argument is moot and not convincing.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 16-21 and 25-30 are rejected under 35 U.S.C. 103 as being unpatentable over Gokman (US20180075338A1) (“Gokmen”) in view of Strachan et al. (US20200073755A1) (“Strachan”) and further in view of Payvand, Melika, et al. "Error-triggered Three-Factor Learning Dynamics for Crossbar Arrays." arXiv preprint arXiv:1910.06152 (2019) (“Payvand”).
Regarding claim 16 and analogous claim 25, Gokmen teaches A neural network learning system comprising: a neural network including a memristive crossbar array including multiple conductive rows, multiple conductive columns, a memristive device provided at the intersection of each row and column, and neural circuits coupled to the memristive crossbar array configured to generate output signals (Gokman FIG. 8
PNG
media_image1.png
782
978
media_image1.png
Greyscale
[a neural network including a memristive crossbar array including multiple conductive rows, multiple conductive columns,]
Para 0088 line 1-9, FIG. 8 is a diagram of a two-dimensional (2D) crossbar array 800 that performs forward matrix multiplication, backward matrix multiplication and weight updates according to the present description. Crossbar array 800 is formed from a set of conductive row wires 802, 804, 806 and a set of conductive colunm wires 808, 810, 812, 814 that
intersect the set of conductive row wires 802, 804, 806. The intersections between the set of row wires and the set of column wires are separated by RPUs [a memristive device provided at the intersection of each row and column, and neural circuits coupled to the memristive crossbar array configured to generate output signals],);
input circuitry configured to apply input signals to the neural network ((Gokman para 0071 line 6-14, FIG. 6 illustrates an example flowchart for training a CNN with one or more convolutional layers 500. The example logic can be implemented by a processor, such as a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), or any other processor or a combination thereof. Alternatively or in addition, the training can be performed by a system that is equipped with an RPU array as described herein [input circuitry]. The CNN can be initialized using random weights for the neurons, as shown at block 602. The training data for the CNN 300 is received, as shown at block 605. In an example, the CNN can be pre-set with sample convolutional kernels and biases, which are refined to give consistent and efficient results. The training data typically includes many ( on the order of tens of thousands) input training samples 400, which are associated with an expected output 460 [configured to apply input signals to the neural network]);
and weight update circuitry configured to adjust conductances of the memristive devices to update synaptic weight values stored by the devices (Gokman
Fig. 19,
PNG
media_image2.png
642
687
media_image2.png
Greyscale
Para 0076, FIG. 7A depicts a simplified illustration of a typical read-process-write weight update operation, wherein CPU/ GPU cores (i.e., simulated "neurons") read a memory (i.e., a simulated "synapse") and perform weight update processing operations, then write the updated weights back to memory.
para 0089 line13-22, For weight updates, which are described in greater detail below, voltages are applied to column wires and row wires at the same time, and the conductance values stored in the relevant RPU devices all update in parallel. Accordingly, the multiplication and addition operations required to perform weight updates are performed locally at each RPU 820 of array 800 using the RPU device itself plus the relevant row or column wire of array 800 [weight update circuitry]. Thus, in accordance with the present invention, no read-update-write cycles (shown in FIG. 7A) are required in array 800 [configured to adjust conductances of the memristive devices to update synaptic weight values stored by the devices]),
wherein the neural network is configured to operate in an inference mode in which the neural circuits provide output signals to an error calculation circuit of the weight update circuitry configured to calculate errors associated with individual rows of the crossbar array (Gokman para 0075, The CNN, for each layer, adapts a matrix of weights A and a vector of biases a to optimize E. To this end, in the forward pass, a value for each value of a next layer (B, b) is calculated using values of the current layer (A, a). For example, the computations in the forward pass for a layer can be represented as X=f(S)=Bcp(AS+a)+b, where, A is the matrix of weights of a current layer, a is a bias vector of the current layer, and Band bare weight matrix and bias of the next layer of the CNN. The function cp represents an element- wise non-linear relation. In the forward pass, the predicted outputs corresponding to the inputs are evaluated according to the above equation [wherein the neural network is configured to operate in an inference].
Para 0110 line 5-12, A neuron interface 1908 controls neurons on the CNN, determining whether the neurons are in feed forward mode, back propagation mode, or weight update mode. The neuron interface 1908 furthermore provides inputs to input neurons and receives the output from output neurons. An error calculation module 1910 compares the outputs from the neurons to training data 1906 to determine an error signal. Neuron interface 1908 applies the error signal to the output neurons during a back propagation mode and subsequently triggers a weight update mode to train the weights of the CNN accordingly [in which the neural circuits provide output signals to an error calculation circuit of the weight update circuitry].
PNG
media_image3.png
646
982
media_image3.png
Greyscale
Para 0122 line 5-8, The RPU array 800 facilitates performing the vector matrix multiplications for the backward pass as described herein (see FIG. 16). FIG. 26 illustrates the computations that are performed for the backward pass.
Para 0123 line 1-10, During back propagation, the output neurons provide a voltage back across the array of RPU devices. The output layer compares the generated network response to training data and computes an error. The error is applied to the RPU array 800 as a voltage pulse, where the height and/or duration of the pulse is modulated proportional to the error value. In this example, a row of RPU devices receives a voltage from a respective output neuron in parallel and converts that voltage into a current which adds column-wise to provide an input to hidden neurons [update circuitry configured to calculate errors associated with individual rows of the crossbar array].),
wherein the weight update circuitry is configured to temporarily suspend inference operations [for the rows for which an error event corresponding to the calculated error exceeding an error threshold occurs and to transition those particular rows into a learning mode in which synaptic weight update signals are selectively applied to those rows] (Gokman para 0072 line 4-18, Using the input maps 410 and the convolutional kernels 420, the output maps 430 are generated as described herein, as shown at block 610. Generating the output maps 330 is commonly referred to as a "forward pass." Further, the method includes using the output maps 430, to determine how close or far off of the expected character recognition and the CNN was, as shown at block 615. A degree of error with relation to each of the matrices, which include the CNN is determined, such as using a gradient descent. Determining the relative errors is referred to as a "backward pass." The method further includes modifying or updating the matrices to adjust for the error, as shown at block 625. The adjusting the convolution kernels 420 based on the output error information and using it to determine modifications for each neural network matrix, is referred to as an "update pass."
Gokman para 0122 line 5-8, The RPU array 800 facilitates performing the vector matrix multiplications for the backward pass as described herein (see FIG. 16). FIG. 26 illustrates the computations that are performed for the backward pass [wherein the weight update circuitry is configured to temporarily suspend inference operations].), and
wherein the weight update circuitry is further configured to resume inference operations [for the rows for which inference operations were suspended once a synaptic weight update signal has been applied to each row for which the calculated error exceeded the error threshold] (Gokman
Para 0072 line 1-10, In one or more embodiments, the CNN training is performed using batches. Accordingly, a batch of the input data to be used for training is selected, as shown at block 608. Using the input maps 410 and the convolutional kernels 420, the output maps 430 are generated as described herein, as shown at block 610. Generating the output maps 330 is commonly referred to as a "forward pass." Further, the method includes using the output maps 430, to determine how close or far off of the expected character recognition and the CNN was, as shown at block 615.
PNG
media_image4.png
460
566
media_image4.png
Greyscale
para 0116 line 1-7, As shown in FIG. 24, the method includes initializing the RPU array 800 according to the dimensions of the parameters associated with the CNN layer being trained, as shown at block 2410. In one or more embodiments, the RPU array 800 is initialized to Mxk2N dimensions, where M is the number of convolution kernels, k is the dimension of the convolution kernel, and N is the number of input maps [wherein the weight update circuitry is further configured to resume inference operations]).
However Gokman does not explicitly teach [wherein the weight update circuitry is configured to temporarily suspend inference operations] for the rows for which an error event corresponding to the calculated error exceeding an error threshold occurs and to transition those particular rows into a learning mode in which synaptic weight update signals are selectively applied to those rows, and [wherein the weight update circuitry is further configured to resume inference operations] for the rows for which inference operations were suspended once a synaptic weight update signal has been applied to each row for which the calculated error exceeded the error threshold.
However Strachan [wherein the weight update circuitry is configured to temporarily suspend inference operations] for the rows for which an error event corresponding to the calculated error exceeding an error threshold occurs and to transition those particular rows into a learning mode in which synaptic weight update signals are selectively applied to those rows (Strachan figure 2. 202, 206,
PNG
media_image5.png
553
870
media_image5.png
Greyscale
para 0022, In examples, the system 100 can implement a calibration step to determine a set of threshold error values for each layer of the data flow, where each threshold value defines a boundary at which the error value transitions from insignificant to significant. The set of threshold error values at individual layers can be determined through forward propagation of detected or introduced error values through the data flow, after which the forward propagated values can be evaluated for significance. In this way, the forward propagation can identify a sensitivity of the data flow 10 to the introduced error value.
Para 0023, In examples, an output error array 23 may be determined in connection with the result array 21, where the output error array 23 represents the forward propagation of error values that are introduced through implementation of the data flow. The output error array 23 can be evaluated to determine those nodes or cells for which the output error value is significant. The output error array 23 can be subjected to a backward propagation process which correlates the output error array to the error arrays 32, 34, 36, 38 of each of the respective layers. Each cell or node of the output error array 23 which is deemed significant can be correlated to the respective cell(s) or node(s) of one or more of the error arrays, with the value of the significant cells or nodes providing a basis for setting the threshold error value for the respective cells or nodes of the individual error arrays 32, 34, 36, 38. The calibration step can be repeated over time, to tune the value of the error thresholds, and to populate error thresholds for individual cells or nodes of the respective error arrays 32, 34, 36, 38 [for the rows for which an error event corresponding to the calculated error exceeding an error threshold occurs and]),
Gokmen and Strachan are both considered are both considered to be analogous to the claimed invention of machine learning. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Gokmen to incorporate the teachings of Strachan and have an error threshold value that is adjustable. Doing so to evaluate the error of arrays and determine the significant of each cell or node and set a threshold error value for setting the threshold error value for the respective cells or nodes of the individual error arrays (Strachan Para 0023 In examples, an output error array 23 may be determined in connection with the result array 21, where the output error array 23 represents the forward propagation of error values that are introduced through implementation of the data flow. The output error array 23 can be evaluated to determine those nodes or cells for which the output error value is significant. The output error array 23 can be subjected to a backward propagation process which correlates the output error array to the error arrays 32, 34, 36, 38 of each of the respective layers. Each cell or node of the output error array 23 which is deemed significant can be correlated to the respective cell(s) or node(s) of one or more of the error arrays, with the value of the significant cells or nodes providing a basis for setting the threshold error value for the respective cells or nodes of the individual error arrays 32, 34, 36, 38. The calibration step can be repeated over time, to tune the value of the error thresholds, and to populate error thresholds for individual cells or nodes of the respective error arrays 32, 34, 36, 38.).
Payvand teaches and [wherein the weight update circuitry is further configured to resume inference operations] for the rows for which inference operations were suspended once a synaptic weight update signal has been applied to each row for which the calculated error exceeded the error threshold (Payvand Fig. 2,
PNG
media_image6.png
573
1274
media_image6.png
Greyscale
[for the rows for which inference operations were suspended]
As a result, a voltage is developed on the gate of the M1 connected to a differential pair which re-normalizes the sum of the currents from the crossbar to Inorm. This ensures that the currents remain in the subthreshold regime for the next-stage of the computation which is the ternary error generation as is specified in equation (4). This is done through the Variable Width Bump (VWBump) circuit that compares Inn to the target ^y, with a stop region. Thus, the VWBump circuit output indicates the sign of the weight update (up or down) or stop-learning (no update). The circuit (not shown) is based on the bump circuit [18], which consists of a differential pair for the comparison and a current correlator for the stop region, and is modified to have a tunable stop-learning region [19]. The boundaries of this region play the role of _ in (4). The output of the block is plotted in the inset of Fig. 2, which shows the Up, Down and STOP outputs. [once a synaptic weight update signal has been applied to each row for which the calculated error exceeded the error threshold]).
Gokmen and Payvand are both considered are both considered to be analogous to the claimed invention of machine learning. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Gokmen to incorporate the teachings of Payvand and use error triggered updates. Doing so to update only the selected devices leading to an extended lifetime of the devices and less write energy consumption (Payvand Page4-5 Conclusion para 1 line 1-8, In this article, we demonstrated an error-triggered learning rule that is particularly well-suited for implementation in crossbars. Our implementation leverages the linear property of the subthreshold dynamics, such that the memory required for computing the gradients (i.e. the synaptic traces) grows linearly with the neurons (hence one Pj per input neuron). By updating weights asynchronously (when errors occur), the number of weight writes can be drastically reduced.
Para. 3 line 1-9, Despite of the huge benefit of the crossbar array structure, the memristor devices suffer from many challenges that might affect the performance unless taken into consideration in the training such as asymmeteric nonlinearity, precision and retention. Fortunately, online learning helps with other problems such as sneak path (i.e wire resistance) and endurance. With error-triggered learning rule, only selected devices are updated which leads to extending the lifetime of the devices and less write energy consumption).
Regarding claim 17 and analogous claim 26, Gokmen in view of Strachan and Payvand teach the system of claim 16 and analogous 25.
Gokman, Stracan and Payvand are combined with the same rationale used in claim 16 and analogous claim 25.
Gokman further teaches wherein the system performs no backward propagation through the neural network (Gokman Para 0072 line 1-10, In one or more embodiments, the CNN training is performed using batches. Accordingly, a batch of the input data to be used for training is selected, as shown at block 608. Using the input maps 410 and the convolutional kernels 420, the output maps 430 are generated as described herein, as shown at block 610. Generating the output maps 330 is commonly referred to as a "forward pass." Further, the method includes using the output maps 430, to determine how close or far off of the expected character recognition and the CNN was, as shown at block 615 [wherein the system performs no backward propagation through the neural network] (i.e. 610 no backward propagation is done).
Regarding claim 18 and analogous claim 27, Gokmen in view of Strachan and Payvand teach the system of claim 16 and analogous 25.
Gokman, Strachan and Payvand are combined with the same rationale used in claim 16 and analogous claim 25.
Payvand teaches wherein the system continuously operates in the inference mode until an asynchronous error event occurs (Payvand Figure 2,
PNG
media_image7.png
580
1247
media_image7.png
Greyscale
[wherein the system continuously operates in the inference mode]
D. Inference and Learning Circuit para 2 line 1-16, For every neuron, different voltages (corresponding to Pj ) are applied to the top electrode of the corresponding memristive device whose bottom electrode is pinned by the crossbar front-end highlighted in yellow (Fig. 2). This block pins the entire column to a reference voltage (V ref) and reads out the sum of the currents generated by the application of Ps across the memristors in the column. As a result, a voltage is developed on the gate of the M1 connected to a differential pair which re-normalizes the sum of the currents from the crossbar to Inorm. This ensures that the currents remain in the subthreshold regime for the next-stage of the computation which is the ternary error generation as is specified in equation (4). This is done through the Variable Width Bump (VWBump) circuit that compares Inn to the target ^y, with a stop region. Thus, the VWBump circuit output indicates the sign of the weight update (up or down) or stop-learning (no update) [until an asynchronous error event occurs]).
Regarding claim 19 and analogous claim 28, Gokmen in view of Strachan and Payvand teach the system of claim 16 and analogous 25.
Gokman, Strachan and Payvand are combined with the same rationale used in claim 16 and analogous claim 25.
Payvand teaches wherein the error events are locally generated on a per-row basis based on the output signals generated by the neural circuits (Payvan Page 2,
PNG
media_image8.png
330
616
media_image8.png
Greyscale
Page 3 D. Inference and Learning Circuits para 3, The Up and Down signals trigger the oscillators highlighted in blue which generate the bipolar Ei events. According to Eq. (5), the magnitude of the weight update is Pj , and thus Pj must be sampled at the onset of Ei. To do so, we regenerate the exponential current in the entire row by propagating pbias shown in the DPI circuit block (green) and sample it by the up and down events. This is done through the sampling circuit (shown in purple) whose core consists of two PMOS transistors in series connected to the up/down events and pbias respectively. An NMOS transistor is biased to generate a current much smaller than that of the DPI and as a result, the higher the DPI current, the higher the input of the following inverter during the event pulse, and thus it takes longer for the NMOS to discharge that node. This results in a pulse width which varies linearly with Pj , in agreement with Eq. (5). The linear pulse width can be approximated with multiple pulses which results in a linear conductance update (with a soft bound) in memristive devices [20] [based on the output signals generated by the neural circuits].
Fig. 2,
PNG
media_image9.png
466
1100
media_image9.png
Greyscale
[wherein the error events are locally generated on a per-row basis]).
Regarding claim 20 and analogous claim 29, Gokmen in view of Strachan and Payvand teach the system of claim 16 and analogous 25.
Gokman, Strachan and Payvand are combined with the same rationale used in claim 16 and analogous claim 25.
Strachan teaches wherein the synaptic weight update signals are only applied to the rows for which the calculated error exceeds the error threshold (Strachan Para 0034 line 15-24, As an addition or alternative, the implementation logic 218 can communicate process control input 223 to initiate a remedial process to correct for error values that are significant. For example, the implementation logic 218 can cause the process controller 204 to determine, or otherwise identify weights, or initiate a training process to determine weights that correct for individual nodes 208 of respective memory array components 202 which are associated with error values that are deemed significant.
para 0036, FIG. 3 illustrates an example method for determining the significance of error values in a multi-layered process that utilizes analog memory arrays. In describing an example of FIG. 3, reference may be made to elements of FIG. 2 for purpose of illustrating suitable components for performing a step or sub-step being described.
Para 0040, line 7-15, By way of example, the remedial operation(s) can include the computer system 100 performing an operation to indicate that a portion of a result is suspect, based on the determination that error values of the error array 215 of a particular memory array component 202 are significant. As an addition or variation, weights can be learned or applied to nodes of individual array data structures which correlate to nodes ( or cells) of the respective array data structure that are deemed significant [weight update signals are only applied to the rows for which the calculated error exceeds the error threshold]).
Regarding claim 21 and analogous claim 30, Gokmen in view of Strachan and Payvand teach the system of claim 16 and analogous 25.
Gokman, Strachan and Payvand are combined with the same rationale used in claim 16 and analogous claim 25.
Payvand teaches wherein the synaptic weight update signals are applied together to all of the memristive devices of the rows for which the calculated error exceeds the error threshold (Figure 1,
PNG
media_image10.png
301
569
media_image10.png
Greyscale
[wherein the synaptic weight update signals]
Page 4 Figure 2, Figure 2. Details of the architecture and learning circuits. Green: DPI circuit generating P in the current form. Red: Pseudo resistors converting input current into a voltage driving the crossbar array. Pink: Synapse with the controlling switches. Purple: Sampling circuitry generating pulses to program the devices. Yellow: Crossbar front-end and normalization of the crossbar current. Dark blue: Bump circuit comparing the crossbar current to a target and generating the direction of the error. Light blue: Bidirectional neuron producing up and down events [memristive devices of the rows for which the calculated error exceeds the error threshold.]).
Claim 22-24 and 31-33 is rejected under 35 U.S.C. 103 as being unpatentable over Gokmen in view of Strachan and Payvand and further in view of Li, Zhiwei, et al. "Design of ternary neural network with 3-D vertical RRAM array." IEEE Transactions on Electron Devices 64.6 (2017): 2721-2727 (“Li”).
Regarding claim 22 and analogous claim 31, Gokmen in view of Strachan and Payvand teach the system of claim 16 and analogous 25.
Gokman, Strachan and Payvand are combined with the same rationale used in claim 16 and analogous claim 25.
Grokman does not explicitly teach wherein only a single row is updated at any point in time.
However Li teaches wherein only a single row is updated at any point in time (Li page 2723, B. Adaptation of the 2-d Cross-Point Array para 3, line 1-8, For weight updates, illustrated in Fig. 2(b) and (c), we use read before write and row-by-row write scheme to update the conductance of RRAM cells. For read operation to calculatethe weight update amount (ΔW), the WLs connected to the selected cell are biased to Vr, while all the other WLs and BLs are biased to 0, and the current of the selected cells are read out by the current sense amplifier and the ΔW is calculated by the peripheral neuron circuits For write operation, the WLs and BLs connected to the selected cell are biased to VW and 0, respectively, while all the unselected lines are applied with VW/2. Then the new W is written back to the array [wherein only a single row is updated at any point in time].).
Gokmen and Li are both considered are both considered to be analogous to the claimed invention of machine learning. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Gokmen to incorporate the teachings of Li and disclose only updating a single row at a time. Doing so to update the weights row by row and calculate the update amount(Li page 2723, B. Adaptation of the 2-d Cross-Point Array para 3, line 1-8, For weight updates, illustrated in Fig. 2(b) and (c), we use read before write and row-by-row write scheme to update the conductance of RRAM cells. For read operation to calculatethe weight update amount (ΔW), the WLs connected to the selected cell are biased to Vr, while all the other WLs and BLs are biased to 0, and the current of the selected cells are read out by the current sense amplifier and the ΔW is calculated by the peripheral neuron circuits For write operation, the WLs and BLs connected to the selected cell are biased to VW and 0, respectively, while all the unselected lines are applied with VW/2. Then the new W is written back to the array).
Regarding claim 23 and analogous claim 32, Gokmen in view of Strachan and Payvand teach the system of claim 22 and analogous 31.
Gokman, Strachan and Payvand are combined with the same rationale used in claim 16 and analogous claim 25.
Gokman and Li are combined with the same rationale used in claim 22 and analogous claim 31.
Li teaches wherein the weight update circuitry comprises arbitration circuitry configured to select one row at a time to update (Li
Page 2723 Fig. 3,
PNG
media_image11.png
348
484
media_image11.png
Greyscale
Page 2723 A. 3-D V-RRAM Architecture para 1 line1-5, The schematic of 3-D V-RRAM comprises RRAM memory cells, vertical transistors, k word planes (WLs), m select lines (SLs), and n bit lines (BLs) [23], as illustrated in Fig. 3. The WLs, the BLs, and the SLs are used to decode the 3-D array [wherein the weight update circuitry comprises arbitration circuitry].
B. Adaptation to the TNN para 3 line 1-5, In the weights update mode, read before write scheme is used and the 3-D V-RRAM operates as the traditional memory. For the read operation, we follow the “read-in-a-row” scheme: the WL voltage of the selected layer is biased to Vr , while other WLs and BLs are grounded, as shown in Fig. 4(b) [configured to select one row at a time to update]).
Regarding claim 24 and analogous claim 33, Gokmen in view of Strachan and Payvand teach the system of claim 16 and analogous 25.
Gokman, Strachan and Payvand are combined with the same rationale used in claim 16 and analogous claim 25.
Gokman and Li are combined with the same rationale used in claim 22 and analogous claim 31.
Li teaches wherein the synaptic weight updates of the synaptic weight update signals are ternary weight updates having a sign of -1, 0, or 1 (Li page 2 I. Introduction para. 4, The remaining sections are organized as follows: In the Section II, we introduce a TNN case study that we compress the multilayer neural network (MLP) to (+1, 0,−1) ternary weights for feedforward/backward inference, and then describe how we map such neural network to the 2-D RRAM array architecture. Section III explains 3-D V-RRAM architecture and the proposed operation scheme to implement TNN [ternary weight updates having a sign of -1, 0, or 1]. Section IV shows the comparison results between 2-D and 3-D implementations including the design space exploration on write/read margin, and the performance evaluation on area/latency/energy.
page 2723, B. Adaptation of the 2-d Cross-Point Array para 3, line 1-8, For weight updates, illustrated in Fig. 2(b) and (c), we use read before write and row-by-row write scheme to update the conductance of RRAM cells. For read operation to calculatethe weight update amount (ΔW), the WLs connected to the selected cell are biased to Vr, while all the other WLs and BLs are biased to 0, and the current of the selected cells are read out by the current sense amplifier and the ΔW is calculated by the peripheral neuron circuits For write operation, the WLs and BLs connected to the selected cell are biased to VW and 0, respectively, while all the unselected lines are applied with VW/2. Then the new W is written back to the array [wherein the synaptic weight updates of the synaptic weight update signals].).
Pertinent Prior Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Malaya (US20190005377A1) – teaches a ANN receiving data and inferencing until a difference threshold or timing indicates that inference time exceeded the timing threshold.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALFREDO CAMPOS whose telephone number is (571)272-4504. The examiner can normally be reached 7:00 - 4:00 pm M - F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael J. Huntley can be reached at (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ALFREDO CAMPOS/Examiner, Art Unit 2129
/MICHAEL J HUNTLEY/Supervisory Patent Examiner, Art Unit 2129
Prosecution Insights