DETAILED ACTION
This action is responsive to the application filed on 02/12/2026. Claims 1, 3-21 are pending and have been examined. This action is Non-final.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Applicant’s claim for the benefit of a prior-filed application under 35 U.S.C. 119(e) or under 35 U.S.C.
120, 121, 365(c), or 386(c) is acknowledged.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 02/12/2026 has been entered.
Response to Arguments
Argument 1: Applicant argues on pages 9-14 of the Remarks that claims 7 and 15 are patent eligible. Applicant first argues under Step 2A Prong One that claim 7 does not recite an abstract idea, but at most merely involves or is based on an abstract idea, and therefore should be eligible under pathway B. Applicant further argues under Step 2A Prong Two that, even if an abstract idea is recited, the claimed elements integrate the alleged exception into a practical application because the claims address a technical problem in computing-in-memory and artificial-intelligence systems, namely accuracy degradation caused by drifted weights in non-volatile memory devices. Applicant relies on Ex parte Desjardins, Enfish, and McRO to argue that the claims improve the functioning of a memory device and neural-network technology rather than merely implementing an abstract idea on generic computer components. Applicant also argues under Step 2B that the claimed combination is not well-understood, routine, and conventional, or at least amounts to an inventive concept when considered as an ordered combination, because enabling a second memory block with backup neurons and using a second model formed by original and backup neurons improves inference accuracy and consistency in view of memory-device aging.
Response to Argument 1: The examiner has considered the argument set forth above, however, the argument is not persuasive. As set forth in the rejection, claim 7 recites “determining whether the aging condition meets a predetermined aging condition and drifting of weights stored in the first memory block,” and claim 15 recites substantially similar controller functionality. This determining step remains directed to evaluating whether a condition meets a predetermined condition and whether stored weights are drifted, which can be performed using observation, judgment, and evaluation, and is therefore properly characterized as a mental process under Step 2A Prong 1. Applicant’s reliance on the claims being directed to a memory system does not remove the abstract idea from the claims because the additional elements merely obtain an aging condition, store neuron or weight information in memory blocks, determine whether aging and drifted-weight conditions are met, and enable or select a second memory block for computation. These limitations are recited at a high level of generality and do not set forth a particular memory architecture, circuit configuration, drift-detection circuit, read/write technique, or other specific technological improvement to the operation of the memory hardware itself. Rather, the claims use conventional memory blocks and a controller as tools to gather condition data, evaluate that data, and select between stored neuron/model resources. Applicant’s reliance on Ex parte Desjardins, Enfish, and McRO is not persuasive because the claims here do not recite a specific model-training improvement, specific rule-based automation, or specific data-structure/computer-functionality improvement comparable to the claims found eligible in those cases. Instead, the claims broadly recite evaluating aging/drift conditions and enabling backup neuron storage resources without specifying how the memory-device drift is detected, how the drifted weights are technically corrected at the hardware level, or how the claimed memory structure itself is improved. Further, under Step 2B, obtaining condition data, storing model or weight information, transferring stored weight information, and enabling or selecting memory blocks for computation are well-understood, routine, and conventional computer and memory operations. Considering the claim elements individually and as an ordered combination does not add significantly more because the ordered combination merely applies the abstract determination using generic memory-system components. Accordingly, the rejection of claims 7 and 15, and the claims depending therefrom, under 35 U.S.C. 101 is maintained.
Argument 2: Applicant argues on pages 14-17 of the Remarks that the cited prior art fails to teach or suggest the amended limitations of independent claims 1, 7, and 15. For claim 1, Applicant argues that neither Wiezbicki nor Zhang discloses converting original weights of original neurons into drifted weights according to a drifting table recording a relationship between a weight stored in a memory cell and an aging condition of the memory cell. Applicant contends that Wiezbicki’s cited equations and weighting functions relate to compensating sensor drift or external environmental drift in input data, not drift occurring on a memory device, and that Zhang does not cure this deficiency because Zhang allegedly does not disclose converting weights to reflect aging of the memory device. For claim 7, Applicant argues that Wiezbicki, Zhang, and Palmer-Brown, either individually or in combination, do not disclose determining whether an aging condition meets a predetermined aging condition and whether weights stored in the first memory block are drifted, nor do they disclose enabling a second memory block storing a backup neuron and using a second model formed by original neurons and the backup neuron after the aging and drifted-weight conditions are met. For claim 15, Applicant argues that the claim recites similar amended subject matter to claim 7 in system form and is likewise not disclosed by the cited combination. Applicant further argues that dependent claims 3-6, 8-14, and 16-21 are patentable because they depend from allegedly allowable independent claims and therefore overcome the obviousness rejections.
Response to Argument 2: The examiner has considered the argument set forth above, however, the argument is not persuasive. Applicant’s argument does not overcome the rejection because it does not address the specific manner in which the references are combined in the updated rejection. Wiezbicki is relied upon for training and using neural-network models in the context of drift detection and drift compensation over time, including weighting functions and adaptation in response to detected drift. Zhang is relied upon to supply the memory-device implementation and the amended memory-cell/drifted-weight features. In particular, Zhang teaches that non-volatile memristors allow synapse weights to be stored in-situ, that memristor crossbar arrays are vulnerable to resistance drift, temperature variations, and stuck-at-fault defects, and that an effective matrix is realized when a weight matrix is mapped to a memristor crossbar array with stuck-at-faults, such that the effective weight value depends on the condition of the corresponding memristor. Thus, Zhang teaches or suggests the relationship between stored neural-network weight values and memory-cell conditions that cause the realized or effective weight to differ from the intended or original weight. Zhang further teaches inserting redundant neurons to surgically repair neurons connected to rows and columns in MCAs with many stuck-at-faults, which supports the claimed backup neuron/second memory block functionality. Palmer-Brown is additionally relied upon for using snap-drift learning and combined original/adapted neural-network parameters for continued computation. Therefore, the amended language does not overcome the rejection because the combination of Wiezbicki, Zhang, and Palmer-Brown teaches or at least suggests determining that stored neural-network weights have become drifted or effective values due to memory-device conditions, and enabling redundant or backup neuron resources for continued computation. It would have been obvious to a person of ordinary skill in the art to combine Wiezbicki’s drift-aware neural-network operation with Zhang’s memristor-based DNN redundancy techniques to maintain classification accuracy and fault tolerance when neural-network weights stored in memory are affected by device drift, temperature variation, or stuck-at-fault conditions. Accordingly, the rejection of independent claims 1, 7, and 15, and the claims depending therefrom, under 35 U.S.C. 103 is maintained.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition
of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the
conditions and requirements of this title.
Claims 7-21 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claim 7,
Step 1: The claim is directed to a method, which falls under the category of process. The claim satisfies step 1.
Step 2A Prong 1:
(a) “determining whether the aging condition meets a predetermined aging condition and drifting of weights stored in the first memory block;” -- The limitation is directed to determining if a condition meets a predetermined condition and whether weights stored in the first memory block are drifted. The limitation is directed to a process that can be performed in the human mind using evaluation, observation, and judgement, and thus it is directed to a mental process.
Step 2A Prong 2 and Step 2B:
(a) “An operating method of a memory system, comprising: using a first model formed by a plurality of original neurons stored in a first memory block of the memory system for computation; … and when it is determined that the aging condition meets the predetermined aging condition and that the weights stored in the first memory block are drifted, enabling a second memory block storing at least one backup neuron and using a second model formed by the original neurons and the at least one backup neuron stored in the first memory block and the second memory block for computation.” -- The limitation recites using a first memory block that stores a first model formed by original neurons for computation, determining whether an aging condition and drifted-weight condition are met, and, after the conditions are met, (like the aging condition and stored weights are drifted), enabling a second memory block that stores at least one backup neuron and using a second model formed by the original neurons and the backup neuron(s) across the first and second memory blocks for computation. These steps amount to nothing more than using generic memory blocks and selecting between different sets of neuron weights/models for computation based on evaluated aging and drift conditions. They are recited at a high level of generality and do not specify any particular improvement to the underlying memory hardware or computer technology (e.g., no new memory architecture, circuit configuration, drift-detection circuit, or read/write technique). As such, they are mere instructions to apply the abstract idea using conventional memory blocks and cannot be integrated into a practical application, nor can they provide significantly more than the judicial exception (see MPEP 2106.05(f)).
(b) “obtaining an aging condition of the memory system” -- The limitation recites obtaining (gathering data) of the memory system, which is considered an insignificant, extra-solution activity that cannot be integrated to a practical application (see MPEP 2106.05(g)). Furthermore, under Step 2B, obtaining conditional data from a system is a well-understood, routine, and conventional activity (WURC), that cannot provide significantly more than the judicial exception (see MPEP 2106.05(d)(II)).
Therefore, claim 7 is non-patent eligible.
Regarding claim 8,
Step 1: The claim is directed to a method, which falls under the category of process. The claim satisfies step 1.
Step 2A Prong 1:
“The operating method of claim 7, wherein the aging condition meets the predetermined aging condition when an operating time of the memory system is greater than or equal to a predetermined time, or an operating temperature of the memory system is greater than or equal to a predetermined temperature.” -- The limitation is directed to meeting a predetermined aging condition when a time is greater than the a predetermined time or if a temperature of the system is greater than a predetermined temperate. The limitation is directed to a process that can be completed in the human mind using evaluation, observation, and judgement, and thus the limitation is directed to a mental process.
There are no elements to be evaluated under Step 2A Prong 2B.
Therefore, claim 8 is non-patent eligible.
Regarding claim 9,
Step 1: The claim is directed to a method, which falls under the category of process. The claim satisfies step 1.
There are no elements to be evaluated under Step 2A Prong 1.
Step 2A Prong 2 and Step 2B:
“The operating method of claim 7, wherein the first memory block is programmed to store original weights of the original neurons of the first model,” - The limitation recites programming a memory block to store weight values of the original neurons of the first model. The limitation amounts to no more than programming to store data, which is an insignificant, extra-solution activity that cannot be integrated to a practical application (see MPEP 2106.05(g)). Furthermore, under Step 2B, storing data to be retrieved is a well-understood, routine, and conventional activity (WURC) that cannot provide significantly more than the judicial exception (see MPEP 2106.05(d)(II)).
“wherein the first memory block is used for performing computation as the first model before it is determined that the aging condition meets the predetermined aging condition.” - The limitation recites that the memory block will be used to perform computation before it has been determined and evaluated that the condition has been met. The limitation does not amount to no more than mere instructions of just applying a memory block to perform computation, and it cannot be integrated to a practical application, nor cannot provide significantly more than the judicial exception (see MPEP 2106.05(f)).
Therefore, claim 9 is non-patent eligible.
Regarding claim 10,
Step 1: The claim is directed to a method, which falls under the category of process. The claim satisfies step 1.
There are no elements to be evaluated under Step 2A Prong 1.
Step 2A Prong 2 and Step 2B:
“The operating method of claim 9, wherein when it is determined that the aging condition meets the predetermined aging condition, the original weights of the original neurons stored in the first memory block are transferred as drifted weights.”—The limitation recites that once it the aging condition has been determined to meet the predetermined condition threshold, it will transfer the stored weight values as drifted weights. Transferring gathered data is an insignificant, extra-solution activity that cannot be integrated to a practical application (see MPEP 2106.05(g)). Furthermore, under Step 2B, the act of transmitting data over a network is a well-understood, routine, and conventional activity (WURC) that cannot provide significantly more than the judicial exception (see MPEP 2106.05(d)(II)).
Therefore, claim 10 is non-patent eligible.
Regarding claim 11,
Step 1: The claim is directed to a method, which falls under the category of process. The claim satisfies step 1.
There are no elements to be evaluated under Step 2A Prong 1.
Step 2A Prong 2 and Step 2B:
“The operating method of claim 10, wherein the second memory block is programmed to store predicted weights of the at least one backup neuron” - The limitation recites a memory block storing predicted weights of the backup neuron. Merely storing gathered data (predicted weight) is an insignificant, extra-solution activity that cannot be integrated to a practical application (see MPEP 2106.05(g)). Furthermore, under Step 2B, storing gathered data is also considered a well-understood, routine, and conventional activity (WURC) that cannot provide significantly more than the judicial exception (see MPEP 2106.05(d)(II)).
Therefore, claim 11 is non-patent eligible.
Regarding claim 12,
Step 1: The claim is directed to a method, which falls under the category of process. The claim satisfies step 1.
There are no elements to be evaluated under Step 2A Prong 1.
Step 2A Prong 2 and Step 2B:
“The operating method of claim 11, wherein when it is determined that the aging condition meets the predetermined aging condition, the predicted weights of the at least one backup neuron is transferred as backup weights of the second model.” - The limitation recites that once it the aging condition has been determined to meet the predetermined condition threshold, the predicted weights will be transferred as backup weights for the second model. Transferring gathered data is an insignificant, extra-solution activity that cannot be integrated to a practical application (see MPEP 2106.05(g)). Furthermore, under Step 2B, the act of transmitting data over a network is a well-understood, routine, and conventional activity (WURC) that cannot provide significantly more than the judicial exception (see MPEP 2106.05(d)(II)).
Therefore, claim 12 is non-patent eligible.
Regarding claim 13,
Step 1: The claim is directed to a method, which falls under the category of process. The claim satisfies step 1.
There are no elements to be evaluated under Step 2A Prong 1.
Step 2A Prong 2 and Step 2B:
“The operating method of claim 12, wherein when it is determined that the aging condition meets the predetermined aging condition, the second memory block is enabled and the at least one backup neuron is added to the first model to generate the second model,” - The limitation recites that when the aging condition has met the predetermined aging condition, the second memory block will then be enabled and the backup neuron will then be added to the model to generate another model. The limitation does not amount to no more than merely instructing that data will be enabled and the backup neuron data applied to the second model, and thus it cannot be integrated to a practical application, nor can it provide significantly more than the judicial exception (see MPEP 2106.05(f)).
“wherein the first memory block storing the drifted weights of the original neurons and the second memory block storing the backup weights of the at least one backup neuron are used for performing computation as the second model.” - The limitation recites that the stored drift and backup weights for both the original and backup neural will be used to perform computation for the second model. The limitation amounts to no more than merely storing data to be used and manipulated for further computation, which is an insignificant, extra-solution activity that cannot be integrated to a practical application (see MPEP 2106.05(g)). Furthermore, under Step 2B, storing data to be retrieved is a well-understood, routine, and conventional activity (WURC) that cannot provide significantly more than the judicial exception (see MPEP 2106.05(d)(II)).
Therefore, claim 13 is non-patent eligible.
Regarding claim 14,
Step 1: The claim is directed to a method, which falls under the category of process. The claim satisfies step 1.
There are no elements to be evaluated under Step 2A Prong 1.
Step 2A Prong 2 and Step 2B:
“The operating method of claim 8, wherein when the operating temperature returns to be less than the predetermined temperature, the second memory block is disabled and the first memory block is used for computation.” - The limitation recites that when the operating temperature becomes less than the predetermined temp, the second memory block will be disabled and will use the first memory block for computation. The limitation is merely stating that once a predetermine temperature (threshold) has been met, memory will be instructed to be disabled and that the first memory block will be applied to be used for computation purposes, which cannot be integrated to a practical application, nor provide significantly more than the judicial exception (see MPEP 2106.05(f)).
Therefore, claim 14 is non-patent eligible.
Regarding claim 15,
Step 1: The claim is directed to a system, which falls under the category of machine. The claim satisfies step 1.
Step 2A Prong 1:
(a) “determine whether the aging condition meets a predetermined aging condition and drifting of weights stored in the first memory block;” - The limitation is directed to determining when a condition meets a predetermined condition and whether weights stored in the first memory block are drifted. The limitation is directed to a task that can be completed using evaluation, judgment, and observation, and therefore the limitation is directed to a mental process.
Step 2A Prong 2 and Step 2B:
(a) “A memory system, comprising: a memory array, comprising: a first memory block…and a second memory block; and a controller, coupled to the memory array, wherein the controller is configured to;” -- The limitation recites that a memory system comprises memory blocks and a controller coupled to a memory array, for which the controller is configured to perform the other tasks. This limitation amounts to no more than mere instructions on what will be applied to the memory system (computer) and it cannot be integrated to a practical application, nor can it provide significantly more than the judicial exception (see MPEP 2106.05(f)).
(b) “configured to store a plurality of original neurons…configured to store at least one backup neuron;… the original neurons and the at least one backup neuron stored in the first memory block”-- The limitation recites configuring the memory blocks to store the plurality of original neurons and at least one backup neuron, as well as the neurons being stored in the first memory block. The limitation is directed to an insignificant, extra-solution activity that cannot be integrated to a practical application (see MPEP 2106.05(g)). Furthermore, under Step 2B, the act of storing gathered data/information to memory is a well-understood, routine, and conventional activity (WURC) that cannot provide significantly more than the judicial exception (see MPEP 2106.05(d)(II)).
(c) “obtain an aging condition of the memory system;” - The limitation is directed to merely obtaining an aging condition (gathering data) of the memory system, which is directed to an insignificant extra-solution activity that cannot be integrated to a practical application (see MPEP 2106.05(g)). Furthermore, under Step 2B, gathering data to be manipulated is a well-understood, routine, and conventional activity that cannot provide significantly more than the judicial exception (see MPEP 2106.05(d)(II)).
(d) “when it is determined that the aging condition meets the predetermined aging condition and that the weights stored in the first memory block are drifted, the second memory block is enabled…and the second memory block are used for computation” -- The limitation recites that when the aging condition meets the predetermined aging condition and the weights stored in the first memory block are drifted, the second memory block will then be enabled and the first and second memory blocks will be used for computation. These steps amount to mere instructions to implement the abstract idea onto a computer by evaluating aging and drifted-weight conditions and selecting or enabling stored neuron resources for computation, and cannot be integrated to a practical application, nor provide significantly more than the judicial exception (see MPEP 2106.05(f)).
Therefore, claim 15 is non-patent eligible.
Regarding claim 16,
Step 1: The claim is directed to a system, which falls under the category of machine. The claim satisfies step 1.
Step 2A Prong 1:
“The memory system of claim 15, wherein the controller determines that the aging condition meets the predetermined aging condition when an operating time of the memory system is greater than or equal to a predetermined time, or an operating temperature of the memory system is greater than or equal to a predetermined temperature” - The limitation is directed to a controller determining if an aging condition meets the predetermined condition when the operating time of the memory system is greater than or equal to the predetermined time and if the operating temperature is greater than or equal to the predetermined temperature. The process of comparing a value to a predetermined threshold (time with predetermined time and temperature with predetermined temperature) can be performed in the human mind using evaluation, observation and judgement, and therefore the limitation is directed to a process.
There are no elements to be evaluated under Step 2A Prong 2 and Step 2B.
Therefore, claim 16 is non-patent eligible.
Regarding claim 17,
Step 1: The claim is directed to a system, which falls under the category of machine. The claim satisfies step 1.
There are no elements to be evaluated under Step 2A Prong 1.
Step 2A Prong 2 and Step 2B:
“The memory system of claim 16, wherein the first memory block is programmed to store original weights of original neurons of a first model, the second memory block is programmed to store predicted weights of at least one backup neuron” - The limitation recites a memory block storing predicted weights of the backup neuron and another memory block storing the original neuron. Merely storing gathered data (predicted weight) is an insignificant, extra-solution activity that cannot be integrated to a practical application (see MPEP 2106.05(g)). Furthermore, under Step 2B, storing gathered data is also considered a well-understood, routine, and conventional activity (WURC) that cannot provide significantly more than the judicial exception (see MPEP 2106.05(d)(II)).
Therefore, claim 17 is non-patent eligible.
Regarding claim 18,
Step 1: The claim is directed to a system, which falls under the category of machine. The claim satisfies step 1.
There are no elements to be evaluated under Step 2A Prong 1.
Step 2A Prong 2 and Step 2B:
“The memory system of claim 17, wherein when it is determined by the controller that the aging condition meets the predetermined aging condition, the original weights of the original neurons stored in the first memory block are transferred as drifted weights and the predicted weights of the at least one backup neuron is transferred as backup weights of a second model.” - The limitation recites that once the controller determines that the aging condition meets the predetermined aging condition, the original weights of the original neurons will be transferred as drifted weights and the predicted weights will be transferred as backup weights for the second model. The limitation is directed to transferring gathered data, which is an insignificant, extra-solution activity that cannot be integrated to a practical application (see MPEP 2106.05(g)). Furthermore, under Step 2B, the act of transmitting data over a network is a well-understood, routine, and conventional activity that cannot provide significantly more than the judicial exception (see MPEP 2106.05(d)(II)).
Therefore, claim 18 is non-patent eligible.
Regarding claim 19,
Step 1: The claim is directed to a system, which falls under the category of machine. The claim satisfies step 1.
There are no elements to be evaluated under Step 2A Prong 1.
Step 2A Prong 2 and Step 2B:
“The memory system of claim 17, wherein when it is determined by the controller that the aging condition meets the predetermined aging condition, the second memory block is enabled and the at least one backup neuron is added to the first model to generate the second model, wherein the first memory block having the original neurons with the drifted weights and the second memory block having the at least one backup neuron with the backup weights are used for performing computation as the second model.” - The limitation recites that once the controller determines that the aging condition meets the predetermined aging condition, the second memory block is enabled and the backup neuron will be added (applied) onto the first model to generate another model, and the limitation goes on to recite further components that will be executed for performing computation, which cannot be integrated to a practical application, nor can it provide significantly more than the judicial exception (see MPEP 2106.05(f)).
Therefore, claim 19 is non-patent eligible.
Regarding claim 20,
Step 1: The claim is directed to a system, which falls under the category of machine. The claim satisfies step 1.
There are no elements to be evaluated under Step 2A Prong 1.
Step 2A Prong 2 and Step 2B:
“The memory system of claim 19, wherein the at least one backup neuron is added to at least one arbitrary layer rather than a last layer of the first model to generate the second model.” - The limitation recites that at least one neuron will be added (applied) to an arbitrary computing layer rather than the last layer of the first model to generate a second model. The limitation does not amount to no more than mere instructions to add (apply) onto the neural network, and thus it cannot be integrated to a practical application, nor provide significantly more than the judicial exception (see MPEP 2106.05(f)).
Therefore, claim 20 is non-patent eligible.
Regarding claim 21,
Step 1: The claim is directed to a method, which falls under the category of process. The claim satisfies step 1.
There are no elements to be evaluated under Step 2A Prong 1.
Step 2A Prong 2 and Step 2B:
“The training method of claim 1, wherein a conversion of the original neurons from the original weights to the drifted weights corresponds to the predetermined aging condition.” -- The limitation recites additional limitation that merely specifies that the already-recited conversion from original weights to drifted weights is performed such that it “corresponds to the predetermined aging condition.” The limitation is merely limiting the limitations to a field of use/environment, and does not integrate to a practical application, nor provides significantly more than the judicial exception (see MPEP 2106.05(h)).
Therefore, claim 21 is non-patent eligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this
Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not
identically disclosed as set forth in section 102, if the differences between the claimed invention and the
prior art are such that the claimed invention as a whole would have been obvious before the effective filing
date of the claimed invention to a person having ordinary skill in the art to which the claimed invention
pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are
summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1 and 21 is rejected under 35 U.S.C. 103 as being unpatentable over NPL reference “Sensor Drift Compensation using Weighted Neural Networks” by Wiezbicki et. al (referred herein as Wiezbicki) in view of NPL reference “Redundant Neurons and Shared Redundant Synapses for Robust Memristor-based DNNs with Reduced Overhead” by Zhang et al (referred herein as Zhang).
Regarding claim 1, Wiezbicki teaches:
A training method for training a neural network implemented on a memory device, the training method comprising: training the neural network by using an input dataset to obtain a first model formed by a plurality of original neurons; ([Wiezbicki, page 93, sec 3] “Five multilayer perceptron neural networks (MLP ANN) were trained for each of 10 batches available. Each neural network had 128 input neurons representing each numerical characteristic that are derived from basic characteristics for each gas to be classified”, wherein the examiner interprets the description of training multilayer perceptron neural networks with batches of data, each network having input neurons representing characteristics, to be the same as “training the neural network by using an input dataset to obtain a first model formed by a plurality of original neurons” because they are both directed to using input data to train a neural network whose neurons and weights define a first trained model.)
converting original weights of the original neurons in the neural network into drifted weights according to a drifting table; ([Wiezbicki, page 93, sec 2] “The last characteristics are obtained according to:
PNG
media_image1.png
34
361
media_image1.png
Greyscale
using alpha values 0.1, 0.01, 0.001. With this method, a total of 128 characteristics for each gas label to be classified are obtained” and [page 94, sec 3] “The following weighting functions were used:
PNG
media_image2.png
169
360
media_image2.png
Greyscale
… each expression is based on the weighted value to be given to each batch that will compose the C₂ classification”, wherein the examiner interprets the use of explicit equations and weighting functions that transform numeric values over time to be the same as “converting original weights of the original neurons in the neural network into drifted weights according to a drifting table” because they are both directed to systematically changing stored numeric parameters (weights or characteristics) according to predetermined mathematical functions or tables (represented as an entire set of possible weighting equations) to represent drifted weights)
and training the updated neural network by using the input dataset to obtain a second model formed by the original neurons [and the at least one backup neuron]. ([Wiezbicki, page 93] “For initial use, to train the neural networks, a pre-processing is required. Normalization to a scale of -1 to +1 is performed” and [page 94-95, sec 3-Algorithm 1] “For t = 1 to sizeOf(batchList) do … train ANNₜ,ₙ … For each test of batcht, and consequently each of the four functions represented by c, neural networks trained with all previous batches are used…
PNG
media_image3.png
142
794
media_image3.png
Greyscale
wherein the examiner interprets repeated training and use of multiple neural networks (including networks trained with different weighting or drift conditions) to be the same as “training the updated neural network by using the input dataset to obtain a second model formed by the original neurons” because they are both directed to retraining or reusing the network configuration, now including additional neuron resources, on input data to produce a subsequent trained model that incorporates neuron functionality.)
Wiezbicki does not teach and the at least one backup neuron…adding at least one backup neuron to the neural network to generate an updated neural network, wherein the at least one backup neuron corresponds to a predetermined aging condition of the memory device;.
Zhang teaches:
recording a relationship between a weight stored in a memory cell and an aging condition of the memory cell; ([Zhang, page 1] “the non-volatile memristors allow the synapse weights to be stored in-situ [3, 12], which circumvents the communication bottleneck of the von-Neumann architecture [16].” AND [Zhang, page 1] “A challenge to computing paradigm is that MCAs are vulnerable to variations as resistance drift, random telegraph noise, temperature variations, and stuck-at-fault defects.” AND [Zhang, page 2] “Let W e be the effective matrix realized when a weight matrix W is mapped to an MCA with stuck-at-faults … Each element w e k in W e is equal to wk in W if wk is mapped to a memristor without a stuck-at-fault defect. w e k is equal to wmin or wmax if wk is mapped to a memristor stuck-off or stuck-on, respectively.”, wherein the examiner interprets “the non-volatile memristors allow the synapse weights to be stored in-situ” to be the same as “a weight stored in a memory cell” because they are both directed to storing neural-network synapse weight values in memory devices. The examiner further interprets “MCAs are vulnerable to variations as resistance drift, random telegraph noise, temperature variations, and stuck-at-fault defects” to be the same as “an aging condition of the memory cell” because they are both directed to physical condition changes or degradation conditions of memory cells that affect the behavior of the stored weight. The examiner further interprets “the effective matrix realized when a weight matrix W is mapped to an MCA with stuck-at-faults” and the teaching that each effective weight value depends on whether the corresponding memristor is without a stuck-at-fault defect, stuck-off, or stuck-on to be the same as “recording a relationship between a weight stored in a memory cell and an aging condition of the memory cell” because they are both directed to defining a relationship between a stored neural-network weight and a memory-cell condition that determines the effective or drifted weight value used by the neural network).
adding at least one backup neuron to the neural network to generate an updated neural network, wherein the at least one backup neuron corresponds to a predetermined aging condition of the memory device; ([Zhang, page 1] “A key challenge is that stuck-at-fault defects may degrade the classification accuracy of the memristor-based DNNs. A common technique to reduce the negative impact of stuck-at-faults is to utilize redundant synapses, i.e., each row in a weight matrix is realized using two (or r) parallel rows in an MCA. In this paper, we propose to handle stuck-at-faults by inserting redundant neurons and by sharing redundant synapses. The first technique is based on inserting redundant neurons to surgically repair neurons connected to rows and columns in the MCAs with many stuck-at-faults” AND [Zhang, page 3] “The redundant neuron t is inserted by replicating all the incoming and outgoing synapses to the neuron s using the same synapse weights, which is illustrated in Figure 2(b). Moreover, the activation function of the neurons s and t are updated to σ()u, where σ(.)s is the original activation function of neuron s and σ(.)u = σ(.)s/2.”, wherein the examiner interprets inserting redundant neurons to surgically repair neurons connected to rows and columns in MCAs with many stuck-at-faults and inserting and updating an activation of the neurons to be the same as “adding at least one backup neuron to the neural network to generate an updated neural network, wherein the at least one backup neuron corresponds to a predetermined aging condition of the memory device” because they are both directed to providing additional neurons that are associated with degraded or faulty hardware conditions and that, when used, effectively produce an updated network including these backup neurons once the device reaches a certain fault or aging condition.)
Wiezbicki, Zhang, and the instant application are analogous art because they are all directed to neural network-based systems that maintain computational accuracy and reliability of memory or sensor devices subject to degradation, drift, or fault conditions over time.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the method claim 1 disclosed by Wiezbicki to include the “inserting redundant neurons and sharing redundant synapses to handle stuck-at-faults in memristor-based DNNs” disclosed by Zhang. One would be motivated to do so to effectively maintain network performance and fault tolerance under aging or degradation of memory devices, as suggested by Zhang ([Zhang, page 1]) “A common technique to reduce the negative impact of stuck-at-faults is to utilize redundant synapses, i.e, each row in a weight matrix is realized using two (or r) parallel rows in an MCA. In this paper, we propose to handle stuck-at-faults by inserting redundant neurons and by sharing redundant synapses. The first technique is based on inserting redundant neurons to surgically repair neurons connected to rows and columns in the MCAs with many stuck-at-faults.” AND [Zhang, page 1] “A challenge to computing paradigm is that MCAs are vulnerable to variations as resistance drift, random telegraph noise, temperature variations, and stuck-at-fault defects.” AND [Zhang, page 2] “Let W e be the effective matrix realized when a weight matrix W is mapped to an MCA with stuck-at-faults … Each element w e k in W e is equal to wk in W if wk is mapped to a memristor without a stuck-at-fault defect. w e k is equal to wmin or wmax if wk is mapped to a memristor stuck-off or stuck-on, respectively.”)
Regarding claim 21, Wiezbicki and Zhang teaches The training method of claim 1 (see rejection of claim 1).
Wiezbicki further teaches wherein a conversion of the original neurons from the original weights to the drifted weights corresponds to the predetermined aging condition; ([Wiezbicki, page 92] “For real applications the situation described above does not occur because currently available sensors have variations, this variation over time for the same sample with the same controlled variables is known as the drift … The drift is caused by two main factors. The first is the physical and chemical interactions of the components in the microstructure of the sensors. This may be due to the natural aging of sensor surface and also due to poisoning by an external dopant component.”, and [Wiezbicki, page 92] “Chemical gas sensors suffer from drift problem because of the chemical process employed. In this investigation we developed a model that uses an ensemble of neural networks in a parallel way combining the weighted output of classifiers to compensate the drift”, wherein the examiner interprets drift as a “variation over time” caused by natural aging of the sensing elements, and the use of neural networks to compensate that drift, to be the same as a conversion of the original neurons from the original weights to the drifted weights corresponds to the predetermined aging condition because they are both directed to using a trained neural network whose parameters change from original values to drifted values as the underlying device ages, and to treating those drifted values as representing an aging condition of the system at which compensation or adaptation is triggered. The examiner further interprets “developed a model … to compensate the drift” to be the same as a “predetermined aging condition” because both are deterministic approaches to modeling or predetermining what the drift will be so that weights can be adjusted to compensate for the drift.)
Claims 3,4, 7, 8-12, 14-16 are rejected under 35 U.S.C. 103 as being unpatentable over Wiezbicki in view of Zhang further in view of NPL reference “Snap-drift neural network for self-organisation and sequence learning” by Palmer-Brown et. al (referred herein as Palmer-Brown).
Regarding claim 3, Wiezbicki and Zhang teaches The training method of claim 1 (see rejection of claim 1).
Wiezbicki and Zhang do not teach wherein training the updated neural network comprises: keeping the drifted weights of the original neurons in the updated neural network to be fixed, and training the updated neural network by adjusting weights of the at least one backup neuron.
Palmer-Brown teaches wherein training the updated neural network comprises:
keeping the drifted weights of the original neurons in the updated neural network to be fixed, and training the updated neural network by adjusting weights of the at least one backup neuron. ([Palmer-Brown, Abstract] “The SDSOM uses the standard SOM architecture, where a layer of input nodes connects to the self-organising map layer and the weight update consists of either snap (min of input and weight) or drift (LVQ, as in SOM). The RSDNN uses a simple recurrent network (SRN) architecture, with the hidden layer values copied back to the input layer” and [Palmer-Brown, page 901, sec 3.3] “the main purpose of node (neuron) level performance calculation is to enable the s layer to retain the learning of nodes with already successful adaptations, by continuing learning with the same learning mode if that node’s performance has increased since the last performance update. Alternatively, where the performance of a particular node decreased since the last performance calculation, the learning mode of that particular s node will be swapped to the alternative learning mode to encourage relearning (snap and drift learn different features of the data) to occur.”, wherein the examiner interprets “a layer of input nodes connects to the self-organising map layer and the weight update consists of either snap (min of input and weight) or drift… layer values copied back to the input layer… node (neuron) level performance calculation is to enable the s layer to retain the learning of nodes with already successful adaptations, by continuing learning with the same learning mode if that node’s performance has increased since the last performance update…encourage relearning (snap and drift learn different features of the data) to occur” to be the same fixing drifted weights for neurons and training an updated/improved neural network that would involve adjusting weights).
Wiezbicki, Zhang, Palmer-Brown, and the instant application are analogous art, because they are all directed to keeping/fixing drifted weights of neurons in a updated neural network.
It would have been obvious to a person of ordinary skill in the art before the effective filling date of the invention to modify the training method of claim 1 disclosed by Wiezbicki and Zhang to include the “the weight update consists of either snap (min of input and weight) or drift… layer values copied back to the input layer… node (neuron) level performance calculation is to enable the s layer to retain the learning of nodes with already successful adaptations, by continuing learning with the same learning mode if that node’s performance has increased since the last performance update… encourage relearning (snap and drift)” as disclosed by Palmer-Brown. One would be motivated to do so to efficiently train a neural network by fixing drifted weights and adjusting weight values in a neural network using level performance calculation as suggested by Palmer-Brown (see [Palmer-Brown, Abstract and, page 901, sec 3.3]).
Regarding claim 4, Wiezbicki and Zhang teaches The training method of claim 1 (see rejection of claim 1).
Wiezbicki and Zhang do not teach do not teach wherein the first model comprises at least two computing layers.
Palmer-Brown teaches wherein the first model comprises at least two computing layers. ([Palmer-Brown, page 899, sec 3.1] “The learning of both of the layers in the neural system.”, wherein the examiner interprets “The learning of both of the layers in the neural system” to be the same as a model that will computerize two computed layers.)
Wiezbicki, Zhang, Palmer-Brown, and the instant application are analogous art, because they are all directed to neural network/model that comprises at least two computing layers.
It would have been obvious to a person of ordinary skill in the art before the effective filling date of the invention to modify the training method of claim 1 disclosed by Wiezbicki and Zhang to include the “learning of both of the layers in the neural system” as disclosed by Palmer-Brown. One would be motivated to do so to obtain a model as a neural system that has at least two computing layers as suggested by Palmer-Brown (see [Palmer-Brown, page 899, sec 3.1] quote above).
Regarding claim 7, Wiezbicki teaches:
An operating method of a memory system, comprising: using a first model formed by a plurality of original neurons stored in a first memory block of the memory system for computation; ([Wiezbicki, page 93] “five multilayer perceptron neural networks (MLP ANN) were trained for each of the 10 batches available. Each neural network has 128 input neurons representing each normalized characteristic that are derived from 8 basic characteristics for each one of the 16 sensors in the original dataset…qNN is the number of neural networks (5) that are trained and stored for each batch.”, wherein the examiner interprets Wiezbicki’s disclosure of trained MLP networks and qNN trained, stored and executed in memory for computation to be the same as a first model formed by a plurality of original neurons stored in a first memory block, because both describe a neural-network model composed of multiple neurons whose parameters are retained in memory and used for computation.)
obtaining an aging condition of the memory system; ([Wiezbicki, page 92-93] “Chemical gas sensors suffer from drift problems because of the chemical process employed … The drift is caused by changes over time in the microstructure of the sensors.”, wherein the examiner interprets measurement of sensor drift and hardware changes over time to be the same as obtaining an aging condition of a memory system, because both involve observing degradation or time-dependent change in the characteristics of the underlying hardware components used for computation.)
determining whether the aging condition meets a predetermined aging condition ([Wiezbicki, page 93] “If drift is detected, then the learner is adapted … the ANNs trained with batch i classify batch i + 1.”, wherein the examiner interprets detecting drift and triggering adaptation to be the same as determining whether an aging condition meets a predetermined aging condition, because both describe comparing measured change against a defined threshold to decide when retraining or adjustment is needed.)
Wiezbicki does not teach when it is determined that the aging condition meets the predetermined aging condition and that the weights stored in the first memory block are drifted, enabling a second memory block storing at least one backup neuron; using a second model formed by the original neurons and the at least one backup neuron stored in the first memory block and the second memory block for computation;.
Zhang teaches:
when it is determined that the aging condition meets the predetermined aging condition…enabling a second memory block storing at least one backup neuron; ([Zhang, page 1-2] “Redundant neurons and shared redundant synapses are used for robust memristor-based DNNs with reduced overhead … Stuck-at-fault defects may degrade the classification accuracy of the memristor-based DNNs. This technique aims to improve the robustness of a DNN to stuck-at-faults by inserting redundant neurons.”, wherein the examiner interprets the activation of redundant neurons to compensate for faulted or degraded memory cells to be the same as enabling a second memory block storing at least one backup neuron, because both describe adding or activating additional neurons in alternate hardware locations when aging or faults are detected.)
and drifting of weights stored in the first memory block. ([Zhang, page 1] “the non-volatile memristors allow the synapse weights to be stored in-situ [3, 12], which circumvents the communication bottleneck of the von-Neumann architecture [16].”AND [Zhang, page 1] “A challenge to computing paradigm is that MCAs are vulnerable to variations as resistance drift, random telegraph noise, temperature variations, and stuck-at-fault defects.” AND [Zhang, page 2] “Let W e be the effective matrix realized when a weight matrix W is mapped to an MCA with stuck-at-faults.” AND [Zhang, page 2] “Each element w e k in W e is equal to wk in W if wk is mapped to a memristor without a stuck-at-fault defect. w e k is equal to wmin or wmax if wk is mapped to a memristor stuck-off or stuck-on, respectively.”, wherein the examiner interprets “the non-volatile memristors allow the synapse weights to be stored in-situ” to be the same as “weights stored in the first memory block” because they are both directed to storing neural-network weight values in memory devices. The examiner further interprets “MCAs are vulnerable to variations as resistance drift, random telegraph noise, temperature variations, and stuck-at-fault defects” and “the effective matrix realized when a weight matrix W is mapped to an MCA with stuck-at-faults” to be the same as “drifting of weights” because they are both directed to stored neural-network weights having effective changed values due to memory-cell variation, drift, or fault conditions).
when it is determined that the aging condition meets the predetermined aging condition and that the weights stored in the first memory block are drifted, enabling a second memory block storing at least one backup neuron; ([Zhang, page 1] “In this paper, we propose to handle stuck-at-faults by inserting redundant neurons and by sharing redundant synapses.” AND [Zhang, page 1] “The first technique is based on inserting redundant neurons to surgically repair neurons connected to rows and columns in the MCAs with many stuck-at-faults.” AND [Zhang, page 2] “This technique aims to improve the robustness of a DNN to stuck-at-faults by inserting redundant neurons to replicate neurons connected rows and columns in the MCAs with many stuck-at-faults, which is shown in Figure 1(c).”, wherein the examiner interprets “inserting redundant neurons” and “surgically repair neurons connected to rows and columns in the MCAs with many stuck-at-faults” to be the same as “enabling a second memory block storing at least one backup neuron” because they are both directed to activating or adding redundant neuron resources in memory hardware to compensate for faulty or degraded memory-cell conditions. The examiner further interprets “with many stuck-at-faults” and Zhang’s teaching that MCAs are vulnerable to “resistance drift” and “temperature variations” to be the same as “when it is determined that the aging condition meets the predetermined aging condition and that the weights stored in the first memory block are drifted” because they are both directed to using redundant neuron resources after a memory-device degradation or drift condition affects stored neural-network weights).
Wiezbecki and Zhang do not teach using a second model formed by the original neurons and the at least one backup neuron stored in the first memory block and the second memory block for computation.
Palmer-Brown teaches using a second model formed by the original neurons and the at least one backup neuron stored in the first memory block and the second memory block for computation. ([Palmer-Brown, page 897-898] “Snap-drift neural networks employ modal learning combining snap (initial setup) and drift (weight adaptation) modes … the weight update consists of either snap (initial input of weights) or drift (update).”, wherein the examiner interprets the use of both original (snap) weights and updated (drift) weights together in subsequent computation to be the same as using a second model formed by the original neurons and the at least one backup neuron stored in the first and second memory blocks, because both describe computation performed using a combination of pre-existing and newly adapted neuronal parameters.)
Wiezbicki, Zhang, Palmer-Brown, and the instant application are analogous art because they are all directed to operating methods of a memory system using neural networks that monitor aging or drift conditions and employ redundant/adaptive neurons so that computation accuracy is maintained over time.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the method of using stored memory and predetermined values disclosed by Wiezbicki to include the “redundant neurons and shared redundant synapses for robust memristor-based DNNs with reduced overhead,” and “snap-drift neural network employs modal learning combining snap and drift modes,” disclosed by Zhang and Palmer-Brown, respectively. One would be motivated to do so to effectively maintain robust and accurate neural-network computation in the presence of aging, drift, and device faults, as suggested by Zhang (Zhang, page 1-2] “improve the robustness of a DNN to stuck-at-faults by inserting redundant neurons”, by Palmer-Brown (Palmer-Brown, page 897-898] “snap-drift neural networks … maintain performance over time”)
Regarding claim 8, Wiezbicki, Zhang, and Palmer-Brown teaches The operating method of claim 7 (see rejection of claim 7).
Wiezbicki further teaches wherein the aging condition meets the predetermined aging condition when an operating time of the memory system is greater than or equal to a predetermined time, or an operating temperature of the memory system is greater than or equal to a predetermined temperature. ([Wiezbicki, page 92, sec 1] “The drift is caused by two main factors. The first is the physical and chemical interactions of the components in the microstructure of the sensors. This may be due to the natural aging of sensor surface and also due to poisoning by an external dopant component. The second factor is due to uncontrollable changes in the environment such as temperature and humidity variations”, wherein the examiner interprets “The drift is caused by two main factors…This may be due to the natural aging of sensor surface… uncontrollable changes in the environment such as temperature and humidity variations”, to be the same as determining if an aging condition, operating time, and operating temperature are greater than or equal to a predetermined threshold).
Regarding claim 9, Wiezbicki, Zhang, and Palmer-Brown teaches The operating method of claim 7 (see rejection of claim 7).
Wiezbicki further teaches wherein the first memory block is programmed to store original weights of the original neurons of the first model, wherein the first memory block is used for performing computation as the first model before it is determined that the aging condition meets the predetermined aging condition. ([Wiezbicki, page 93, sec 3] “In order to test the decrease of correct classification percentage due to drift effects, a neural network was trained with the data of the first batch and tested in subsequent batches”, wherein the examiner interprets “first batch” to be the same as original weights that would be trained at an initial timepoint for computation prior to detecting drift effects, which would be similar to some aging condition, and that testing the data with subsequent “batches” means that drift aging in the neural network’s performance is done over time solidifies that it is “predetermined”.)
Regarding claim 10, Wiezbicki, Zhang, and Palmer-Brown teaches The operating method of claim 9 (see rejection of claim 9).
Palmer-Brown further teaches wherein when it is determined that the aging condition meets the predetermined aging condition, the original weights of the original neurons stored in the first memory block are transferred as drifted weights. ([Palmer-Brown, page 901, sec 3.3] “the learning mode of each of the s nodes varies during each learning epoch…. The combined effect in the algorithm of continually selecting the better performing mode on each neuron, and of a probability of learning that is dependent on performance, is to reinforce the otherwise entirely unsupervised learning in its most successful directions, with different modes predominating on different neurons, and rapid learning slowing and converging as performance improves.”, wherein the examiner interprets “The combined effect in the algorithm of continually selecting the better performing mode on each neuron, and of a probability of learning that is dependent on performance, is to reinforce the otherwise entirely unsupervised learning in its most successful directions, with different modes predominating on different neurons, and rapid learning slowing and converging as performance improves” to be the same as transferring weights as drifted once a condition regards to its aging has been met.)
Wiezbicki, Zhang, Palmer-Brown, and the instant application are analogous art because they are all directed to meeting a certain condition for drifting weights.
It would have been obvious to one of ordinary skill in the art before the effect of filing date to modify the method of claim 9 disclosed by Wiezbicki, Zhang, and Palmer-Brown to include the “combined effect in the algorithm of continually selecting the better performing mode on each neuron, and of a probability of learning that is dependent on performance” as disclosed by Palmer-Brown. One would be motivated to do so to efficiently transfer weights after meeting a condition for improvement as suggested by Palmer Brown (see [Palmer-Brown, page 901, sec 3.3] quote above.)
Regarding claim 11, Wiezbicki, Zhang, and Palmer-Brown teaches The operating method of claim 10 (see rejection of claim 10).
Wiezbicki further teaches wherein the second memory block is programmed to store predicted weights of the at least one backup neuron. ([Wiezbicki, Abstract] “In this investigation we developed a model that uses an ensemble of neural networks in a parallel way combining the weighted output of classifiers to compensate the drift.”, wherein the examiner interprets “we developed a model that uses an ensemble of neural networks in a parallel way combining the weighted output of classifiers” to be the same as block of memory that is programmed to store weights.)
Regarding claim 12, Wiezbicki, Zhang, and Palmer-Brown teaches The operating method of claim 11 (see rejection of claim 11).
Palmer-Brown further teaches wherein when it is determined that the aging condition meets the predetermined aging condition, the predicted weights of the at least one backup neuron is transferred as backup weights of the second model. ([Palmer, page 900, sec 3.3] “In the s layer, a ‘quality assurance’ threshold is used: if the net input of the most active s node is above the threshold, that s node is accepted as the winner, and defines the category of the input pattern; otherwise a new uncommitted output node will be recruited as the winner and its weights initialised with the current d layer pattern (the D winning nodes)”, wherein the examiner interpret “the threshold, that s node is accepted as the winner, and defines the category of the input pattern; otherwise a new uncommitted output node will be recruited as the winner and its weights initialised with the current d layer pattern” to be the same as a backup neuron as other neurons age once considered by a threshold and transferring predicted weights conditionally.)
Wiezbicki, Zhang, Palmer-Brown, and the instant application are analogous art because they are all directed to meeting a predetermined aging condition for predicted/backup weight values.
It would have been obvious to one of ordinary skill in the art before the effect of filing date to modify the method of claim 11 as disclosed by Wiezbicki, Zhang, and Palmer-Brown to include the “the threshold, that s node is accepted as the winner, and defines the category of the input pattern; otherwise a new uncommitted output node will be recruited as the winner and its weights initialised with the current d layer pattern” as disclosed by Palmer-Brown. One would be motivated to do so to effectively use a threshold for winning nodes, and weeding out “uncommitted” or aged values as suggested by Palmer-Brown (see [Palmer, page 900, sec 3.3] quote above).
Regarding claim 14, Wiezbicki, Zhang, and Palmer-Brown teaches The operating method of claim 8 (see rejection of claim 8).
Wiezbicki further teaches The operating method of claim 8, wherein when the operating temperature returns to be less than the predetermined temperature, the second memory block is disabled and the first memory block is used for computation. ([Wiezbicki, page 92, sec 1] “the drift is caused by two main factors. The first is the physical and chemical interactions of the components in the microstructure of the sensors. This may be due to the Natural aging of sensor surface and also due to poisoning by an external dopant component. The second factor is due to uncontrollable changes in the environment such as temperature and humidity variation”, wherein the examiner interprets “uncontrollable changes in the environment such as temperature variations” to be the same as operating temperature returning to be less than the predetermined temperature, as both are directed to environmental temperature changes that trigger operational adjustments).
Regarding claim 15, Wiezbicki teaches:
A memory system, comprising: a memory array, comprising: a first memory block configured to store a plurality of original neurons; ([Wiezbicki, abstract] “In gas classification systems with multiple sensors, the individual sensor drift affects the system classification capacity over time. A model created to classify data at certain time, doesn't present the same efficiency to classify a sample in a future time. Depending on the problem, this time interval can be days, weeks or months. Chemical gas sensors suffer from drift problem because of the chemical process employed. In this investigation we developed a model that uses an ensemble of neural networks in a parallel way combining the weighted output of classifiers to compensate the drift.”, wherein the examiner interprets “a model that uses an ensemble of neural networks in a parallel way combining the weighted output of classifiers” to be the same as “a memory system … a first memory block configured to store a plurality of original neurons” because they are both directed to a stored neural-network model (or ensemble of networks) whose neurons and weights are held in memory and used for later computation.)
a controller, coupled to the memory array, wherein the controller is configured to: use the original neurons stored in the first memory block for computation; ([Wiezbicki, page 92, Abstract] “Results show that performance of correct classifications of the gas samples using both methods improved when compared to classifiers trained with just recent data… Measuring systems based on an array of gas sensors have the purpose of sensing the type and concentration of the substance that is analyzed. They have wide range of applications in the food industries, control and monitoring, public safety, etc…([Wiezbicki, page 93] “five multilayer perceptron neural networks (MLP ANN) were trained for each of the 10 batches available. Each neural network has 128 input neurons representing each normalized characteristic that are derived from 8 basic characteristics for each one of the 16 sensors in the original dataset…qNN is the number of neural networks (5) that are trained and stored for each batch.”, wherein the examiner interprets “correct classifications of the gas samples” performed by the neural-network model to be the same as a controller using the original neurons stored in the first memory block for computation because they are both directed to executing the stored neural-network parameters to compute classification outputs.)
obtain an aging condition of the memory system; ([Wiezbicki, page 92-93] “Chemical gas sensors suffer from drift problems because of the chemical process employed … The drift is caused by changes over time in the microstructure of the sensors.”, wherein the examiner interprets measurement of sensor drift and hardware changes over time to be the same as obtaining an aging condition of a memory system, because both involve observing degradation or time-dependent change in the characteristics of the underlying hardware components used for computation.)
determine whether the aging condition meets a predetermined aging condition; ([Wiezbicki, page 93] “If drift is detected, then the learner is adapted … the ANNs trained with batch i classify batch i + 1.”, wherein the examiner interprets detecting drift and triggering adaptation to be the same as determining whether an aging condition meets a predetermined aging condition, because both describe comparing measured change against a defined threshold to decide when retraining or adjustment is needed.)
Wiezbicki does not teach when it is determined that the aging condition meets the predetermined aging condition, the second memory block is enabled and the original neurons and the at least one backup neuron stored in the first memory block and the second memory block are used for computation.
Zhang teaches:
and a second memory block configured to store at least one backup neuron; ([Zhang, page 1-2] “Redundant neurons and shared redundant synapses are used for robust memristor-based DNNs with reduced overhead … Stuck-at-fault defects may degrade the classification accuracy of the memristor-based DNNs. This technique aims to improve the robustness of a DNN to stuck-at-faults by inserting redundant neurons.”, wherein the examiner interprets the activation of redundant neurons to compensate for faulted or degraded memory cells to be the same as enabling a second memory block storing at least one backup neuron, because both describe adding or activating additional neurons in alternate hardware locations when aging or faults are detected.)
when it is determined that the aging condition meets the predetermined aging condition … the second memory block is enabled ([Zhang, page 1-2] “Redundant neurons and shared redundant synapses are used for robust memristor-based DNNs with reduced overhead … Stuck-at-fault defects may degrade the classification accuracy of the memristor-based DNNs. This technique aims to improve the robustness of a DNN to stuck-at-faults by inserting redundant neurons.”, wherein the examiner interprets the activation of redundant neurons to compensate for faulted or degraded memory cells to be the same as enabling a second memory block when a predetermined condition has been met, because both describe adding or activating additional neurons in alternate hardware locations when aging or faults are detected.)
and drifting of weights stored in the first memory block; ([Zhang, page 1] “the non-volatile memristors allow the synapse weights to be stored in-situ [3, 12], which circumvents the communication bottleneck of the von-Neumann architecture [16].” AND [Zhang, page 1] “A challenge to computing paradigm is that MCAs are vulnerable to variations as resistance drift, random telegraph noise, temperature variations, and stuck-at-fault defects.” AND [Zhang, page 2] “Let W e be the effective matrix realized when a weight matrix W is mapped to an MCA with stuck-at-faults.” AND [Zhang, page 2] “Each element w e k in W e is equal to wk in W if wk is mapped to a memristor without a stuck-at-fault defect. w e k is equal to wmin or wmax if wk is mapped to a memristor stuck-off or stuck-on, respectively.”, wherein the examiner interprets “the non-volatile memristors allow the synapse weights to be stored in-situ” to be the same as “weights stored in the first memory block” because they are both directed to storing neural-network weight values in memory devices. The examiner further interprets “MCAs are vulnerable to variations as resistance drift, random telegraph noise, temperature variations, and stuck-at-fault defects” and “the effective matrix realized when a weight matrix W is mapped to an MCA with stuck-at-faults” to be the same as “drifting of weights” because they are both directed to stored neural-network weights having effective changed values due to memory-cell variation, drift, or fault conditions).
when it is determined that the aging condition meets the predetermined aging condition and that the weights stored in the first memory block are drifted, the second memory block is enabled and the original neurons and the at least one backup neuron; ([Zhang, page 1] “In this paper, we propose to handle stuck-at-faults by inserting redundant neurons and by sharing redundant synapses. “AND [Zhang, page 1] “The first technique is based on inserting redundant neurons to surgically repair neurons connected to rows and columns in the MCAs with many stuck-at-faults.” AND [Zhang, page 2] “This technique aims to improve the robustness of a DNN to stuck-at-faults by inserting redundant neurons to replicate neurons connected rows and columns in the MCAs with many stuck-at-faults, which is shown in Figure 1(c).”, wherein the examiner interprets “inserting redundant neurons” and “surgically repair neurons connected to rows and columns in the MCAs with many stuck-at-faults” to be the same as “enabling a second memory block storing at least one backup neuron” because they are both directed to activating or adding redundant neuron resources in memory hardware to compensate for faulty or degraded memory-cell conditions. The examiner further interprets “with many stuck-at-faults” and Zhang’s teaching that MCAs are vulnerable to “resistance drift” and “temperature variations” to be the same as “when it is determined that the aging condition meets the predetermined aging condition and that the weights stored in the first memory block are drifted” because they are both directed to using redundant neuron resources after a memory-device degradation or drift condition affects stored neural-network weights).
Wiezbicki and Zhang do not teach the original neurons and the at least one backup neuron stored in the first memory block and the second memory block are used for computation.
Palmer-Brown teaches the original neurons and the at least one backup neuron stored in the first memory block and the second memory block are used for computation. ([Palmer-Brown, page 897-898] “Snap-drift neural networks employ modal learning combining snap (initial setup) and drift (weight adaptation) modes … the weight update consists of either snap (initial input of weights) or drift (update).”, wherein the examiner interprets the use of both original (snap) weights and updated (drift) weights together in subsequent computation to be the same as using a second model formed by the original neurons and the at least one backup neuron stored in the first and second memory blocks, because both describe computation performed using a combination of pre-existing and newly adapted neuronal parameters.)
Wiezbicki, Zhang, Palmer-Brown, and the instant application are analogous art because they are all directed to a memory-based neural network system that monitors an aging or degradation condition of the underlying hardware and, when that condition is met, enables additional neural resources so that computation can continue using both original and adjusted or backup neurons.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the system of predetermined conditions and storing neural data to a memory system disclosed by Wiezbicki to include the “we propose to handle stuck-at-faults by inserting redundant neurons and by sharing redundant synapses” disclosed by Zhang and the “the learning mode of that particular s node will be swapped to the alternative learning mode to encourage relearning (snap and drift learn different features of the data)” disclosed by Palmer-Brown. One would be motivated to do so to effectively using computation performed using neuron parameters stored Palmer-Brown (Palmer-Brown, page 901, section 3.2, “the learning mode of that particular s node will be swapped to the alternative learning mode to encourage relearning”).
Regarding claim 16, Wiezbicki, Zhang, and Palmer-Brown teaches The memory system of claim 15, (see rejection to claim 15).
Wiezbicki further teaches wherein the controller determines that the Aging condition meets the predetermined aging condition when it operating time of the memory system is greater than or equal to a predetermined time, operating temperature of the memory system is greater than or equal to a predetermined temperature. ([Wiezbicki, page 92, sec 1] “The drift is caused by two main factors. The first is the physical and chemical interactions of the components in the microstructure of the sensors. This may be due to the natural aging of sensor surface and also due to poisoning by an external dopant component. The second factor is due to uncontrollable changes in the environment such as temperature and humidity variations”, wherein the examiner interprets “natural aging of sensor surface” to be the same as operating time of the memory system being greater than or equal to a predetermined time, and “the controllable changes in the environment such as temperature variations” to be the same as an operating temperature of the memory system being greater than or equal to a predetermined temperature).
Claim 13, 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over Wiezbicki in view of Zhang in view of Palmer-Brown further in view of NPL reference “Using Redundancy to Improve the Performance of Artificial Neural Networks*” by Medler et. al (referred herein as Medler).
Regarding claim 13, Wiezbicki, Zhang, and Palmer-Brown teaches The operating method of claim 12 (see rejection of claim 12).
Palmer-Brown further teaches wherein when it is determined that the aging condition meets the predetermined aging condition, the second memory block is enabled and the at least one backup neuron is added to the first model to generate the second model, ([Palmer, page 901, sec 3.2], “the main purpose of node (neuron) level performance calculation is to enable the s layer to retain the learning of nodes with already successful adaptations, by continuing learning with the same learning mode if that node’s performance has increased since the last performance update. Alternatively, where the performance of a particular node decreased since the last performance calculation, the learning mode of that particular s node will be swapped to the alternative learning mode to encourage relearning (snap and drift learn different features of the data) to occur”, wherein the examiner interprets “enable the s layer to retain the learning of nodes with already successful adaptations, by continuing learning with the same learning mode if that node’s performance has increased since the last performance update. Alternatively, where the performance of a particular node decreased since the last performance calculation, the learning mode of that particular s node will be swapped to the alternative learning mode to encourage relearning (snap and drift learn different features of the data) to occur” to be the same as for a certain predetermined condition, memory is enabled for learning of the nodes to take place).
Wiezbicki, Zhang, Palmer-Brown, and the instant application are analogous art because they are all directed to operating methods of neural network-based memory systems that modify or reconfigure neurons in response to degradation, drift, or aging conditions in order to maintain accurate computation.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the method claim 7 disclosed by Wiezbicki and Zhang to include the “the learning mode of that particular s node will be swapped to the alternative learning mode to encourage relearning (snap and drift learn different features of the data) to occur” disclosed by Palmer-Brown. One would be motivated to do so to effectively allow the system to adapt to degraded performance or aging conditions by triggering an alternative learning mode when node performance decreases, as suggested by Palmer-Brown ([Palmer-Brown, page 901, sec 3.2] “courage relearning (snap and drift learn different features of the data) to occur”).
Wiezbicki and Palmer-Brown does not teach wherein the first memory block storing the drifted weights of the original neurons and the second memory block storing the backup weights of the at least one backup neuron are used for performing computation as the second model.
Medler teaches wherein the first memory block storing the drifted weights of the original neurons and the second memory block storing the backup weights of the at least one backup neuron are used for performing computation as the second model. ([Medler, page 3, sec 2.1.1] “The redundant network architecture was created by replicating the hidden unit layer and the output unit layer a set number of times. Each of the replicated output units was then connected to a Decision Unit, which acts as the redundant network's output unit. Fall connections leading to the Decision Unit are modifiable”, wherein the examiner interprets the replicated output units with modifiable connections providing inputs to a Decision Unit to be the same as using original neurons with drifted weights and backup neurons [that would be stored unto the Decision Unit] with backup weights for computation as the second model).
Wiezbicki, Zhang, Palmer-Brown, Medler, and the instant application are analogous art, because they are directed to when a condition is met, to enable memory blocks for the neurons for a model/neural network.
It would have been obvious to one of ordinary skill in the art before the effect of filing date to modify the method of claim 12 as disclosed by Wiezbicki, Zhang, and Palmer-Brown to include the “enable the s layer to retain the learning of nodes with already successful adaptations, by continuing learning with the same learning mode if that node’s performance has increased since the last performance update” as disclosed by Medler. One would be motivated to do so to efficiently enable memory through a layer of a neural network/model as suggested by Medler (see [Palmer, page 901, sec 3.2] quote above.)
Regarding claim 17, Wiezbicki, Zhang, and Palmer-Brown teaches The memory system of claim 16, (see rejection to claim 16).
Wiezbicki, Zhang, and Palmer-Brown do not teach wherein the first memory block is programmed to store original weights of the original neurons of a first model, the second memory block is programmed to store predicted weights of at least one the backup neuron.
Medler teaches wherein the first memory block is programmed to store original weights of the original neurons of a first model, the second memory block is programmed to store predicted weights of at least one the backup neuron. ([Medler, page 3] “ever done at work architecture was created by replicating the hidden unit later in the output unit layer a set number of times. Each of the replicated output units was then connected to a decision unit, at the Redundant networks output unit. All connections leading into the decision unit or modifiable; the decision unit's response is a weighted sum of the replicated output units.”, wherein the examiner interprets replicating the hidden unit layer and output unit layer with modifiable connections providing a weighted sum to be the same as programming a second memory block to store predicted weights of at least one backup neuron, father original layers between original weights and the first memory block).
Wiezbicki, Zhang, Palmer-Brown, Medler, and the instant application are analogous art, because they are all directed to memory systems that manage model performance by controlling how different set of neural weights are stored and utilized for computation.
It would have been obvious to a person of ordinary skill in the art before the effect of filing date of The Invention to modify the method of claim 16 as disclosed by Wiezbicki, Zhang, and Palmer-Brown to include the redundant network architecture using replicated hidden and output layers disclosed by Medler. One would be motivated to do so to effectively improve system adapting to environmental drift, a.k.a. temperature using the redundant network architecture as suggested by Medler (see [Medler, page 3] quote above.)
Regarding claim 18, Wiezbicki, Zhang, Palmer-Brown, and Medler teaches The memory system of claim 17, (see rejection to claim 17).
Palmer-Brown further teaches wherein when it is determined by the controller that the aging condition meets the predetermined aging condition, the original weights of the original neurons stored in the first memory block are transferred as drifted weights and the predicted weights of the at least one backup neuron is transferred as backup weights of a second model. ([Palmer-Brown, page 901, sec 3.3] “Alternatively, where the performance of a particular node decreased since the last performance calculation, the learning mode of that particular s node will be swapped to the alternative learning mode to encourage relearning (snap and drift learn different features of the data) to occur. This is also applied to the d nodes. Thus, the learning mode of each of the s nodes varies during each learning epoch.”, wherein the examiner interprets swapping in the learning mode of a node to encourage swapping the learning mode of a node to encourage relearning by using snap and drift nodes, which learn different features of the data to be the same as transferring predicted weights of a second model).
Wiezbicki, Zhang, Palmer-Brown, Medler, and the instant application are analogous art because they are all directed to memory systems that manage neuron weight transitions to maintain or improve model performance in response to degradation or changing conditions.
It would have been obvious to a person of ordinary skill in the art before the effect of filing date of The Invention to modify the method of claim 17 as disclosed by Wiezbicki, Zhang, Palmer-Brown, and Medler to include “the learning mode of that particular s node will be swapped to the alternative learning mode to encourage relearning (snap and drift learn different features of the data) to occur” as disclosed by Palmer-Brown. One would be motivated to do so to initiate relearning of drifted or weight values by swapping learning modes as a method for transferring weights as suggested by Palmer-Brown (see [Palmer-Brown, page 901, sec 3.3] quote above).
Regarding claim 19, Wiezbicki, Zhang, Palmer-Brown, and Medler teaches The memory system of claim 17, (see rejection to claim 17).
Medler further teaches wherein the first memory block having the original neurons with the drifted weights and the second memory block having the at least one backup neuron with the backup weights are used for performing computation as the second model ([Medler, page 3, sec 2.1.1] “The redundant network architecture was created by replicating the hidden unit layer and the output unit layer a set number of times. Each of the replicated output units was then connected to a Decision Unit, which acts as the redundant network's output unit. Fall connections leading to the Decision Unit are modifiable”, wherein the examiner interprets the replicated output units with modifiable connections providing inputs to a Decision Unit to be the same as using original neurons with drifted weights and backup neurons with backup weights for computation as the second model).
Palmer-Brown further teaches wherein when it is determined by the controller that the Aging condition meets the predetermined aging condition, the second memory block is enabled and the at least one backup neuron is added to the first model to generate the second model. ([Palmer-Brown, page 901, sec 3.2], “the main purpose of node (neuron) level performance calculation is to enable the s layer to retain the learning of nodes with already successful adaptations, by continuing learning with the same learning mode if that node’s performance has increased since the last performance update. Alternatively, where the performance of a particular node decreased since the last performance calculation, the learning mode of that particular s node will be swapped to the alternative learning mode to encourage relearning (snap and drift learn different features of the data) to occur”, wherein the examiner interprets “enable the s layer to retain the learning of nodes with already successful adaptations, by continuing learning with the same learning mode if that node’s performance has increased since the last performance update. Alternatively, where the performance of a particular node decreased since the last performance calculation, the learning mode of that particular s node will be swapped to the alternative learning mode to encourage relearning (snap and drift learn different features of the data) to occur” to be the same as for a certain predetermined condition, memory is enabled for learning of the nodes to take place.)
Wiezbicki, Zhang, Palmer-Brown, Medler, and the instant application are analogous art, because there are all directed to memory systems that manage computation by selectively combining original and backup neuron weights based on operating conditions.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of The Invention to modify the method of claim 17 as disclosed by Wiezbicki, Zhang, Palmer-Brown, and Medler to include the modifiable connections providing inputs to a Decision Unit disclosed by Medler and “enable the s layer to retain the learning of nodes with already successful adaptations, by continuing learning with the same learning mode if that node’s performance...s node will be swapped to the alternative learning mode to encourage relearning (snap and drift learn different features of the data) to occur” as disclosed by Palmer-Brown. One would be motivated to do so to effectively use drifted or any type of weights for performing computation on a model when a certain condition is met as suggested by Palmer-Brown (see [Palmer-Brown, page 901, sec 3.2] quote above.)
Regarding claim 20, Wiezbicki, Zhang, Palmer-Brown, and Medler teaches The memory system of claim 19, (see rejection to claim 19).
Palmer-Brown further teaches wherein the at least one backup neuron is added to at least one arbitrary layer rather than the last layer of the first model to generate the second model. ([Palmer-Brown, page 900, sec 3.3] “The distributed d layer groups the input patterns according to their features using snap- drift. The D most activated (winning) d nodes lose weight prototypes best match the current input pattern are used as the input data to the selection, s layer, for the purposes of feature classification”, wherein the examiner interprets using the D most activated d nodes as input to the s layer to be the same as adding a backup neuron to an arbitrary layer rather than the last layer.)
Wiezbicki, Zhang, Palmer-Brown, Medler, and the instant application are analogous art because they are all directed to a neural network memory system that manages placement of additional competition elements, like back up neurons, within a model architecture.
It would have been obvious to a person of ordinary skill in the art before the effective filing date to modify the method of claim 19 as disclosed by Wiezbicki, Zhang, Palmer-Brown, and Medler to include the “The D most activated (winning) d nodes lose weight prototypes best match the current input pattern are used as the input data to the selection, s layer, for the purposes of feature classification” as disclosed by Palmer-Brown. One would be motivated to do so to use feature classification to add neurons to layers within a model as suggested by Palmer-Brown. (see [Palmer-Brown, page 900, sec 3.3] quote above.)
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Wiezbicki in view of Zhang further in view Medler.
Regarding claim 5, Wiezbicki and Zhang teaches The training method of claim 1 (see rejection of claim 1).
Wiezbicki and Zhang do not teach wherein the first model and the second model have the same amount of outputs.
Medler teaches wherein the first model and the second model have the same amount of outputs. ([Medler, page 3, sec 2.1.1] “The redundant network architecture was created by replicating the hidden unit layer and the output unit layer a set number times. Each of the replicated output units was then connected to a Decision Unit, which acts as the Redundant Networks output unit”, wherein the examiner interprets “replicating... output unit layer a set number of times” to be the same as maintaining the same amount of outputs in both first and second models).
Wiezbicki, Zhang, Medler, and the instant application are analogous art because they are all directed to neural network training methods that have the same amount of outputs.
It would have been obvious to a person of ordinary School in the art before the affected finally do the invention to modify the method of claim 1 disclosed by Wiezbicki and Zhang to include the “replicating...output unit layer a set number of times” as disclosed by Medler. One would be motivated to do so to set output units to a set number of times to equalize the number of outputs from models as suggested by Medler (see [Medler, page 3, sec 2.1.1] quote above.)
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Wiezbicki in view of Zhang further in view of NPL reference “When Neurons Fail” by Mhamdi et al (referred herein as Mhamdi).
Regarding claim 6, Wiezbicki and Zhang teaches The training method of claim 1, (see rejection of claim 1).
Wiezbicki and Zhang do not teach wherein the at least one backup neuron is added to at least one arbitrary computing layer rather than a last computing layer of the neural network.
Mhamdi teaches wherein the at least one backup neuron is added to at least one arbitrary computing layer rather than a last computing layer of the neural network. ([Mhamdi, page 5, sec 4] “This section generalizes Theorem 1. While that theorem says that we can derive a tight bound on how many neurons can crash without losing -accuracy, it does not capture the situation where neurons can send values different from those expected, whether this difference is arbitrary or controlled.”, wherein the examiner interprets “where neurons can send values different from those expected, whether this difference is arbitrary or controlled” to be the same as a neuron being added to an arbitrary layer of a neural network vs a computing layer of a neural network )
Wiezbicki, Zhang, Mhamdi, and the instant application are analogous art, because they are all directed to neurons being added to arbitrary layer over the last computing layer of the neural network.
It would have been obvious to a person of ordinary skill in the art before the effective filling date of the invention to modify the training method of claim 1 disclosed by Wiezbicki and Zhang to include the “where neurons can send values different from those expected, whether this difference is arbitrary or controlled” as disclosed by Mhamdi. One would be motivated to do so to efficiently send neurons to an arbitrary or controlled layer of a neural network as suggested by Mhamdi (see [Mhamdi, page 5, sec 4] quote above).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DEVAN KAPOOR whose telephone number is (703)756-1434. The examiner can normally be reached Monday - Friday: 9:00AM - 5:00 PM EST (times may vary).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David Yi can be reached at (571) 270-7519. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DEVAN KAPOOR/Examiner, Art Unit 2126
/DAVID YI/Supervisory Patent Examiner, Art Unit 2126
Prosecution Insights