DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Drawings
The drawings filed on 04/03/2024 are accepted.
Specification
The specification filed on 04/03/2024 is accepted.
Information Disclosure Statement
The examiner has considered the information disclosure statements (IDS) submitted on 04/03/2024.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 5 and 8 are rejected under 35 U.S.C. 103 as being unpatentable over Lathrop et al. (US Pub. 2021/0406648) in view of Pyragas et al. (Using reservoir computer to predict and prevent extreme events) and further in view of Khatami et al. (US Pub. 2021/0406726).
As per claim 1, Lathrop teaches an information processing device [abstract, “An integrated circuit device for reservoir computing”] comprising:
an input layer [Fig. 1, paragraph 0032, “reservoir computing according to this disclosure can include a device having an input layer, input-processing components, an internal network having a reservoir, output-processing components, and an output layer … input layer 110”];
a reservoir layer connected to the input layer [Fif. 1, paragraphs 0032-0033, “reservoir computing according to this disclosure can include a device having an input layer, input-processing components, an internal network having a reservoir, output-processing components, and an output layer … Input layer 110 can be coupled to input nodes of the reservoir of internal network 120”];
an output layer connected to the reservoir layer and configured to apply a connection weight to a second signal which is output from the reservoir layer [Fig. 1 shows output layer 130 connecting to the reservoir layer of internal network 120; paragraph 0037, “an output-processing component of the device 100 may receive output signals from output nodes of the reservoir and apply weights Wout 132 to the output signals before passing them on to output channels of output layer 130. The output signals can be weighted using values Wout, 132 determined during training of the device 100”];
an evaluation circuit configured to calculate a distribution of connection weights in the output layer [Fig. 18, paragraph 0125, discloses the optimal output weights Wout that minimizing error between output y and target output y0 are calculated];
Lathrop in Fig. 1 also shows a feedback signal is received by the input layer 110 from the output layer 130.
Lathrop does not explicitly teach
a reservoir layer connected to the input layer and configured to generate a feature space including information of a first signal input from the input layer (emphasis added);
evaluate whether the distribution of connection weights is a prescribed distribution;
an adjustment circuit configured to change adjustment parameters for adjusting the first signal when the distribution of connection weights is not the prescribed distribution.
Pyragas teaches
a reservoir layer connected to the input layer and configured to generate a feature space including information of a first signal input from the input layer [page 2, Fig. 1, “The input vector u(t) ε Rd is mapped to the reservoir state space r(t) ε RN by the input weight matrix Win”];
evaluate whether the distribution of connection weights … [page 2, Fig. 1, “The input vector u(t) ε Rd is mapped to the reservoir state space r(t) ε RN by the input weight matrix Win … The reservoir output is v(t) = WToutr(t), where Wout is the output matrix whose elements are obtained during the training stage to get an approximate equality v(t) ≈ u(t)”; page 1, Col. 2, 2nd paragraph, “In this paper, we appeal to a reservoir computer approach … and uses linear regression to choose "output weights" that fit the network output to a set of "training data."”];
an adjustment circuit configured to change adjustment parameters for adjusting the first signal when the distribution of connection weights … [Fig. 1, page 3, Col.1, 2nd and 3rd paragraphs, “Our goal is to predict the signal u(t) corresponding to the output … We train the network by choosing the output layer matrix Wout so that the reservoir output v(t) approximates the input u(t) … To predict the input signal for times t > 0, we use both RCs shown in Fig. 1 … the output of the predicting RC allows us to estimate the predicted value of the input signal as
PNG
media_image1.png
28
230
media_image1.png
Greyscale
Fig. 1 also shows the input signal that provides to the reservoir layer comprises input u(t) and input weight Win which examiner interprets as the adjustment parameters].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified the integrated circuit device for reservoir computing of Lathrop to include evaluating the distribution of connection weights and adjusting the input signal based on the connection weights of Pyragas. Doing so would help determining the input signal corresponding to the output weights (Pyragas, page 3, Col. 1).
Lathrop and Pyragas do not explicitly teach
evaluate whether the distribution of connection weights is a prescribed distribution;
change adjustment parameters for adjusting the first signal when the distribution of connection weights is not the prescribed distribution.
Khatami teaches
evaluate whether the distribution of connection weights is a prescribed distribution [paragraph 0038, “Predicted values are generated at step 206. The predicted values are obtained using the trained predictive model(s). At 208, the MSE distributions are computed on the predicted values. As previously noted, the MSE for data is determined for a given window size, where the MSE data is calculated by comparison of the predicted value with the real value for the data … detect outliers by defining normal MSE as the data that lies within 99% of the MSE normal distribution within a window”; paragraph 0044, “The Kolmogorov-Smirnov (KS) approach is used in some embodiments of the invention for comparing the difference between two MSE distributions of normal data. Generally, this method is used to test whether two underlying one-dimensional probability distributions differ … The null hypothesis is H0, with both samples coming from a population with the same distribution. In KS, the null hypothesis is rejected at significant level α if Dn,m>Dn,m,α where Dn,m,α is the critical value” ; Fig. 7, paragraph 0056, discloses a process of using Kolmogorov-Smirnov (KS) approach to compare and determine if the difference between two MSE distributions; paragraph 0033, “The general idea is that if the model is trained well, then the MSE distribution for normal data (with exclusion of the outliers) should stay almost the same for the new upcoming data”; Fig. 5, paragraph 0053, “normal MSE 502 can be defined to be value that lie within 99% of the MSE normal distribution in the winmse”; It can be seen that Kolmogorov-Smirnov (KS) approach is used to test for normal distribution detection, where normality test may be based on a null hypothesis and a significance level, if the null hypothesis is rejected of normality (at significant level α), it means the data are not normally distributed, and examiner interprets a prescribed distribution as a normal distribution];
Lathrop (as modified) teaches the process of determining distribution of output weights, and adjusting the first signal based on the distribution of the output weights” (Pyragas, page 1, Col. 2, 2nd paragraph and Fig. 1, page 3, Col.1, 2nd and 3rd paragraphs); Lathrop (as modified), however, is silent of “evaluate whether the distribution of connection weights is a prescribed distribution”,
While Khatami teaches a process of determining MSE distribution (which relates directly to output weights), and using Kolmogorov-Smirnov (KS) approach, null hypothesis and a significance level to evaluate if the distribution is a normal distribution (prescribed distribution) in the winmse to further determining whether the model needs to update, therefore, the combination of Lathrop (as modified) and Khatami teaches the above claim limitation;
change adjustment parameters for adjusting the first signal when the distribution of connection weights is not the prescribed distribution [paragraphs 0043-0045, “For a well-trained model, the MSE distribution for normal data excluding the outliers should stay approximately the same for the new upcoming data … whenever the MSE distribution for normal data is changed … the model should get updated … The Kolmogorov-Smirnov (KS) approach is used in some embodiments of the invention for comparing the difference between two MSE distributions of normal data … The null hypothesis is H0, with both samples coming from a population with the same distribution. In KS, the null hypothesis is rejected at significant level α … The threshold parameter α is therefore employed from the KS comparison for detecting the necessity of updating a model”; Fig. 7, paragraph 0056, discloses a process of using Kolmogorov-Smirnov (KS) approach to compare the difference between two MSE distributions, “compares the MSE distributions on the normal data … This approach permits diagnosis of the necessity of updating a model. After comparing MSE normal distributions, if a model needs to get updated, then at 706, the approach stores new data and uses them for the model training purposes”; Fig. 9, paragraph 0062, “whenever the criteria for detecting the model drift is met … collecting data for the training processes in the memory heap … the training process can start training model using the new data”; It can be seen that based on the comparison of the MSE distributions using Kolmogorov-Smirnov (KS) approach to evaluate if the distribution is a normal distribution, and when model drift is detected (the distribution is not a prescribed distribution), updating the model using new data such as new input data and training data (adjusting first signal by changing adjustment parameters such as input data, training data, weight between the layers) from a memory heap and training the model with new data, wherein, paragraphs 0065 and 0074, “the trained model metadata will be stored in array 1002 (Array A) and array 1004 (Array B) … These two arrays will be stored in a heap … The trained parameters stored in the two arrays pertain in some embodiments to the weights between the layers in the recurrent neural network”, therefore, the reciting of Khatami teaches the above claim limitation].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified the integrated circuit device for reservoir computing of Lathrop to include evaluating whether the distribution is a prescribed distribution, and change adjustment parameters for adjusting the first signal when the distribution is not the prescribed distribution of Khatami. Doing so would help retraining the model with new data set to improve model performance (Khatami, 0035).
As per claim 2, Lathrop, Pyragas and Khatami teach the information processing device according to claim 1.
Khatami teaches
the prescribed distribution is a normal distribution [paragraph 0038, “Predicted values are generated at step 206. The predicted values are obtained using the trained predictive model(s). At 208, the MSE distributions are computed on the predicted values. As previously noted, the MSE for data is determined for a given window size, where the MSE data is calculated by comparison of the predicted value with the real value for the data … detect outliers by defining normal MSE as the data that lies within 99% of the MSE normal distribution within a window”; paragraph 0044, “The Kolmogorov-Smirnov (KS) approach is used in some embodiments of the invention for comparing the difference between two MSE distributions of normal data. Generally, this method is used to test whether two underlying one-dimensional probability distributions differ … The null hypothesis is H0, with both samples coming from a population with the same distribution. In KS, the null hypothesis is rejected at significant level α if Dn,m>Dn,m,α where Dn,m,α is the critical value” ; Fig. 7, paragraph 0056, discloses a process of using Kolmogorov-Smirnov (KS) approach to compare and determine if the difference between two MSE distributions; paragraph 0033, “The general idea is that if the model is trained well, then the MSE distribution for normal data (with exclusion of the outliers) should stay almost the same for the new upcoming data”; Fig. 5, paragraph 0053, “normal MSE 502 can be defined to be value that lie within 99% of the MSE normal distribution in the winmse”; It can be seen that Kolmogorov-Smirnov (KS) approach is used to test for normal distribution detection, where normality test may be based on a null hypothesis and a significance level, if the null hypothesis is rejected of normality (at significant level α), it means the data are not normally distributed];
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified the integrated circuit device for reservoir computing of Lathrop to include the prescribed distribution is a normal distribution of Khatami. Doing so would help determining whether the model needs to update based on the comparison of MSE normal distributions (Khatami, 0056).
As per claim 5, Lathrop, Pyragas and Khatami teach the information processing device according to claim 1.
Lathrop further teaches
parameters are connection weights which are multiplied by an input signal applied to the input layer [Fig. 1, paragraph 0053, “Input data u(t) can include, for example, input signals uk(t) (k=K). Input weight unit 322 can apply weights to input signals uk(t) by, for example, multiplying input signals uk(t) by weights Win 112 … to produce weight input signals”];
Khatami further teaches
the adjustment parameters are connection weights [paragraphs 0043-0045, “whenever the MSE distribution for normal data is changed … the model should get updated … The Kolmogorov-Smirnov (KS) approach is used in some embodiments of the invention for comparing the difference between two MSE distributions of normal data … The null hypothesis is H0, with both samples coming from a population with the same distribution. In KS, the null hypothesis is rejected at significant level α … The threshold parameter α is therefore employed from the KS comparison for detecting the necessity of updating a model”; Fig. 7, paragraph 0056, discloses a process of using Kolmogorov-Smirnov (KS) approach to compare the difference between two MSE distributions, “compares the MSE distributions on the normal data … This approach permits diagnosis of the necessity of updating a model. After comparing MSE normal distributions, if a model needs to get updated, then at 706, the approach stores new data and uses them for the model training purposes”; Fig. 9, paragraph 0062, “whenever the criteria for detecting the model drift is met … collecting data for the training processes in the memory heap … the training process can start training model using the new data”; It can be seen that based on the comparison of the MSE distributions using Kolmogorov-Smirnov (KS) approach to evaluate if the distribution is a normal distribution, when model drift is detected (the distribution is not a prescribed distribution), updating the model using new data such as new input data and training data (adjusting first signal by changing adjustment parameters such as input data, training data, weight between the layers) from a memory heap and training the model with new data, wherein, paragraphs 0065 and 0074, “the trained model metadata will be stored in array 1002 (Array A) and array 1004 (Array B) … These two arrays will be stored in a heap … The trained parameters stored in the two arrays pertain in some embodiments to the weights between the layers in the recurrent neural network”; Since Khatami teaches a process of using new data such as input data, training data and connection weights between the layers (including input layer) to retrain the model (updating/retrain the model with new data such as connection weights) based on the comparison using Kolmogorov-Smirnov approach, and Lathrop teaches connection weights which are multiplied by an input signal applied to the input, therefore, the combination of Lathrop and Khatami teaches the above claim limitation].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified the integrated circuit device for reservoir computing of Lathrop to include the adjustment parameters are connection weights of Khatami. Doing so would help updating the model by adjusting the connection weights to improve model performance (Khatami, 0033).
As per claim 8, Lathrop, Pyragas and Khatami teach the information processing device according to claim 1.
Pyragas further teaches
a distribution of the adjustment parameters is a uniform distribution [page 3, Col. 1, 1st paragraph, “The elements of each column of the input matrix Win are randomly chosen from a uniform distribution in [-σ, σ], where the parameter σ represents the scalar input strength”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified the integrated circuit device for reservoir computing of Lathrop to include a distribution of the adjustment parameters is a uniform distribution of Pyragas. Doing so would help preventing extreme outlier values and keeping signal variances stable across the layers.
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Lathrop et al. in view of Pyragas et al. in view of Khatami et al. and further in view of Sather et al. (US Patent 12,136,039).
As per claim 3, Lathrop, Pyragas and Khatami teach the information processing device according to claim 1.
Khatami teaches
the evaluation circuit evaluates whether the distribution of connection weights is a prescribed distribution at the time of updating the connection weights applied to the second signal [paragraph 0038, “Predicted values are generated at step 206. The predicted values are obtained using the trained predictive model(s). At 208, the MSE distributions are computed on the predicted values. As previously noted, the MSE for data is determined for a given window size, where the MSE data is calculated by comparison of the predicted value with the real value for the data”; paragraphs 0043-0045, “For a well-trained model, the MSE distribution for normal data excluding the outliers should stay approximately the same for the new upcoming data … whenever the MSE distribution for normal data is changed … the model should get updated … The Kolmogorov-Smirnov (KS) approach is used in some embodiments of the invention for comparing the difference between two MSE distributions of normal data”; Fig. 7, paragraph 0056, discloses a process of using Kolmogorov-Smirnov (KS) approach to compare the difference between two MSE distributions, “compares the MSE distributions on the normal data … This approach permits diagnosis of the necessity of updating a model. After comparing MSE normal distributions, if a model needs to get updated, then at 706, the approach stores new data and uses them for the model training purposes”; Fig. 9, paragraph 0062, “whenever the criteria for detecting the model drift is met … collecting data for the training processes in the memory heap … the training process can start training model using the new data”; paragraphs 0065 and 0074, “the trained model metadata will be stored in array 1002 (Array A) and array 1004 (Array B) … These two arrays will be stored in a heap … The trained parameters stored in the two arrays pertain in some embodiments to the weights between the layers in the recurrent neural network”. It can be seen that based on the comparison of the MSE distributions using Kolmogorov-Smirnov (KS) approach to evaluate if the distribution is a normal distribution, and when model drift is detected, updating the model using new data such as new input data, training data, weights between the layers (including output weights) from a memory heap and training the model with new data].
Lathrop, Pyragas and Khatami do not teach
the evaluation circuit evaluates the distribution … when a change in connection weight between before updating and after updating is less than or equal to a prescribed value (emphasis added).
Sather teaches
the evaluation circuit evaluates the distribution when a change in connection weight between before updating and after updating is less than or equal to a prescribed value [Col. 36, lines 28-46, “After the weights (and any other network parameters, such as biases) are updated, the process 1500 determines (at 1535) whether to perform additional training … the training system only stops training the network once the weights have changed by less than a threshold for a particular number of training iterations”; Since , Lathrop (as modified) teaches the system evaluates if the distribution is a normal distribution based on the comparison of the MSE distributions using Kolmogorov-Smirnov (KS) approach, and when model drift is detected, updating/retraining the model using new data such as new input data, training data, weights between the layers including output weights from a memory heap and training the model with new data (Khatami, paragraphs 0043-0045 and 0056), while Sather teaches retraining the model by updating the weights, and stops the training once the weights have changed by less than a threshold for a particular number of training iterations, therefore, the combination of Lathrop (as modified) and Sather teaches the above claim limitation].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified the integrated circuit device for reservoir computing of Lathrop to include the evaluation circuit evaluates the distribution when a change in connection weight between before updating and after updating is less than or equal to a prescribed value of Sather. Doing so would help determine when to stop training the network (Sather, Col. 36, lines 32-33).
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Lathrop et al. in view of Pyragas et al. in view of Khatami et al. and further in view of Chang et al. (US Pub. 2019/0302713).
As per claim 4, Lathrop, Pyragas and Khatami teach the information processing device according to claim 1.
Pyragas in Fig. 1, page 3, Col.1, 2nd and 3rd paragraphs, teaches a process of changing the adjustment parameters based on the output values.
Lathrop, Pyragas and Khatami do not teach
the adjustment circuit selects an optimal adjustment parameter out of the changed adjustment parameters when the number of times of change of the adjustment parameters reaches a prescribed number.
Chang teaches
the adjustment circuit selects an optimal adjustment parameter out of the changed adjustment parameters when the number of times of change of the adjustment parameters reaches a prescribed number [paragraphs 0085-0097, “The BP neural network is trained as follows … the parameters of the BP neural network is initialized … the initial weights of the neural network is obtained and the genetic algorithm is used to initialize the weights and thresholds, and when the genetic algorithm is executed for a generation, the new generation of weights is reinserted into the neural network, and the weights can be evolved … the output error values of the hidden layer and of the output layer are calculated … the connection weights and thresholds of all layers are constantly adjusted … the operations of selection, hybridization, mutation, and calculation of fitness values are performed repeatedly until the number of evolutions is reached and the optimal initial weights are obtained”; examiner interprets the weights of the neural network as the adjustment parameter].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified the integrated circuit device for reservoir computing of Lathrop to include selecting an optimal adjustment parameter out of the changed adjustment parameters when the number of times of change of the adjustment parameters reaches a prescribed number of Chang. Doing so would help training the neural network predictive control model with optimal training data (Chang, 0083).
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Lathrop et al. in view of Pyragas et al. in view of Khatami et al. and further in view of Baumstein et al. (US Pub. 2021/0318458).
As per claim 6, Lathrop, Pyragas and Khatami teach the information processing device according to claim 1.
Lathrop, Pyragas and Khatami do not teach
the adjustment parameters are filter coefficients of filters for selectively passing frequency components constituting an input signal applied to the input layer.
Baumstein teaches
the adjustment parameters are filter coefficients of filters for selectively passing frequency components constituting an input signal applied to the input layer [paragraph 0089, “pairs from the first group and the second group are provided to the neural network as training input (e.g., high frequency OBN data) … a measure of misfit between the pairs of input/output is computed, and then parameters of the neural network (e.g., coefficients of the convolution filters) are iteratively updated to minimize this misfit in order to perform the training of the neural network”; Since Lathrop (as modified) teaches, the model/neural network may be updated based on the comparison of MSE distributions (for example, using Kolmogorov-Smirnov approach to test for normality), the adjustment parameters of the neural network are adjusted using new data such as input data, training data, weights (Khatami, Fig. 7, paragraphs 0044-0045 and 0056), and Baumstein teaches the adjustment parameters of the neural network include filter coefficients for reducing misfit, therefore, the combination of Lathrop (as modified) and Baumstein teaches the above claim limitation].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified the integrated circuit device for reservoir computing of Lathrop to include the adjustment parameters are filter coefficients of filters for selectively passing frequency components constituting an input signal applied to the input layer of Baumstein. Doing so would help minimizing the misfit to perform the training of the neural network (Baumstein, 0089)
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Lathrop et al. in view of Pyragas et al. in view of Khatami et al. and further in view of Griffith et al. (US Pub. 2022/0383166).
As per claim 7, Lathrop, Pyragas and Khatami teach the information processing device according to claim 1.
Lathrop, Pyragas and Khatami do not explicitly teach
a distribution of the adjustment parameters is a normal distribution.
Griffith teaches
a distribution of the adjustment parameters is a normal distribution [Figs. 1 and 4, paragraphs 0028 and 0046, “The reservoir computing device 100 comprises an input layer 110, a reservoir 120, an output layer 130, and feedback 140. The input layer 110 provides one or more input signals (e.g., u(t)) to the reservoir 120. The input signals can be weighted using values determined during training of the reservoir computing device … For Win, randomly connect each node to each RC input with probability σ. The weight for each connection is drawn randomly from a normal distribution with mean 0 and variance ρ.sup.2.sub.in. Together, σ and ρ.sub.in are enough to generate a random instantiation of W.sub.in”; examiner interprets in put weights (Win) as the adjustment parameters].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified the integrated circuit device for reservoir computing of Lathrop to include a distribution of the adjustment parameters is a normal distribution of Griffith. Doing so would help optimizing a topology for reservoir computing (Griffith, abstract).
Prior Art
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
Kanazawa (US Pub. 2019/0130252) describes a self-organizing reservoir computing system.
Rao et al. (US Patent 10,162,378) describes a neuromorphic processor for signal denoising and separation.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TRI T NGUYEN whose telephone number is 571-272-0103. The examiner can normally be reached M-F, 8 AM-5 PM, (CT).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, OMAR FERNANDEZ can be reached at 571-272-2589. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TRI T NGUYEN/Examiner, Art Unit 2128
/OMAR F FERNANDEZ RIVAS/Supervisory Patent Examiner, Art Unit 2128
Prosecution Insights