DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-2, and 6-11 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Zhu et al. (US 20210397941 A1) (hereinafter Zhu).
Regarding claim 1, Zhu discloses:
A method comprising:
providing first data to a machine learning model to generate second data; [See Zhu, Figs. 1 and 2, ¶ 00042-0044, 0071-0072, 0076 discloses first data (201) being input to feature extractors 202 and 220 embodied as a neural network (such as a 3-layer fully-connected neural network), wherein outputs 206 and 222 constitute “second data”.]
determining errors based on the second data and target second data; [See Zhu, Fig. 2, ¶ 0031-0034, 0042-0044, 0075-0077 discloses computing an error (mean squared error) via a surrogate loss function.]
determining loss gradients based on the errors; [See Zhu, ¶ 0042, 0085 discloses that a machine learning model can be updated using gradients obtained from a learned surrogate loss function.]
updating running sums of prior loss gradients by adding the loss gradients to the running sums; and [See Zhu, ¶ 0042-0044, 0067 discloses a neuron generating output signals dependent upon accumulated inputs and weighted signals can be propagated over successive layers of the network from an input to an output layer.]
updating model parameters of the machine learning model based on the updated running sums. [See Zhu, ¶ 0042-0044 discloses updating a model using gradients obtained from a learned surrogate loss function. Further, that a neuron generates output signals dependent on its accumulated inputs, and weighted signals can be propagated over successive layers of the network from an input to an output neuron layer. An artificial neural network machine learning model can undergo a training phase in which the sets of weights associated with respective neuron layers are determined. The network is exposed to a set of training data, in an iterative training scheme in which the weights are repeatedly updated as the network “learns” from the training data.]
Regarding claim 2, Zhu discloses all the limitations of claim 1.
Zhu discloses:
wherein updating model parameters of the machine learning model based on the running sums includes updating model parameters of the machine learning model based on a combination of the loss gradients and the updated running sums. [See Zhu, ¶ 0042-0044 discloses updating a model using gradients obtained from a learned surrogate loss function. Further, that a neuron generates output signals dependent on its accumulated inputs, and weighted signals can be propagated over successive layers of the network from an input to an output neuron layer. An artificial neural network machine learning model can undergo a training phase in which the sets of weights associated with respective neuron layers are determined. The network is exposed to a set of training data, in an iterative training scheme in which the weights are repeatedly updated as the network “learns” from the training data.]
Regarding claim 6, Zhu discloses all the limitations of claim 1.
Zhu discloses:
wherein the machine learning model includes a neural network model. [See Zhu, ¶ 0042-0044 discloses a feature extractor or predictor embodied as a neural network model.]
Regarding claim 7, Zhu discloses all the limitations of claim 6.
Zhu discloses:
wherein the model parameters include weight elements, and the loss gradients are determined based on the first data and the errors. [See Zhu, ¶ 0042-0044, 0085 discloses a machine learning model can be updated using gradients obtained from the learned surrogate loss function. In an embodiment, the surrogate loss function can be learned via a neural network parameterized by a weight.]
Regarding claim 8, Zhu discloses all the limitations of claim 6.
Zhu discloses:
wherein the model parameters include bias parameters, and the loss gradients are determined based on the errors. [See Zhu, ¶ 0042-0044, 0085 discloses a machine learning model can be updated using gradients obtained from the learned surrogate loss function. In an embodiment, the surrogate loss function can be learned via a neural network parameterized by a weight. It is understood that a weight parameter is a way to influence/train or “bias” parameters.]
Regarding claim 9, Zhu discloses all the limitations of claim 6.
Zhu discloses:
wherein the neural network model includes at least one of: a convolutional neural network, a deep neural network, or an autoencoder. [See Zhu, ¶ 0042-0044 discloses an artificial neural network (ANN) or neural network (NN) is a machine learning model, which can be trained to predict or classify input data. An artificial neural network can include a succession of layers of neurons, which are interconnected so that output signals of neurons in one layer are weighted and transmitted to neurons in the next layer. One of ordinary skill would readily recognize that a deep neural network is an artificial neural network possessing multiple layers between the input and output layers.]
Regarding claim 10, Zhu discloses all the limitations of claim 1.
Zhu discloses:
wherein the machine learning model processes a batch of input data at a time, and the first data includes a single batch of the input data. [See Zhu, ¶ 0042-0046 discloses processing input data by batch size, wherein the batch size is selectable, and particularly disclosing assuming a batch size equal to 1.]
Regarding claim 11, Zhu discloses all the limitations of claim 1.
Zhu discloses:
further comprising updating the machine learning model per batch of input data. [See Zhu, ¶ 0042-0046 discloses processing input data by batch size, wherein the batch size is selectable, and particularly disclosing assuming a batch size equal to 1. Further, updating a task-oriented estimator, and a prediction model according to learned surrogate loss.]
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 3, 12, 14-17, and 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhu et al. (US 20210397941 A1) (hereinafter Zhu) in view of Sumioka (US 20220131480 A1) (hereinafter Sumioka).
Regarding claim 3, Zhu discloses all the limitations of claim 2.
Zhu does not appear to explicitly disclose:
further comprising determining the combination using a proportional integral controller.
However, Sumioka discloses:
further comprising determining the combination using a proportional integral controller. [See Sumioka, ¶ 0007, 0079-0081, 0090, 0102-0107 discloses calculating an average of square error gradients of weights in a machine learning model. Further, performing adaptive control by acquiring learning data using a learning model the performance of which has been ensured to some extent or using a different controller such as a proportional-integral-derivative (PID) controller.]
It would have been obvious to the person having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention disclosed by Zhu to add the teachings of Sumioka with respect to proportional-integral controllers in order to enable finer adjustment of a control system.
Regarding claim 12, Zhu discloses all the limitations of claim 1.
Zhu discloses:
the prior loss gradients are generated from prior first data from the sensor. [See Zhu, ¶ 0040, 0042-0046, discloses warming up a prediction model using a warm-up loss function.]
Sumioka discloses:
wherein the first data is provided by a sensor, the method is performed by a device including the sensor, and [See Sumioka, ¶ 0113-0115 discloses an environment sensor which detects an environmental condition. Then, when a change in environment has been detected by the environment sensor, the learned model can be subjected to machine learning. The environment sensor can be configured to be at least one of a temperature sensor and a humidity sensor.]
It would have been obvious to the person having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention disclosed by Zhu to add the teachings of Sumioka with respect to proportional-integral controllers in order to enable finer adjustment of a control system.
Regarding claim 14, Zhu discloses:
An integrated circuit comprising: [See Zhu, ¶ 0086 discloses that one or more hardware processors 602 such as a central processing unit (CPU), a graphic process unit (GPU), and/or a Field Programmable Gate Array (FPGA), an application specific integrated circuit (ASIC), and/or another processor, may be coupled with a memory device 604, and generate a task-based prediction model.]
a memory configured to store data and instructions; and [See Zhu, 0085-0086 discloses that a memory device 604 may include random access memory (RAM), read-only memory (ROM) or another memory device, and may store data and/or processor instructions for implementing various functionalities associated with the methods and/or systems described
a processor configured to execute the instructions to: receive first data via the sensor interface; [See Zhu, 0085-0086 discloses that one or more hardware processors 602 may receive training data, receive contextual information associated with a task-based criterion, and train a machine learning model using the training data, wherein a loss function computed during training of the machine learning model integrates the task-based criterion, and wherein minimizing the loss function during training iterations includes minimizing the task-based criterion.
receive, from the memory, at least a subset of the data representing a machine learning model, model parameters of the machine learning model, and running sums of prior loss gradients; [See Zhu, ¶ 0042-0044, 0067 discloses a neuron generating output signals dependent upon accumulated inputs and weighted signals can be propagated over successive layers of the network from an input to an output layer.]
generate second data by providing the first data to the machine learning model; [See Zhu, Figs. 1 and 2, ¶ 00042-0044, 0071-0072, 0076 discloses first data (201) being input to feature extractors 202 and 220 embodied as a neural network (such as a 3-layer fully-connected neural network), wherein outputs 206 and 222 constitute “second data”.]
determine errors based on the second data and target second data; [See Zhu, Fig. 2, ¶ 0031-0034, 0042-0044, 0075-0077 discloses computing an error (mean squared error) via a surrogate loss function.]
determine loss gradients based on the errors; [See Zhu, ¶ 0042, 0085 discloses that a machine learning model can be updated using gradients obtained from a learned surrogate loss function.]
update the running sums based on adding the loss gradients to the running sums; [See Zhu, ¶ 0042-0044, 0067 discloses a neuron generating output signals dependent upon accumulated inputs and weighted signals can be propagated over successive layers of the network from an input to an output layer.]
update the model parameters based on the updated running sums; and [See Zhu, ¶ 0042-0044 discloses updating a model using gradients obtained from a learned surrogate loss function. Further, that a neuron generates output signals dependent on its accumulated inputs, and weighted signals can be propagated over successive layers of the network from an input to an output neuron layer. An artificial neural network machine learning model can undergo a training phase in which the sets of weights associated with respective neuron layers are determined. The network is exposed to a set of training data, in an iterative training scheme in which the weights are repeatedly updated as the network “learns” from the training data.]
store the updated model parameters and the updated running sums in the memory. [See Zhu, ¶ 0042-0046, 0076, 0085-0086 discloses updating machine learning model parameters, and recites storing elements in various memory devices.]
Sumioka discloses:
a sensor interface; [See Sumioka, ¶ 0113-0115 discloses when a change in environment has been detected by the environment sensor, the learned model can be subjected to machine learning.]
It would have been obvious to the person having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention disclosed by Zhu to add the teachings of Sumioka with respect to proportional-integral controllers in order to enable finer adjustment of a control system.
Regarding claim 15, Zhu in view of Sumioka discloses all the limitations of claim 14.
Zhu discloses:
wherein the machine learning model is configured to process a batch of input data at a time, and the first data includes a single batch of the input data. [See Zhu, ¶ 0042-0046 discloses processing input data by batch size, wherein the batch size is selectable, and particularly disclosing assuming a batch size equal to 1.]
Regarding claim 16, Zhu in view of Sumioka discloses all the limitations of claim 14.
Zhu discloses:
wherein the processor is configured to execute the instructions to update the model parameters of the machine learning model based on a combination of the loss gradients and the updated running sums. [See Zhu, ¶ 0042-0044 discloses updating a model using gradients obtained from a learned surrogate loss function. Further, that a neuron generates output signals dependent on its accumulated inputs, and weighted signals can be propagated over successive layers of the network from an input to an output neuron layer. An artificial neural network machine learning model can undergo a training phase in which the sets of weights associated with respective neuron layers are determined. The network is exposed to a set of training data, in an iterative training scheme in which the weights are repeatedly updated as the network “learns” from the training data.]
Regarding claim 17, Zhu in view of Sumioka discloses all the limitations of claim 16.
Sumioka discloses:
wherein the processor is configured to execute the instructions to implement a proportional integral controller and determine the combination using the proportional integral controller. [See Sumioka, ¶ 0007, 0079-0081, 0090, 0102-0107 discloses calculating an average of square error gradients of weights in a machine learning model. Further, performing adaptive control by acquiring learning data using a learning model the performance of which has been ensured to some extent or using a different controller such as a proportional-integral-derivative (PID) controller.]
It would have been obvious to the person having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention disclosed by Zhu to add the teachings of Sumioka with respect to proportional-integral controllers in order to enable finer adjustment of a control system.
Regarding claim 19, Zhu in view of Sumioka discloses all the limitations of claim 14.
Zhu discloses:
wherein the processor includes a machine learning hardware accelerator. [See Zhu, ¶ 0047 discloses experimental models can be trained with a training process performed for a number of epochs, for example, 50 epochs, for example, using a batch size of 1024. An Adam optimizer can be used with a learning rate of 3e-5, and early stopping can be employed to accelerate the training process and prevent overfitting.]
Regarding claim 20, Zhu in view of Sumioka discloses all the limitations of claim 14.
Zhu discloses:
wherein the machine learning model includes at least one of: a convolutional neural network, a deep neural network, or an autoencoder. [See Zhu, ¶ 0042-0044 discloses an artificial neural network (ANN) or neural network (NN) is a machine learning model, which can be trained to predict or classify input data. An artificial neural network can include a succession of layers of neurons, which are interconnected so that output signals of neurons in one layer are weighted and transmitted to neurons in the next layer. One of ordinary skill would readily recognize that a deep neural network is an artificial neural network possessing multiple layers between the input and output layers.]
Claim(s) 5 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Zhu et al. (US 20210397941 A1) (hereinafter Zhu) in view of Hartman et al. (US 20070282766 A1) (Hartman).
Regarding claim 5, Zhu discloses all the limitations of claim 1.
Zhu does not appear to disclose:
further comprising clamping the running sums.
However, Hartman discloses:
further comprising clamping the running sums. [See Hartman, ¶ 0299-0301 discloses machine learning model training and operated using input, output, and training input data scaled within a fixed range. Particularly, that sanity checks may be used in the method of one embodiment of the present invention to prevent erroneous training, prediction, and control. Whenever any data value fails to pass the sanity checks, the data may be clamped at the limit(s), or the operation/control may be disabled.]
It would have been obvious to the person having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention disclosed by Zhu to add the teachings of Hartman in order to enable increased robustness of training within a machine learning model.
Regarding claim 13, Zhu discloses all the limitations of claim 1.
Hartman discloses:
further comprising performing a fault detection operation based on the second data. [See Hartman, ¶ 0298 discloses that sanity checks on the data being specified may be specified by the user using steps and/or modules 3212, 3214 and 3216 as follows. The user may specify a high limit value using step or module 3212, and may specify a low limit value using step or module 3214. Since sensors sometimes fail, for example, this sanity check may allow the user to prevent the system and method of one embodiment of the present invention from using false data from a failed sensor. Other examples of faulty data may also be detected by setting these limits.]
It would have been obvious to the person having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention disclosed by Zhu to add the teachings of Hartman in order to enable increased robustness of training within a machine learning model.
Claim(s) 21 is rejected under 35 U.S.C. 103 as being unpatentable over Zhu in view of Sumioka in further view of Hartman.
Regarding claim 21, Zhu in view of Sumioka discloses all the limitations of claim 14.
Zhu in view of Sumioka does not appear to explicitly disclose:
wherein the processor is configured to execute the instructions to provide a fault detection indication based on the second data.
However, Hartman discloses:
wherein the processor is configured to execute the instructions to provide a fault detection indication based on the second data.[See Hartman, ¶ 0298 discloses that sanity checks on the data being specified may be specified by the user using steps and/or modules 3212, 3214 and 3216 as follows. The user may specify a high limit value using step or module 3212, and may specify a low limit value using step or module 3214. Since sensors sometimes fail, for example, this sanity check may allow the user to prevent the system and method of one embodiment of the present invention from using false data from a failed sensor. Other examples of faulty data may also be detected by setting these limits.]
It would have been obvious to the person having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention disclosed by Zhu to add the teachings of Hartman in order to enable increased robustness of training within a machine learning model.
Allowable Subject Matter
Claims 4 and 18 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PATRICK E DEMOSKY whose telephone number is (571)272-8799. The examiner can normally be reached Monday - Friday 7-4 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jamie Atala can be reached at 5712727384. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PATRICK E DEMOSKY/ Primary Examiner, Art Unit 2486
Prosecution Insights