DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is responsive to the Application filed on January 22, 2024. Claims 1-19 are pending in the case. Claims 1, 7, and 14 are the independent claims.
This action is non-final.
Claim Objections
Claim 13 is objected to because of the following informalities: Claim 13 recites dependency upon claim 5, where dependency upon claim 7 was perhaps intended. Appropriate correction is required.
Claim Rejections – 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-9 and 11-19 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea (mental steps) without significantly more. This judicial exception is not integrated into a practical application because any additional elements amount to implementing the abstract idea on a generic computer. The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Regarding independent claims 1, 7, and 14, and relying on the evaluation flowchart in MPEP 2106:
Step 1 (Is the claim to a process, machine, manufacture, or composition of matter?): Yes. Claim 1 is a device (machine). Claim 7 is a method (process). Claim 14 is a system (machine).
Claim 1 - Step 2a Prong One (Does the claim recite an abstract idea?): Yes.
Claim 1 recites:
perform a simulated annealing process on a plurality of neuron weights (a mental process, including a mental process involving a mathematical calculation, including a mental process performed using a physical aid such as pen and paper);
extract a plurality of features from the runtime dataset (a mental process of observation and evaluation);
obtain a confidence level (a mental process of determination);
output a prediction indication based on the confidence level (a mental process of determination).
Under the broadest reasonable interpretation, these steps may be performed mentally, using mental observation and mental determination, including by a human using a physical aid such as pen and paper, including a human mentally performing observations and mentally performing mathematical calculations, and therefore correspond to the Mental Processes grouping.
Claim 1 - Step 2a Prong Two (Does the claim recite additional elements that integrate the judicial exception into a practical application?): No. Claim 1 additionally recites:
A device, comprising, an electronic processor having: a set of input pins; a set of output pins; and a layout of circuit gates implementing an optimized neural network model…on a plurality of neuron weights in a trained neural network model (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f));
wherein the layout causes the electronic processor to:… (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f))
when receiving a runtime dataset via the set of input pins (insignificant extra-solution activity as discussed in MPEP 2106.05(g));
apply the plurality of features to the optimized neural network model to obtain a confidence level (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f)); and
output a prediction…via the output pins (insignificant extra-solution activity as discussed in MPEP 2106.05(g)).
Therefore, in view of the considerations set forth in MPEP 2106.04(d), 2106.05(a)-(c) and (e)-(h), the additional elements as disclosed above alone or in combination do not integrate the judicial exception into a practical application as they are mere insignificant extra solution activity, combined with implementing the abstract idea using generic computer components.
Claim 1 - Step 2b (Does the claim recite additional elements that amount to siqnificantly more than the judicial exception): No. Relying on the same analysis as Step 2a Prong Two (see MPEP 2106.05.I.A: Limitations that the courts have found not to be enough to qualify as “significantly more” when recited in a claim with a judicial exception include:…Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, e.g., a limitation indicating that a particular function such as creating and maintaining electronic records is performed by a computer, as discussed in Alice Corp., 573 U.S. at 225-26, 110 USPQ2d at 1984 (see MPEP 2106.05(f));…Simply appending well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception...; Adding insignificant extra-solution activity to the judicial exception, as discussed in MPEP 2106.05(g);…)), claim 1 does recite any additional elements that amount to significantly more than the abstract idea. As discussed above, Claim 1 recites:
A device, comprising, an electronic processor having: a set of input pins; a set of output pins; and a layout of circuit gates implementing an optimized neural network model…on a plurality of neuron weights in a trained neural network model (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f));
wherein the layout causes the electronic processor to:… (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f))
when receiving a runtime dataset via the set of input pins (insignificant extra-solution activity as discussed in MPEP 2106.05(g), where the insignificant extra-solution activity of receiving data can further be re-evaluated in Step 2B as a well understood, routine, and conventional activity MPEP 2106.05(d) of receiving or transmitting data over a network e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362);
apply the plurality of features to the optimized neural network model to obtain a confidence level (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f)); and
output a prediction…via the output pins (insignificant extra-solution activity as discussed in MPEP 2106.05(g), where the insignificant extra-solution activity outputting the prediction can further be re-evaluated in Step 2B as a well understood, routine, and conventional activity MPEP 2106.05(d) of receiving or transmitting data over a network e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362).
The additional elements as discussed above, in combination with the abstract idea, are not sufficient to amount to significantly more than the judicial exception as they are well, understood, routine and conventional activity as disclosed in combination with generic computer functions and components used to implement the abstract idea.
Claim 7 - Step 2a Prong One (Does the claim recite an abstract idea?): Yes.
Claim 7 recites:
hardware optimization (a mental process of determination).
performing a simulated annealing process for the plurality of neuron weights (a mental process, including a mental process involving a mathematical calculation, including a mental process performed using a physical aid such as pen and paper);
generating a plurality of new weights for one of the plurality of neuron layers (a mental process, including a mental process involving a mathematical calculation, including a mental process performed using a physical aid such as pen and paper);
generating an optimized circuit layout (a mental process of determination).
Under the broadest reasonable interpretation, these steps may be performed mentally, using mental observation and mental determination, including by a human using a physical aid such as pen and paper, including a human mentally performing observations and mentally performing mathematical calculations, and therefore correspond to the Mental Processes grouping.
Claim 7 - Step 2a Prong Two (Does the claim recite additional elements that integrate the judicial exception into a practical application?): No. Claim 7 additionally recites:
obtaining a trained neural network model (insignificant extra-solution activity as discussed in MPEP 2106.05(g)).
the trained neural network model comprising a plurality of neurons, the plurality of neurons comprising a plurality of neuron layers and a plurality of neuron weights (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f));
retraining the trained neural network model using the plurality of new weights (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f));
obtaining an updated plurality of neuron weights model (insignificant extra-solution activity as discussed in MPEP 2106.05(g))
obtaining an optimized neural network model using the updated plurality of neuron weights (insignificant extra-solution activity as discussed in MPEP 2106.05(g) with respect to obtaining, mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f) with respect to the obtaining being via a learning/training process);
for hardware that implements the optimized neural network model obtained using the updated plurality of neuron weights (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f)).
Therefore, in view of the considerations set forth in MPEP 2106.04(d), 2106.05(a)-(c) and (e)-(h), the additional elements as disclosed above alone or in combination do not integrate the judicial exception into a practical application as they are mere insignificant extra solution activity, combined with implementing the abstract idea using generic computer components.
Claim 7 - Step 2b (Does the claim recite additional elements that amount to siqnificantly more than the judicial exception): No. Relying on the same analysis as Step 2a Prong Two (see MPEP 2106.05.I.A: Limitations that the courts have found not to be enough to qualify as “significantly more” when recited in a claim with a judicial exception include:…Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, e.g., a limitation indicating that a particular function such as creating and maintaining electronic records is performed by a computer, as discussed in Alice Corp., 573 U.S. at 225-26, 110 USPQ2d at 1984 (see MPEP 2106.05(f));…Simply appending well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception...; Adding insignificant extra-solution activity to the judicial exception, as discussed in MPEP 2106.05(g);…)), claim 7 does recite any additional elements that amount to significantly more than the abstract idea. As discussed above, Claim 7 recites:
obtaining a trained neural network model (insignificant extra-solution activity as discussed in MPEP 2106.05(g), where the insignificant extra-solution activity of obtaining the model/data can further be re-evaluated in Step 2B as a well understood, routine, and conventional activity MPEP 2106.05(d) of receiving or transmitting data over a network e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362).
the trained neural network model comprising a plurality of neurons, the plurality of neurons comprising a plurality of neuron layers and a plurality of neuron weights (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f));
retraining the trained neural network model using the plurality of new weights (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f));
obtaining an updated plurality of neuron weights model (insignificant extra-solution activity as discussed in MPEP 2106.05(g), where the insignificant extra-solution activity of obtaining the weights/data can further be re-evaluated in Step 2B as a well understood, routine, and conventional activity MPEP 2106.05(d) of receiving or transmitting data over a network e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362)
obtaining an optimized neural network model using the updated plurality of neuron weights (insignificant extra-solution activity as discussed in MPEP 2106.05(g) with respect to obtaining, where the insignificant extra-solution activity of obtaining the model can further be re-evaluated in Step 2B as a well understood, routine, and conventional activity MPEP 2106.05(d) of receiving or transmitting data over a network e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362; mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f) with respect to the obtaining being via a learning/training process);
for hardware that implements the optimized neural network model obtained using the updated plurality of neuron weights (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f)).
The additional elements as discussed above, in combination with the abstract idea, are not sufficient to amount to significantly more than the judicial exception as they are well, understood, routine and conventional activity as disclosed in combination with generic computer functions and components used to implement the abstract idea.
Claim 14 - Step 2a Prong One (Does the claim recite an abstract idea?): Yes.
Claim 14 recites:
hardware optimization (a mental process of determination).
generate a plurality of new weights for one of the plurality of neuron layers (a mental process, including a mental process involving a mathematical calculation, including a mental process performed using a physical aid such as pen and paper);
generate an optimized circuit layout (a mental process of determination).
Under the broadest reasonable interpretation, these steps may be performed mentally, using mental observation and mental determination, including by a human using a physical aid such as pen and paper, including a human mentally performing observations and mentally performing mathematical calculations, and therefore correspond to the Mental Processes grouping.
Claim 14 - Step 2a Prong Two (Does the claim recite additional elements that integrate the judicial exception into a practical application?): No. Claim 14 additionally recites:
the system comprising: an electronic processor, and a non-transitory computer-readable medium storing machine-executable instructions, which, when executed by the electronic processor, cause the electronic processor to weights (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f));
obtain a trained neural network model (insignificant extra-solution activity as discussed in MPEP 2106.05(g)).
the trained neural network model comprising a plurality of neurons, the plurality of neurons comprising a plurality of neuron layers and a plurality of neuron weights (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f));
retrain the trained neural network model using the plurality of new weights (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f));
obtain an optimized neural network model using the updated plurality of neuron weights (insignificant extra-solution activity as discussed in MPEP 2106.05(g) with respect to obtaining, mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f) with respect to the obtaining being via a learning/training process);
for hardware that implements the optimized neural network model obtained using the updated plurality of neuron weights (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f)).
Therefore, in view of the considerations set forth in MPEP 2106.04(d), 2106.05(a)-(c) and (e)-(h), the additional elements as disclosed above alone or in combination do not integrate the judicial exception into a practical application as they are mere insignificant extra solution activity, combined with implementing the abstract idea using generic computer components.
Claim 14 - Step 2b (Does the claim recite additional elements that amount to significantly more than the judicial exception): No. Relying on the same analysis as Step 2a Prong Two (see MPEP 2106.05.I.A: Limitations that the courts have found not to be enough to qualify as “significantly more” when recited in a claim with a judicial exception include:…Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, e.g., a limitation indicating that a particular function such as creating and maintaining electronic records is performed by a computer, as discussed in Alice Corp., 573 U.S. at 225-26, 110 USPQ2d at 1984 (see MPEP 2106.05(f));…Simply appending well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception...; Adding insignificant extra-solution activity to the judicial exception, as discussed in MPEP 2106.05(g);…)), claim 7 does recite any additional elements that amount to significantly more than the abstract idea. As discussed above, Claim 14 recites:
the system comprising: an electronic processor, and a non-transitory computer-readable medium storing machine-executable instructions, which, when executed by the electronic processor, cause the electronic processor to weights (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f));
obtain a trained neural network model (insignificant extra-solution activity as discussed in MPEP 2106.05(g), where the insignificant extra-solution activity of obtaining the model/data can further be re-evaluated in Step 2B as a well understood, routine, and conventional activity MPEP 2106.05(d) of receiving or transmitting data over a network e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362).
the trained neural network model comprising a plurality of neurons, the plurality of neurons comprising a plurality of neuron layers and a plurality of neuron weights (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f));
retrain the trained neural network model using the plurality of new weights (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f));
obtain an optimized neural network model using the updated plurality of neuron weights (insignificant extra-solution activity as discussed in MPEP 2106.05(g) with respect to obtaining, where the insignificant extra-solution activity of obtaining the model can further be re-evaluated in Step 2B as a well understood, routine, and conventional activity MPEP 2106.05(d) of receiving or transmitting data over a network e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362; mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f) with respect to the obtaining being via a learning/training process);
for hardware that implements the optimized neural network model obtained using the updated plurality of neuron weights (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f)).
The additional elements as discussed above, in combination with the abstract idea, are not sufficient to amount to significantly more than the judicial exception as they are well, understood, routine and conventional activity as disclosed in combination with generic computer functions and components used to implement the abstract idea.
Regarding dependent claims 2, 8, and 15:
Step 2a Prong One: incorporates the rejection of claims 1, 7, and 14. Claims 2, 8, and 15 further recite wherein the simulated annealing process comprises determining a perturbation value, the perturbation value proportional to a plurality of annealing temperatures (a mental process of determination).
Step 2a Prong Two: the claims recite no additional limitations.
Step 2b: the claims recite no additional limitations.
Regarding dependent claims 3 and 16:
Step 2a Prong One: incorporates the rejection of claims 2 and 15; the claims further recite wherein the perturbation value is a percentage of weight to be removed from each of the plurality of neural weights (a mental process of determination).
Step 2a Prong Two: the claims do not recite any other limitations in addition to the abstract idea discussed above.
Step 2b: the claims do not recite any other limitations in addition to the abstract idea discussed above.
Regarding dependent claims 4 and 17:
Step 2a Prong One: incorporates the rejection of claims 3 and 16; the claims further recite wherein the simulated annealing process further comprises rounding each of the plurality of neural weights to an integer value (a mental process of determination).
Step 2a Prong Two: the claims do not recite any other limitations in addition to the abstract idea discussed above.
Step 2b: the claims do not recite any other limitations in addition to the abstract idea discussed above.
Regarding dependent claim 5:
Step 2a Prong One: incorporates the rejection of claim 1.
Step 2a Prong Two: the claim additionally recites wherein the runtime dataset comprises a heart disease dataset, and wherein the confidence level comprises a possibility indication of heart disease in a patient (a field of use and technological environment as discussed in MPEP 2106.05(h)).
Step 2b: the claim additionally recites wherein the runtime dataset comprises a heart disease dataset, and wherein the confidence level comprises a possibility indication of heart disease in a patient (a field of use and technological environment as discussed in MPEP 2106.05(h)).
Regarding dependent claim 6:
Step 2a Prong One: incorporates the rejection of claim 1.
Step 2a Prong Two: the claim additionally recites wherein the runtime dataset comprises a breast cancer dataset, and wherein the confidence level comprises a possibility indication of breast cancer in a patient (a field of use and technological environment as discussed in MPEP 2106.05(h)).
Step 2b: the claim additionally recites wherein the runtime dataset comprises a breast cancer dataset, and wherein the confidence level comprises a possibility indication of breast cancer in a patient (a field of use and technological environment as discussed in MPEP 2106.05(h)).
Regarding dependent claim 9:
Step 2a Prong One: incorporates the rejection of claim 7.
Step 2a Prong Two: the claim additionally recites providing the optimized circuit layout to a fabrication facility (insignificant extra-solution activity as discussed in MPEP 2106.05(g)).
Step 2b: the claims additionally recite providing the optimized circuit layout to a fabrication facility (insignificant extra-solution activity as discussed in MPEP 2106.05(g), where the insignificant extra-solution activity of providing the layout can further be re-evaluated in Step 2B as a well understood, routine, and conventional activity MPEP 2106.05(d) of receiving or transmitting data over a network e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362).
Regarding dependent claims 11 and 18:
Step 2a Prong One: incorporates the rejection of claims 7 and 14.
Step 2a Prong Two: the claims additionally recite wherein the trained neural network model further comprises a feed forward neural network model (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f)).
Step 2b: the claims additionally recite wherein the trained neural network model further comprises a feed forward neural network model (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f)).
Regarding dependent claims 12 and 19:
Step 2a Prong One: incorporates the rejection of claims 8 and 18.
Step 2a Prong Two: the claims additionally recite wherein the trained neural network model further comprises a multilayer perceptron neural network model (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f)).
Step 2b: the claims additionally recite wherein the trained neural network model further comprises a multilayer perceptron network model (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f)).
Regarding dependent claim 13:
Step 2a Prong One: incorporates the rejection of claim 5.
Step 2a Prong Two: the claim additionally recites wherein the trained neural network model comprises an input layer of the plurality of neurons, a hidden layer of the plurality of neurons, and an output layer of the plurality of neurons (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f))
Step 2b: the claim additionally recites wherein the trained neural network model comprises an input layer of the plurality of neurons, a hidden layer of the plurality of neurons, and an output layer of the plurality of neurons (mere instructions to apply the exception using generic computer components as discussed in MPEP 2106.05(f)).
Therefore, in view of the considerations set forth in MPEP 2106.04(d), 2106.05(a)-(c) and (e)-(h), the additional elements as recited in the dependent claims discussed above alone or in combination do not integrate the judicial exception into a practical application as they are mere insignificant extra solution activity, combined with implementing the abstract idea using generic computer components, and limitations describing a field of use or technological environment. The additional elements as discussed above, in combination with the abstract idea, are not sufficient to amount to significantly more than the judicial exception as they are well, understood, routine and conventional activity as disclosed in combination with generic computer functions and components used to implement the abstract idea, and limitations describing a field of use or technological environment.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 14 and 15 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 14 recites, on lines 10-11, “the updated plurality of neuron weights.” This limitation lacks antecedent basis. Prior to this limitation, the claim recites “generate a plurality of new weights” and “the plurality of new weights.” However, the claim does not recite any step of updating a plurality of neuron weights, and it is unclear whether the recited “updated plurality of neuron weights” is the same, or different, from the “plurality of new weights.”
Claim 15 recites, “the simulated annealing process.” This limitation lacks antecedent basis. There is no recitation of any simulated annealing process in either of claim 15 or claim 14, upon which claim 15 depends.
Claim Rejections – 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 14, 18, and 19 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Timofejevs (US 12327182 B2).
With respect to claim 14, Timofejevs teaches a system for hardware optimization, the system comprising: an electronic processor, and a non-transitory computer-readable medium storing machine-executable instructions, which, when executed by the electronic processor, cause the electronic processor (e.g. col. 19 lines 1-10, computer system having processors, memory and display, programs including instructions for performing described methods; storage medium storing programs for execution by computer system having processors, memory, and display, the programs including instructions for performing described methods) to:
obtain a trained neural network model comprising a plurality of neurons, the plurality of neurons comprising a plurality of neuron layers and a plurality of neuron weights (e.g. col. 3 lines 8-9, process begins with a trained neural network; col. 3 lines 32-33, obtaining neural network topology and weights of trained neural network; col. 3 lines 55-58, neural network topology includes layers of neurons; col. 21 line 24, Fig. 1A, showing that the process begins with an obtained trained neural network; col. 22 lines 22-24, Fig. 1C, user develops and trains neural network 164 and inputs it 166 to SDK 180);
generate a plurality of new weights for one of the plurality of neuron layers (e.g. col. 3 lines 25-26, new weights of the retrained neural network; col. 3 lines 50-51, obtaining new weights for the trained neural network; col. 22 lines 53-54, Fig. 1C, process includes recalculating the weights, since pruning includes retraining of the whole network;);
retrain the trained neural network model using the plurality of new weights (e.g. col. 3 lines 25-26, retrained neural network; col. 21 line 57, Fig. 1A, the trained neural network is retrained 142 (shown as 124 in Fig. 1A); col. 22 lines 29-30, retraining 182 the neural net to obtain an updated neural net; col. 22 lines 38-54, if estimated error exceeds threshold, prompting user to reconfigure, redevelop, or retrain the neural network; process is iterated multiple times until error reduced below threshold; process includes recalculating the weights, since pruning includes retraining of the whole network; i.e. the processes shown in Figs. 1A and 1C include iteratively retraining a neural network multiple times, including recalculating new weights for use in retraining the neural network);
obtain an optimized neural network model using the updated plurality of neuron weights (e.g. col. 3 lines 25-26, new weights of the retrained neural network; col. 21 lines 40-44, Fig. 1A, generating schematic model for implementing analog neural network 104 based on the weights 106; optimizing resistance values/weights 106 to form optimized analog neural network 114; col. 21 line 57-61, when the trained neural networks are retrained, the system regenerates/recalculates resistance values and weights 106, schematic model, etc.; col. 22 lines 25-33, Fig. 1C, if the complexity of the neural net can be reduced, pruning and retraining the neural net; once the complexity of the trained neural net is reduced, transforming 170 the trained neural net 166 into a sparse network of analog components; col. 22 lines 38-54, if estimated error exceeds threshold, prompting user to reconfigure, redevelop, or retrain the neural network; process is iterated multiple times until error reduced below threshold; process includes recalculating the weights, since pruning includes retraining of the whole network; i.e. the processes shown in Figs. 1A and 1C include iteratively optimizing and retraining a neural network multiple times, including recalculating new weights, resulting at the end in obtaining an optimized (both in terms of reduced complexity and optimal error rate below a threshold) neural network using the weights); and
generate an optimized circuit layout for hardware that implements the optimized neural network model obtained using the updated plurality of neuron weights (e.g. col. 21 lines 44-50, Fig. 1A, using optimized analog neural network to generate schematic model 108 and generate 132 lithographic masks for connections and analog neurons, to be used to fabricate analog integrated circuits 118 that implement the analog neural network 104; col. 21 line 57-61, when the trained neural networks are retrained, the system regenerates/recalculates schematic model and lithographic masks; col. 22 lines 34-35, Fig. 1C generating circuit model 172 of the analog network; generating lithography masks 174; i.e. the optimized neural network is used to generate a schematic/circuit model and lithographic masks for implementing the optimized neural network in hardware).
With respect to claim 18, Timofejevs teaches all of the limitations of claim 14 as previously discussed, and further teaches wherein the trained neural network model further comprises a feed forward neural network model (e.g. col. 3 line 7, feed-forward neural network; col. 18 lines 18-19, trained neural network is a feed-forward neural network).
With respect to claim 19, Timofejevs teaches all of the limitations of claim 18 as previously discussed, and further teaches wherein the trained neural network further comprises a multilayer perceptron neural network model (e.g. col. 7 lines 66-67, neural network topology includes a multi-layer perceptron; col. 10 lines 21-22, neural network topology includes multilayer perceptron network).
Claim Rejections – 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims under pre-AIA 35 U.S.C. 103(a), the examiner presumes that the subject matter of the various claims was commonly owned at the time any inventions covered therein were made absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and invention dates of each claim that was not commonly owned at the time a later invention was made in order for the examiner to consider the applicability of pre-AIA 35 U.S.C. 103(c) and potential pre-AIA 35 U.S.C. 102€, (f) or (g) prior art under pre-AIA 35 U.S.C. 103(a).
Claims 7-13, 15, and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Timofejevs in view of Amir Hossein Alavi, Amir Hossein Gandomi. Prediction of principal ground-motion parameters using a hybrid method coupling artificial neural networks and simulated annealing. Computers & Structures, Volume 89, Issues 23–24, 2011, Pages 2176-2194, ISSN 0045-7949, https://doi.org/10.1016/j.compstruc.2011.08.019. (https://www.sciencedirect.com/science/article/pii/S0045794911002422). (hereinafter Alavi).
With respect to claim 7, Timofejevs teaches a method for hardware optimization, comprising :
obtaining a trained neural network model, the trained neural network model comprising a plurality of neurons, the plurality of neurons comprising a plurality of neuron layers and a plurality of neuron weights (e.g. col. 3 lines 8-9, process begins with a trained neural network; col. 3 lines 32-33, obtaining neural network topology and weights of trained neural network; col. 3 lines 55-58, neural network topology includes layers of neurons; col. 21 line 24, Fig. 1A, showing that the process begins with an obtained trained neural network; col. 22 lines 22-24, Fig. 1C, user develops and trains neural network 164 and inputs it 166 to SDK 180);
generating a plurality of new weights for one of the plurality of neuron layers (e.g. col. 3 lines 25-26, new weights of the retrained neural network; col. 3 lines 50-51, obtaining new weights for the trained neural network; col. 22 lines 53-54, Fig. 1C, process includes recalculating the weights, since pruning includes retraining of the whole network);
retraining the trained neural network model using the plurality of new weights (e.g. col. 3 lines 25-26, retrained neural network; col. 21 line 57, Fig. 1A, the trained neural network is retrained 142 (shown as 124 in Fig. 1A); col. 22 lines 29-30, retraining 182 the neural net to obtain an updated neural net; col. 22 lines 38-54, if estimated error exceeds threshold, prompting user to reconfigure, redevelop, or retrain the neural network; process is iterated multiple times until error reduced below threshold; process includes recalculating the weights, since pruning includes retraining of the whole network; i.e. the processes shown in Figs. 1A and 1C include iteratively retraining a neural network multiple times, including recalculating new weights for use in retraining the neural network);
obtaining an updated plurality of neuron weights (e.g. col. 3 lines 25-26, new weights of the retrained neural network; col. 3 lines 50-51, obtaining new weights for the trained neural network; col. 22 lines 53-54, Fig. 1C, iterative process includes recalculating the weights, since pruning includes retraining of the whole network);
obtaining an optimized neural network model using the updated plurality of neuron weights (e.g. col. 3 lines 25-26, new weights of the retrained neural network; col. 21 lines 40-44, Fig. 1A, generating schematic model for implementing analog neural network 104 based on the weights 106; optimizing resistance values/weights 106 to form optimized analog neural network 114; col. 21 line 57-61, when the trained neural networks are retrained, the system regenerates/recalculates resistance values and weights 106, schematic model, etc.; col. 22 lines 25-33, Fig. 1C, if the complexity of the neural net can be reduced, pruning and retraining the neural net; once the complexity of the trained neural net is reduced, transforming 170 the trained neural net 166 into a sparse network of analog components; col. 22 lines 38-54, if estimated error exceeds threshold, prompting user to reconfigure, redevelop, or retrain the neural network; process is iterated multiple times until error reduced below threshold; process includes recalculating the weights, since pruning includes retraining of the whole network; i.e. the processes shown in Figs. 1A and 1C include iteratively optimizing and retraining a neural network multiple times, including recalculating new weights, resulting at the end in obtaining an optimized (both in terms of reduced complexity and optimal error rate below a threshold) neural network using the weights); and
generating an optimized circuit layout for hardware that implements the optimized neural network model obtained using the updated plurality of neuron weights (e.g. col. 21 lines 44-50, Fig. 1A, using optimized analog neural network to generate schematic model 108 and generate 132 lithographic masks for connections and analog neurons, to be used to fabricate analog integrated circuits 118 that implement the analog neural network 104; col. 21 line 57-61, when the trained neural networks are retrained, the system regenerates/recalculates schematic model and lithographic masks; col. 22 lines 34-35, Fig. 1C generating circuit model 172 of the analog network; generating lithography masks 174; i.e. the optimized neural network is used to generate a schematic/circuit model and lithographic masks for implementing the optimized neural network in hardware).
Timofejevs does not explicitly disclose performing a simulated annealing process for the plurality of neuron weights. However Alavi teaches performing a simulated annealing process for the plurality of neuron weights (e.g. page 2178, section 2.3, first column, first and second paragraphs, in neural network initialization using simulated annealing (SA) for assigning good starting values to the weights; for the ANN training, SA randomly perturbs the weights of the network following a cooling schedule).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Timofejevs and Alavi in front of him to have modified the teachings of Timofejevs (directed to optimizations for analog hardware realization of trained neural networks), to incorporate the teachings of Alavi (directed to a hybrid method coupling artificial neural networks and simulated annealing) to include the capability to perform a simulated annealing process for the plurality of neuron weights. One of ordinary skill would have been motivated to perform such a modification in order to make the processes of training the neural network more sophisticated by using powerful methods such as simulated annealing for assigning values to neural network weights as described in Alavi (page 2178, section 2.3, first column, first paragraph).
With respect to claim 8, Timofejevs in view of Alavi teaches all of the limitations of claim 7 as previously discussed, and Alavi further teaches wherein the simulated annealing process comprises determining a perturbation value, the perturbation value proportional to a plurality of annealing temperatures (page 2178, section 2.3, first column, first and second paragraphs, SA assigning good starting values to ANN weights, and randomly perturbing the weights following cooling schedule; page 2178, second column, fourth paragraph, indicating that the value y[n] is the SA temperature, and a perturbation ratio used in updating the network’s weights is defined by a proportion based on the current SA temperature y[n] and an initial temperature x[1]).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Timofejevs and Alavi in front of him to have modified the teachings of Timofejevs (directed to optimizations for analog hardware realization of trained neural networks), to incorporate the teachings of Alavi (directed to a hybrid method coupling artificial neural networks and simulated annealing) to include the capability to perform the simulated annealing process for the plurality of neuron weights using a perturbation value which is proportional to a plurality of annealing temperatures. One of ordinary skill would have been motivated to perform such a modification in order to make the processes of training the neural network more sophisticated by using powerful methods such as simulated annealing for assigning values to neural network weights as described in Alavi (page 2178, section 2.3, first column, first paragraph).
With respect to claim 9, Timofejevs in view of Alavi teaches all of the limitations of claim 7 as previously discussed, and Timofejevs further teaches the method further comprising providing the optimized circuit layout to a fabrication facility (e.g. col. 21, lines 49-50, Fig. 1A, fabricating 134/138 the analog integrated circuit 118 that implements the analog neural network 104).
With respect to claim 10, Timofejevs in view of Alavi teaches all of the limitations of claim 9 as previously discussed, and Timofejevs further teaches the method further comprising: manufacturing a chip according to the optimized circuit layout, wherein the chip has a first number of gates less than a second number of gates for a second chip implementing the trained neural network model (e.g. col. 14 line 40-col. 15 line 23, providing integrated circuit including an analog network of analog components resulting from transforming the topology and weights of a trained neural network to an equivalent analog network of analog components; components including plurality of operational amplifiers representing analog neurons and resistors representing respective connections between neurons; col. 21, lines 49-50, Fig. 1A, fabricating 134/138 the analog integrated circuit 118 that implements the analog neural network 104; col. 41 lines 11-16, compression of neural network in order to minimize the number of operational amplifiers and resistors, necessary to realize the analog network on chip; col. 44 lines 38-43, Fig. 18, neuron model 1800; circuit is based on operational amplifier that receives input signals from resistors; col. 46 line 66-col. 47 line 30, Fig. 19A, operational amplifier inputs, outputs, etc.; applying signal to gate of transistor (i.e. indicating that the various components of the amplifier include corresponding gates); i.e. the analog integrated circuit is fabricated/manufactured according to the layout, where the circuit is implemented using operational amplifiers and resistors and includes corresponding gates, and where the circuit is optimized/compressed to include a minimized number of operational amplifiers and resistors (and therefore a minimized number of corresponding gates), such that the compressed/minimized chip implementing the neural network has less gates than an uncompress/non-minimized chip implementing the neural network would have).
With respect to claim 11, Timofejevs in view of Alavi teaches all of the limitations of claim 7 as previously discussed, and Timofejevs further teaches wherein the trained neural network model further comprises a feed forward neural network model (e.g. col. 21, lines 49-50, Fig. 1A, fabricating 134/138 the analog integrated circuit 118 that implements the analog neural network 104).
With respect to claim 12, Timofejevs in view of Alavi teaches all of the limitations of claim 11 as previously discussed, and Timofejevs further teaches wherein the trained neural network model further comprises a multilayer perceptron neural network model (e.g. col. 7 lines 66-67, neural network topology includes a multi-layer perceptron; col. 10 lines 21-22, neural network topology includes multilayer perceptron network).
With respect to claim 13, Timofejevs in view of Alavi teaches all of the limitations of claim 5 as previously discussed, and Timofejevs further teaches wherein the trained neural network model comprises an input layer of the plurality of neurons, a hidden layer of the plurality of neurons, and an output layer of the plurality of neurons (e.g. col. 26 lines 33-40, Fig. 4A, neurons of network organized into multiple layers; input layer of neurons 402 followed by hidden layers of neurons 404 and 406; hidden layers followed by output layer 408).
With respect to claim 15, Timofejevs teaches all of the limitations of claim 14 as previously discussed. Timofejevs does not explicitly disclose wherein the simulated annealing process comprises determining a perturbation value, the perturbation value proportional to a plurality of annealing temperatures.
However, Alavi teaches wherein the simulated annealing process comprises determining a perturbation value, the perturbation value proportional to a plurality of annealing temperatures (page 2178, section 2.3, first column, first and second paragraphs, SA assigning good starting values to ANN weights, and randomly perturbing the weights following cooling schedule; page 2178, second column, fourth paragraph, indicating that the value y[n] is the SA temperature, and a perturbation ratio used in updating the network’s weights is defined by a proportion based on the current SA temperature y[n] and an initial temperature x[1]).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Timofejevs and Alavi in front of him to have modified the teachings of Timofejevs (directed to optimizations for analog hardware realization of trained neural networks), to incorporate the teachings of Alavi (directed to a hybrid method coupling artificial neural networks and simulated annealing) to include the capability to perform the simulated annealing process for the plurality of neuron weights using a perturbation value which is proportional to a plurality of annealing temperatures. One of ordinary skill would have been motivated to perform such a modification in order to make the processes of training the neural network more sophisticated by using powerful methods such as simulated annealing for assigning values to neural network weights as described in Alavi (page 2178, section 2.3, first column, first paragraph).
With respect to claim 16, Timofejevs in view of Alavi teaches all of the limitations of claim 15 as previously discussed, and Alavi further teaches wherein the perturbation value is a percentage of weight to be removed from each of the plurality of neural weights (e.g. page 2178 first paragraph, process simulated by SA to find minimum of function in certain design space; page 2178 first column, section 2.3, using SA to assign good starting values to weights; perturbing the weights following cooling schedule; page 2178 second column, applying cooling schedule to cool down/decrease temperature over time until final temperature is reached, where network’s weights are updated using a recursive equation according to a perturbation ratio based on a current temperature and an initial temperature; i.e. applying cooling schedule to neural network weights according to a perturbation ratio in order to arrive at a minimum is analogous to applying the perturbation value as a percentage/ratio of the weight to be removed (i.e. decreased/cooled according to the cooling schedule) at each iteration).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Timofejevs and Alavi in front of him to have modified the teachings of Timofejevs (directed to optimizations for analog hardware realization of trained neural networks), to incorporate the teachings of Alavi (directed to a hybrid method coupling artificial neural networks and simulated annealing) to include the capability to perform the simulated annealing process for the plurality of neuron weights using a perturbation value which is a ratio/percentage applied to the weight, i.e. an amount to decrease/remove from the weight in order to reach a minimum according to a cooling schedule. One of ordinary skill would have been motivated to perform such a modification in order to make the processes of training the neural network more sophisticated by using powerful methods such as simulated annealing for assigning values to neural network weights as described in Alavi (page 2178, section 2.3, first column, first paragraph).
Claims 1-3, 5, and 6 are rejected under 35 U.S.C. 103 as being unpatentable over Timofejevs in view of Alavi, further in view of Ophir et al. (US 20240260880 A1).
With respect to claim 1, Timofejevs teaches a device, comprising: an electronic processor having:
a set of input pins; a set of output pins; and a layout of circuit gates implementing an optimized neural network model (e.g. col. 14 line 40-col. 15 line 23, providing integrated circuit including an analog network of analog components resulting from transforming the topology and weights of a trained neural network to an equivalent analog network of analog components; components including plurality of operational amplifiers representing analog neurons and resistors representing respective connections between neurons; integrated circuit may further include digital to analog converters, analog signal sampling modules, voltage converter modules, tact signal processing modules, clock modules, analog to digital converters, etc.; col. 44 lines 38-43, Fig. 18, neuron model 1800; circuit is based on operational amplifier 1824 that receives input signals from resistors 1804, 1806, 1816, 1818, 1812, 1808, 1810, 1820, 1822, 1814; col. 46 line 66-col. 47 line 30, Fig. 19A, operational amplifier including positive input 1404, negative input 1406, positive supply voltage 1402 which are contact inputs, and circuit output 1410; applying signal to gate of transistor (i.e. indicating that the various components of the amplifier include corresponding gates); i.e. a circuit for implementing a trained neural network is provided, where the circuit includes at least a plurality of operational amplifiers and a plurality of resistors, where each operational amplifier includes at least one input pin, at least one output pin, and corresponding gates, such that the circuit as a whole includes a set of input pins, a set of output pins, and a layout of circuit gates for implementing the neural network),
the optimized neural network model obtained by performing an optimization process on a plurality of neuron weights in a trained neural network model (e.g. col. 3 lines 25-26, new weights of the retrained neural network; col. 21 lines 40-44, Fig. 1A, generating schematic model for implementing analog neural network 104 based on the weights 106; optimizing resistance values/weights 106 to form optimized analog neural network 114; col. 21 line 57-61, when the trained neural networks are retrained, the system regenerates/recalculates resistance values and weights 106, schematic model, etc.; col. 22 lines 25-33, Fig. 1C, if the complexity of the neural net can be reduced, pruning and retraining the neural net; once the complexity of the trained neural net is reduced, transforming 170 the trained neural net 166 into a sparse network of analog components; col. 22 lines 38-54, if estimated error exceeds threshold, prompting user to reconfigure, redevelop, or retrain the neural network; process is iterated multiple times until error reduced below threshold; process includes recalculating the weights, since pruning includes retraining of the whole network; i.e. the processes shown in Figs. 1A and 1C include iteratively optimizing and retraining a neural network multiple times, including recalculating new weights, resulting at the end in obtaining an optimized (both in terms of reduced complexity and optimal error rate below a threshold) neural network using the weights); and
wherein the layout causes the electronic processor to:
when receiving a runtime dataset via the set of input pins, extract a plurality of features from the runtime dataset (e.g. col. 26 line 62-col. 27 line 18, Fig. 4C, feature learning state of CNN including first layer 430 that extracts features from an input 428);
apply the plurality of features to the optimized neural network model (e.g. col. 27 lines 7-18, Fig. 4C, performing convolution operation on input (extracted features), performing additional opeations in pooling layer, etc., output of pooling layer flattened by layer 438 and input to fully connected neural network with one or more layers); and
output a prediction indication via the output pins (e.g. col. 27 lines 14-18, Fig. 4C, output of fully connected neural network input to softmax layer to classify the output of the layer 442 of the fully connected network to produce one of many different outputs 446 such as object class, type of input image, etc.).
Timofejevs does not explicitly disclose that the optimized neural network model is obtained by performing a simulated annealing process on a plurality of neuron weights. However, Alavi teaches disclose that the optimized neural network model is obtained by performing a simulated annealing process on a plurality of neuron weights (e.g. page 2178, section 2.3, first column, first and second paragraphs, in neural network initialization using simulated annealing (SA) for assigning good starting values to the weights; for the ANN training, SA randomly perturbs the weights of the network following a cooling schedule).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Timofejevs and Alavi in front of him to have modified the teachings of Timofejevs (directed to optimizations for analog hardware realization of trained neural networks), to incorporate the teachings of Alavi (directed to a hybrid method coupling artificial neural networks and simulated annealing) to include the capability to perform a simulated annealing process for the plurality of neuron weights. One of ordinary skill would have been motivated to perform such a modification in order to make the processes of training the neural network more sophisticated by using powerful methods such as simulated annealing for assigning values to neural network weights as described in Alavi (page 2178, section 2.3, first column, first paragraph).
Timofejevs and Alavi do not explicitly disclose obtain a confidence level, where the prediction indication is based on the confidence level. However, Ophir teaches obtain a confidence level, where the prediction indication is based on the confidence level (e.g. paragraph 0056, ML model analyzing signals from sensor data to determine one or more biological conditions and a confidence score; ML model trained on disease state such as positive or negative determination of cancer, etc., paragraph 0057, confidence score calculated based on probability of the disease state and confidence prediction interval; ML model predicting disease state and likelihood value of the disease state).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Timofejevs, Alavi, and Ophir in front of him to have modified the teachings of Timofejevs (directed to optimizations for analog hardware realization of trained neural networks) and Alavi (directed to a hybrid method coupling artificial neural networks and simulated annealing), to incorporate the teachings of Ophir (directed to machine learning based disease detection) to include the capability to obtain a confidence level and provide the prediction based on the confidence level. One of ordinary skill would have been motivated to perform such a modification in order acquire high accuracy in high throughput screening and diagnostic testing resulting in early detection of disease, and allow for more frequent, easier, and more cost-effective screening as described in Ophir (paragraph 0008).
With respect to claim 2, Timofejevs in view of Alavi, further in view of Ophir teaches all of the limitations of claim 1 as previously discussed, and Alavi further teaches wherein the simulated annealing process comprises determining a perturbation value, the perturbation value proportional to a plurality of annealing temperatures (page 2178, section 2.3, first column, first and second paragraphs, SA assigning good starting values to ANN weights, and randomly perturbing the weights following cooling schedule; page 2178, second column, fourth paragraph, indicating that the value y[n] is the SA temperature, and a perturbation ratio used in updating the network’s weights is defined by a proportion based on the current SA temperature y[n] and an initial temperature x[1]).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Timofejevs, Ophir, and Alavi in front of him to have modified the teachings of Timofejevs (directed to optimizations for analog hardware realization of trained neural networks) and Ophir (directed to machine learning based disease detection), to incorporate the teachings of Alavi (directed to a hybrid method coupling artificial neural networks and simulated annealing) to include the capability to perform the simulated annealing process for the plurality of neuron weights using a perturbation value which is proportional to a plurality of annealing temperatures. One of ordinary skill would have been motivated to perform such a modification in order to make the processes of training the neural network more sophisticated by using powerful methods such as simulated annealing for assigning values to neural network weights as described in Alavi (page 2178, section 2.3, first column, first paragraph).
With respect to claim 3, Timofejevs in view of Alavi, further in view of Ophir teaches all of the limitations of claim 2 as previously discussed, and Alavi further teaches wherein the perturbation value is a percentage of weight to be removed from each of the plurality neural weights (e.g. page 2178 first paragraph, process simulated by SA to find minimum of function in certain design space; page 2178 first column, section 2.3, using SA to assign good starting values to weights; perturbing the weights following cooling schedule; page 2178 second column, applying cooling schedule to cool down/decrease temperature over time until final temperature is reached, where network’s weights are updated using a recursive equation according to a perturbation ratio based on a current temperature and an initial temperature; i.e. applying cooling schedule to neural network weights according to a perturbation ratio in order to arrive at a minimum is analogous to applying the perturbation value as a percentage/ratio of the weight to be removed (i.e. decreased/cooled according to the cooling schedule) at each iteration).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Timofejevs, Ophir, and Alavi in front of him to have modified the teachings of Timofejevs (directed to optimizations for analog hardware realization of trained neural networks) and Ophir (directed to machine learning based disease detection), to incorporate the teachings of Alavi (directed to a hybrid method coupling artificial neural networks and simulated annealing) to include the capability to perform the simulated annealing process for the plurality of neuron weights using a perturbation value which is a ratio/percentage applied to the weight, i.e. an amount to decrease/remove from the weight in order to reach a minimum according to a cooling schedule. One of ordinary skill would have been motivated to perform such a modification in order to make the processes of training the neural network more sophisticated by using powerful methods such as simulated annealing for assigning values to neural network weights as described in Alavi (page 2178, section 2.3, first column, first paragraph).
With respect to claim 5, Timofejevs in view of Alavi, further in view of Ophir teaches all of the limitations of claim 1 as previously discussed, and Ophir further teaches wherein the runtime dataset comprises a heart disease dataset, and wherein the confidence level comprises a possibility indication of heart disease in a patient (e.g. paragraph 0003, detecting diseases such as heart disease; paragraph 0056, ML model analyzing signals from sensor data to determine one or more biological conditions and a confidence score; ML model trained on disease state, paragraph 0057, confidence score calculated based on probability of the disease state and confidence prediction interval; ML model predicting disease state and likelihood value of the disease state).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Timofejevs, Alavi, and Ophir in front of him to have modified the teachings of Timofejevs (directed to optimizations for analog hardware realization of trained neural networks) and Alavi (directed to a hybrid method coupling artificial neural networks and simulated annealing), to incorporate the teachings of Ophir (directed to machine learning based disease detection) to include the capability to obtain a confidence level and provide the prediction based on the confidence level, such as a confidence level indicating a possibility/likelihood of heart disease in a patient. One of ordinary skill would have been motivated to perform such a modification in order acquire high accuracy in high throughput screening and diagnostic testing resulting in early detection of disease, and allow for more frequent, easier, and more cost-effective screening as described in Ophir (paragraph 0008).
With respect to claim 6, Timofejevs teaches all of the limitations of claim 1 as previously discussed, and further teaches wherein the runtime dataset comprises a breast cancer dataset, and wherein the confidence level comprises a possibility indication of breast cancer in a patient (e.g. paragraph 0056, ML model analyzing signals from sensor data to determine one or more biological conditions and a confidence score; ML model trained on disease state such as positive or negative determination of cancer, particular type of cancer such as breast cancer, etc., paragraph 0057, confidence score calculated based on probability of the disease state and confidence prediction interval; ML model predicting disease state and likelihood value of the disease state).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Timofejevs, Alavi, and Ophir in front of him to have modified the teachings of Timofejevs (directed to optimizations for analog hardware realization of trained neural networks) and Alavi (directed to a hybrid method coupling artificial neural networks and simulated annealing), to incorporate the teachings of Ophir (directed to machine learning based disease detection) to include the capability to obtain a confidence level and provide the prediction based on the confidence level, such as a confidence level indicating a possibility/likelihood of breast cancer in a patient. One of ordinary skill would have been motivated to perform such a modification in order acquire high accuracy in high throughput screening and diagnostic testing resulting in early detection of disease, and allow for more frequent, easier, and more cost-effective screening as described in Ophir (paragraph 0008).
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Timofejevs in view of Alavi, further in view of Ophir, further in view of Tang et al. (US 20230325642 A1).
With respect to claim 4, Timofejevs teaches all of the limitations of claim 3 as previously discussed. Timofejevs does not explicitly disclose wherein the simulated annealing process further comprises rounding each of the plurality of neural weights to an integer value. However, Tang teaches wherein the simulated annealing process further comprises rounding each of the plurality of neural weights to an integer value (e.g. paragraphs 0074-0075, retraining neural network, including by generating second weight values, including by rounding the second weight values to nearest integers).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Timofejevs, Alavi, Ophir, and Tang in front of him to have modified the teachings of Timofejevs (directed to optimizations for analog hardware realization of trained neural networks), Ophir (directed to machine learning based disease detection), and Alavi (directed to a hybrid method coupling artificial neural networks and simulated annealing), to incorporate the teachings of Tang (directed to neural network with weight quantization) to include the capability to as part of the process for obtaining new weights to train the neural network on (as taught by Timifejevs and Tang, where this process includes simulated annealing as taught by Alavi), round each of the plurality of neural network weights to an integer value (as taught by Tang). One of ordinary skill would have been motivated to perform such a modification in order to improve accuracy of a neural network deployed in a device different from one in which it was trained, and provide improved performance allowing devices with constrained power, resources, etc. to implement the neural network as described in Tang (paragraph 0055).
Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over Timofejevs in view of Alavi, further in view of Tang.
With respect to claim 17, Timofejevs in view of Alavi teaches all of the limitations of claim 16 as previously discussed. Timofejevs does not explicitly disclose wherein the simulated annealing process further comprises rounding each of the plurality of neural weights to an integer value. However, Tang teaches wherein the simulated annealing process further comprises rounding each of the plurality of neural weights to an integer value(e.g. paragraphs 0074-0075, retraining neural network, including by generating second weight values, including by rounding the second weight values to nearest integers).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Timofejevs, Alavi, Ophir, and Tang in front of him to have modified the teachings of Timofejevs (directed to optimizations for analog hardware realization of trained neural networks), Ophir (directed to machine learning based disease detection), and Alavi (directed to a hybrid method coupling artificial neural networks and simulated annealing), to incorporate the teachings of Tang (directed to neural network with weight quantization) to include the capability to as part of the process for obtaining new weights to train the neural network on (as taught by Timifejevs and Tang, where this process includes simulated annealing as taught by Alavi), round each of the plurality of neural network weights to an integer value (as taught by Tang). One of ordinary skill would have been motivated to perform such a modification in order to improve accuracy of a neural network deployed in a device different from one in which it was trained, and provide improved performance allowing devices with constrained power, resources, etc. to implement the neural network as described in Tang (paragraph 0055).
It is noted that any citation to specific pages, columns, lines, or figures in the prior art references and any interpretation of the references should not be considered to be limiting in any way. “The use of patents as references is not limited to what the patentees describe as their own inventions or to the problems with which they are concerned. They are part of the literature of the art, relevant for all they contain,” In re Heck, 699 F.2d 1331, 1332-33, 216 USPQ 1038, 1039 (Fed. Cir. 1983) (quoting in re Lemelson, 397 F.2d 1006, 1009, 158 USPQ 275, 277 (GCPA 1968)). Further, a reference may be relied upon for all that it would have reasonably suggested to one having ordinary skill the art, including nonpreferred embodiments. Merck & Co, v. Biocraft Laboratories, 874 F.2d 804, 10 USPQ2d 1843 (Fed. Cir.), cert, denied, 493 U.S. 975 (1989). See also Upsher-Smith Labs. v. Pamlab, LLC, 412 F,3d 1319, 1323, 75 USPQ2d 1213, 1215 (Fed. Cir, 2005): Celeritas Technologies Ltd. v. Rockwell International Corp., 150 F.3d 1354, 1361, 47 USPQ2d 1516, 1522-23 (Fed. Cir. 1998).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JEREMY L STANLEY whose telephone number is (469)295-9105. The examiner can normally be reached on Monday-Friday from 9:00 AM to 5:00 PM CST.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abdullah Al Kawsar, can be reached at telephone number (571) 270-3169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center and the Private Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from Patent Center or Private PAIR. Status information for unpublished applications is available through Patent Center and Private PAIR for authorized users only. Should you have questions about access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-automated- interview-request-air-form.
/JEREMY L STANLEY/
Primary Examiner, Art Unit 2127
Prosecution Insights