DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Drawings
The drawings are objected to under 37 CFR 1.83(a). The drawings must show every feature of the invention specified in the claims. Therefore, the “digitally controlled circuit” recited in claims 6 and 13 and the “bias voltage setting circuit” recited in claims 7 and 12 must be shown or the feature(s) canceled from the claim(s). No new matter should be entered.
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Objections
Claim 11 is objected to because of the following informalities: in lines 11-12, the claim recites phrase “the first plurality of MJT devices”. Examiner believes the phrase should recite “the first plurality of MTJ devices”. Appropriate correction is required.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-5, 8-11, and 14-20 are rejected under 35 U.S.C. 102(a)(1) as being clearly anticipated by Shao et al. (“Implementation of Artificial Neural Networks Using Magnetoresistive Random-Access Memory-Based Stochastic Computing Units”, 2020), hereinafter Shao.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 6 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Shao, in view of Daniels et al. (“Energy-Efficient Stochastic Computing with Superparamagnetic Tunnel Junctions”, 2020), hereinafter Daniels.
Regarding claim 6, Shao teaches the invention substantially as claimed. See the rejection under 35 U.S.C. 102 above. Shao does not explicitly teach:
a digitally controlled circuit configured to convert oscillations of the MTJ devices into the stochastic bit-streams.
However, Daniels teaches:
a digitally controlled circuit configured to convert oscillations of the MTJ devices into the stochastic bit-streams (Daniels, Figs. 1 and 2; § II, ¶ 2-3, 5-6).
Daniels is considered to be analogous to the claimed invention because it is in the same field of stochastic computing using MTJs. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the ANN of Shao to include the digitally controlled circuit as taught by Daniels. This modification would have been obvious because use of the digitally controlled circuit eliminates the need for current biasing, thereby eliminating a major source of energy consumption (Daniels, § I., ¶ 11).
Claim 13 recites a similar limitation as in claim 6, and it therefore rejected for at least the same reason(s).
Claims 7 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Shao, in view of Manipatruni et al. (US 20160202954 A1), hereinafter Manipatruni.
Regarding claim 7, Shao teaches the invention substantially as claimed. See the rejection under 35 U.S.C. 102 above. Shao further teaches:
…set a bias voltage of the MTJ devices according to a training operation of the ANN (Shao, § IV, ¶ 1-2).
Shao does not explicitly teach:
a bias voltage setting circuit…
However, Manipatruni teaches:
a bias voltage setting circuit (Manipatruni, Fig. 9 and ¶ [0053]).
Manipatruni is considered to be analogous to the claimed invention because it is in the same field of MTJs for random number generation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the implement the bias voltage setting of the MTJs as taught by Shao using a bias voltage setting circuit as taught by Manipatruni. This implementation would have been obvious because a circuit for bias voltage control enables control of the variance of the MTJ generated current (Manipatruni, ¶ [0018]).
Claim 12 recites a similar limitation as in claim 7, and it therefore rejected for at least the same reason(s).
In addition to the foregoing rejections over Shao, claims 1-20 are also rejected under 35 U.S.C. 103 on alternative grounds as detailed below.
Claims 1-2, 8-11, 14, and 18-20 are rejected under 35 U.S.C. 102(a)(1) as being unpatentable over Mondal et al. (“Power Optimizations in MTJ-based Neural Networks through Stochastic Computing”, 2017), hereinafter Mondal.
Regarding claim 1, Mondal teaches:
a plurality of magnetic tunnel junction (MTJ) devices configured as true number generators (TRNGs) to output stochastic bit-streams of random numbers (Mondal, Fig. 5a and § III-B, ¶ 1);
a plurality of input nodes configured to receive respective numerical values for processing by the ANN (Mondal, Fig. 1c);
a plurality of hidden nodes (Mondal, Fig. 1c), at least one of the plurality of hidden nodes in electrical communication with one or more of the plurality of input nodes to receive and output a sum of input values from the one or more of the plurality of input nodes multiplied by a corresponding one of a plurality of first weighting values (Mondal, Fig. 5a; § IV-A ¶ 1 and eqn. 9), each of the plurality of first weighting values corresponding to a respective numerical value from the stochastic bit-streams output by the MTJ devices (Mondal, Fig. 5a; MTJs outputting weights w---1-wn); and
an output node in electrical communication with one or more of the plurality of hidden nodes (Mondal, Fig. 1c) to receive and output a sum of hidden values of the one or more of the plurality of hidden nodes multiplied by a corresponding one of a plurality of second weight values (Mondal, § IV-D describing implementation of 2-layer NN as shown in Fig. 1c; Table III teaching separate weight matrices for output layer (W) and hidden layer (Z)).
Regarding claim 2, Mondal teaches the invention substantially as claimed. See the rejection of claim 1 above. Mondal further teaches:
wherein numerical values of the random numbers are tuned by electrical current through the MTJ devices via spin-transfer torque (Mondal, § III-A ¶ 1).
Claim 14 recites a similar limitation to claim 2 and is rejected for at least the same reason(s).
Regarding claim 8, Mondal teaches the invention substantially as claimed. See the rejection of claim 1 above. Mondal further teaches:
wherein each of the plurality of second weighting values corresponds to a respective numerical value from the stochastic bit-streams output by the MTJ devices (Mondal, § IV-D ¶ 1-2 and eqn. 11; Table III teaches W as output matrix, containing second weighting values).
Regarding claim 9, Mondal teaches the invention substantially as claimed. See the rejection of claim 1 above. Mondal further teaches:
wherein each of the plurality of input nodes is further configured to multiply the input node’s respective numerical value by a corresponding one of a plurality of input weighting values (Mondal, Fig. 1c and § II-A ¶ 1 and eqn. 1), each of the plurality of input weighting values corresponding to a respective numerical value from the stochastic bit-streams output by the MTJ devices (Mondal, Fig. 5a).
Regarding claim 10, Mondal teaches the invention substantially as claimed. See the rejection of claim 1 above. Mondal further teaches:
wherein the plurality of MTJ devices comprises an electrically coupled pair of MTJ devices (Mondal, Fig. 5a showing MTJ pairs).
Claim 18 recites a similar limitation to claim 10 and is rejected for at least the same reason(s).
Regarding claim 11, the claim recites limitations similar to those of claim 1, and Mondal teaches those limitations substantially as claimed. Claim 11 differs from claim 1 by reciting:
a first plurality of magnetic tunnel junction (MTJ) devices configured as true random number generators (TRNGs) to output first stochastic bit-streams of random numbers;
a second plurality of MTJ devices configured as TRNGs to output second stochastic bit-streams of random numbers;
a third plurality of MTJ devices configured as TRNGs to output third stochastic bit-streams of random numbers…
While Mondal does not explicitly show first, second, and third pluralities of MTJ devices as recited in claim 1, Mondal teaches a neural network comprising input, hidden, and output neurons (Mondal, Fig. 1c), teaches the implementation of a neuron comprising a plurality of MTJs for generating stochastic bit-streams representing inputs and weights (Mondal, Fig. 5c; § III-B ¶ 1), and teaches several such neurons in parallel would form a layer and multiple layers connected in series would make up an entire network as shown in Fig. 1c (Mondal, § IV-A). Therefore, Mondal teaches a neural network comprising a first plurality of MTJ devices (implemented in the input neurons), a second plurality of MTJ devices (implemented in the hidden neurons), and a third plurality of MTJ devices (implemented in the output neurons).
Regarding claim 19, Mondal teaches:
a plurality of magnetic tunnel junction (MTJ) devices configured as true random number generators (TRNGs) to output stochastic bit-streams of random numbers (Mondal, Fig. 5a; § III-B);
a plurality of input nodes configured to process respective received numerical values for processing by the ANN (Mondal, Fig. 1c); and
an output node (Mondal, Fig. 1c) configured to:
process one or more of intermediate values resulting from processing by at least the plurality of input nodes to generate a result value (Mondal, Fig. 5a; § IV-D and eqn. 11, where Wlj represents a weight from the output weight matrix and hl represents an output of a hidden neuron), and
output the result value (Mondal, Fig. 5a);
wherein the processing includes multiplication by a weighting value corresponding to a respective numerical value from the stochastic bit-streams output by the plurality of MTJ devices (Mondal, Fig. 5a; § IV-A and eqn. 9).
Regarding claim 20, Mondal teaches the invention substantially as claimed. See the rejection of claim 19 above. Mondal further teaches the ANN further comprising:
a plurality of hidden nodes (Mondal, Fig. 1c), one or more of the plurality of hidden nodes in electrical communication with one or more of the plurality of input nodes to process values resulting from processing by at least one or more of the plurality of input nodes (Mondal, Fig. 5c; § IV-A and eqn. 9).
Claims 3-4 and 15-16 are rejected under 35 U.S.C. 103 as being unpatentable over Mondal, in view of Borders et al. (“Integer factorization using stochastic magnetic tunnel junctions”, 2019), hereinafter Borders.
Regarding claim 3, Mondal teaches the invention substantially as claimed. See the rejection of claim 1 above. Mondal does not explicitly teach:
wherein the MTJ devices comprise a Co/Pt multilayer-based synthetic antiferromagnetic (SAF) structure.
However, Borders teaches:
wherein the MTJ devices comprise a Co/Pt multilayer-based synthetic antiferromagnetic (SAF) structure (Borders, Fig. 1 and § Methods: MTJ fabrication).
Borders is considered to be analogous to the claimed invention because it is in the same field of MTJs as stochastic generators. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the MTJ devices of the ANN as taught by Mondal to be the MTJ devices as taught by Borders. This modification would have been obvious because varying the ferromagnetic free-layer thickness for arbitrary sizes of the MTJs used in typical MRAM results in MTJs with stochasticity suitable for probabilistic computing (Borders, § Main, ¶ 3).
Claim 15 recites limitations similar to the limitations recited in claim 3 and is rejected for at least the same reason(s).
Regarding claim 4, Mondal in view of Borders teaches the invention substantially as claimed. See the rejection of claim 3 above. Borders further teaches:
wherein the SAF structure comprises:
a top electrode comprising an electrically conductive material (Borders, Fig. 1 and § Methods: MTJ fabrication; capping layer comprising Ta(5)/Ru(5)/Ta(50) as top electrode, at least Ta as electrically conductive material);
a first ferromagnetic layer comprising a CoFeB material disposed below the top electrode (Borders, Fig. 1 and § Methods: MTJ fabrication);
a tunnel barrier layer comprising a MgO material disposed below the first ferromagnetic layer (Borders, Fig. 1 and § Methods: MTJ fabrication);
a second ferromagnetic layer comprising a CoFeB material disposed below the tunnel barrier layer (Borders, Fig. 1 and § Methods: MTJ fabrication);
a coupling layer disposed below the second ferromagnetic layer (Borders, Fig. 1 and § Methods: MTJ fabrication; Ta(0.3) as coupling layer);
a SAF layer disposed below the coupling layer (Borders, Fig. 1 and § Methods: MTJ fabrication; [Co/PT]2/Co – Ru – [Co/Pt]7/Co as SAF layer); and
a bottom electrode comprising an electrically conductive material disposed below the SAF layer (Borders, Fig. 1 and § Methods: MTJ fabrication; underlayer as bottom electrode comprising at least Ta as electrically conductive material).
Claim 16 recites limitations similar to the limitations recited in claim 4 and is rejected for at least the same reason(s).
Claims 5 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Mondal in view of Wang et al. (“Hybrid VC-MTJ/CMOS Non-volatile Stochastic Logic for Efficient Computing”, 2017), hereinafter Wang.
Regarding claim 5, Mondal teaches the invention substantially as claimed. See the rejection of claim 1 above. Mondal does not explicitly teach:
wherein at least one of the plurality of MTJ devices is configured to introduce a random reshuffling mechanism.
However, Wang teaches:
wherein at least one of the plurality of MTJ devices is configured to introduce a random reshuffling mechanism (Wang, Fig. 10 and pg. 5, ¶ 1).
Wang is considered to be analogous to the claimed invention because it is in the same field of stochastic computing using MTJs. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the ANN with MTJ TRNGs as taught by Mondal to include the random shuffling mechanism as taught by Wang. This modification would have been obvious because random shuffling eliminates correlation between bit-streams that would result in unwanted false outputs (Wang, pg. 5, ¶ 1).
Claim 17 recites limitations similar to the limitations recited in claim 5 and is rejected for at least the same reason(s).
Claims 6 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Mondal, in view of Daniels.
Regarding claim 6, Mondal teaches the invention substantially as claimed. See the rejection of claim 1 above. Mondal does not explicitly teach:
further comprising a digitally controlled circuit configured to convert oscillations of the MTJ devices into the stochastic bit-streams.
However, Daniels teaches:
further comprising a digitally controlled circuit configured to convert oscillations of the MTJ devices into the stochastic bit-streams (Daniels, Figs. 1 and 2; § II, ¶ 2-3, 5-6).
Daniels is considered to be analogous to the claimed invention because it is in the same field of stochastic computing using MTJs. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the ANN of Shao to include the digitally controlled circuit as taught by Daniels. This modification would have been obvious because use of the digitally controlled circuit eliminates the need for current biasing, thereby eliminating a major source of energy consumption (Daniels, § I., ¶ 11).
Claim 13 recites limitations similar to the limitations recited in claim 6 and is rejected for at least the same reason(s).
Claims 7 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Mondal, in view of Manipatruni, and further in view of Canals et al. (A New Stochastic Computing Methodology for Efficient Neural Network Implementation”, 2016), hereinafter Canals.
Regarding claim 7, Mondal teaches the invention substantially as claimed. See the rejection of claim 1 above. Mondal does not explicitly teach:
a bias voltage setting circuit configured to set a bias voltage of the MTJ devices according to a training operation of the ANN.
However, Manipatruni teaches:
a bias voltage setting circuit configured to set a bias voltage of the MTJ devices (Manipatruni, Fig. 9 and ¶ [0053]).
Manipatruni is considered to be analogous to the claimed invention because it is in the same field of MTJs for random number generation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the implement the bias voltage setting of the MTJs as taught by Shao using a bias voltage setting circuit as taught by Manipatruni. This implementation would have been obvious because a circuit for bias voltage control enables control of the variance of the MTJ generated current (Manipatruni, ¶ [0018]).
Neither Mondal nor Manipatruni explicitly teach:
…set a bias voltage according to a training operation of the ANN.
However, Canals teaches:
…set a bias voltage according to a training operation of the ANN (Canals, Fig. 11 showing linear neuron with two inputs + bias input; § V ¶ 3; § V-A ¶ 1).
Canals is considered to be analogous to the claimed invention because it is in the same field of stochastic computing neural networks. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the ANN implementing MTJ devices with bias voltage setting circuits as taught by the combination of Mondal in view of Manipatruni to implement the training operation as taught by Canals. This modification would have been obvious because training allows the evaluation of the simplest NN structure with the smallest classification error (Canals, § V-A ¶ 1), and finds the smallest fitting error (Canals, § V-B, ¶ 2).
Claim 12 recites limitations similar to the limitations recited in claim 7 and is rejected for at least the same reason(s).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JONATHAN DAVID WARNER whose telephone number is (703)756-5956. The examiner can normally be reached M-F: 9-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Caldwell can be reached at (571)272-3702. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.D.W./
Jonathan David WarnerExaminer, Art Unit 2182 (703)756-5956
/ANDREW CALDWELL/Supervisory Patent Examiner, Art Unit 2182