DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 12/04/2025 has been entered.
Claim Rejection related to 35 USC § 112 regarding to claim 1 and 10 is withdrawn.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 8, 10-12, 17 are rejected under 35 U.S.C. 103 as being unpatentable over Coenen et al. (Coenen) US 2021/0287074 in view of Park US 2019/0212981 and Sun et al. (Sun) 2021/0064985
In regard to claim 1, Coenen disclose An electronic device of updating a neural network model, comprising: ([0031]-[0047] Fig. 1A, the computing device to update the NN)
a transceiver for receiving the neural network model and a piece of training data, wherein the neural network model comprises an input node, a first neuron connected to the input node and a second neuron connected to the first neuron; (Fig. 1A-1D, [0031]-[0047] [0064] 122 for receiving the NN model 106 and training data 105, and the 106 has the first neuron 131-1 connected to input node 130 and the second neuron 131-2 is connected to 131-1) and
a processor coupled to the transceiver, wherein the processor is configured to execute: ([0073]-[0076] Fig. 6, 658 processor)
But Coenen fail to explicitly disclose “inputting the training data from the input node to the first neuron to output a first value to the second neuron, so as to output a first estimated value from the second neuron; inputting the training data from the input node to the third neuron to output a second value to the fourth neuron, so as to output a second estimated value from the fourth neuron, wherein the third neuron is connected to the input node; and updating a first activation function of the first neuron and a second activation function of the second neuron according to the first estimated value and the second estimated value to generate an updated neural network model, wherein the transceiver is configured for outputting the updated neural network model.”
Park disclose inputting the training data from the input node to the first neuron to output a first value to the second neuron, so as to output a first estimated value from the second neuron; ([0033]-[0041] Fig. 2, I1, for example, input the training data from the input node at box 2 to the first neuron at box 3 (b1) and output a result of the computation to the next neuron at box 4 connected with the first neuron, for example , and to output a value from the next neuron at box 4 too with a result of the computation. Note: please further define the first value and first estimated value and the relationship between them to help move forward the prosecution)
inputting the training data from the input node to the third neuron to output a second value to the fourth neuron, so as to output a second estimated value from the fourth neuron; wherein the third neuron is connected to the input node; ([0033]-[0041] Fig. 2, input the training data from the input node I1 at box 2 which is connected to the third neuron at box 3 (b2 or b3) and output a result of the computation to the next neuron at box 4 connected with the third neuron, for example , and to output a value from the next neuron at box 4 too with a result of the computation. the neurons are different and interconnected. Note: please further define the second value and second estimated value and the relationship between them to help move forward the prosecution) and
updating a first activation function of the first neuron and a second activation function of the second neuron according to the first estimated value and the second estimated value to generate an updated neural network model, wherein the transceiver is configured for outputting the updated neural network model. ([0033]-[0041] each of the neurons may determine their activation based on activations and weights received from neurons included in the previous layer and weight is a parameter used to calculate an output activation in each neuron, and output the result and the NN is trained and updated)
It would have been obvious to one having ordinary skill in the art before the effective filing data of the claimed invention was made to incorporate Park‘s training of NN into Coenen’s invention as they are related to the same field endeavor of model training and learning. The motivation to combine these arts, as proposed above, at least because Park‘s training of NN’s with adjusting weights and activation functions of neurons would help to provide more NN training mechanism into Coenen’s system. Therefore it would have been obvious to one having ordinary skill in the art before the effective filing data of the claimed invention was made that adjusting weights and activation functions in training NN model would help to improve accuracy of prediction precision and training efficiency.
But Coenen and Park fail to explicitly disclose “quantizing a first weight of the first neuron to generate a third neuron different from the first neuron, and quantizing a second weight of the second neuron to generate a fourth neuron connected to the third neuron, wherein the first weight corresponds to a first floating point number format, and the quantized first weight corresponds to one of an integer format or a second floating point number format different from the first floating point number format;”
Sun disclose quantizing a first weight of the first neuron to generate a third neuron different from the first neuron, and quantizing a second weight of the second neuron to generate a fourth neuron connected to the third neuron, wherein the first weight corresponds to a first floating point number format, and the quantized first weight corresponds to one of an integer format or a second floating point number format different from the first floating point number; ([0008][0029]-[0047] [0067]-[0074] one learner node (204.sub.1-1) generate another learner node (204.sub.2-1) from quantizing weight using different floating number format from the one learner node (204.sub.1-2) and generate another learner node (204.sub.2-2) from quantizing weight using different floating number format, and components 202.sub.1 and 202.sub.2 are connected which can represent neurons in the layers and the first weight is a floating point format and quantized first weight can use different quantizing format)
It would have been obvious to one having ordinary skill in the art before the effective filing data of the claimed invention was made to incorporate Sun’s ML hardware into Park and Coenen’s invention as they are related to the same field endeavor of model training and learning of NN. The motivation to combine these arts, as proposed above, at least because Sun’s generating NN with adjusted weights would help to provide more NN updating mechanism into Park and Coenen’s system. Therefore it would have been obvious to one having ordinary skill in the art before the effective filing data of the claimed invention was made that generating NN with adjusted weights would improve performance of the updated NN.
In regard to claim 2, Coenen and Park, Sun disclose The electronic device of updating the neural network model according to claim 1, wherein the processor is further configured to execute:
Coenen disclose deleting one bit of the second estimated value to generate the quantized second estimated value; ([0003]-[0007][0031]-[0047] the weight can be pruned and the pruned weight word is one of a most significant or a least significant byte (part, bit))
Coenen and Sun fail to explicitly disclose “updating the first activation function and the second activation function according to the first estimated value and the quantized second estimated value.”
Park disclose updating the first activation function and the second activation function according to the first estimated value and the quantized second estimated value. ([0033]-[0041] determine their activations based on activations and weights received from neurons included in the previous layer)
It would have been obvious to one having ordinary skill in the art before the effective filing data of the claimed invention was made to incorporate Park‘s training of NN into Sun, Coenen’s invention as they are related to the same field endeavor of model training and learning. The motivation to combine these arts, as proposed above, at least because Park‘s training of NN’s with adjusting weights and activation functions of neurons would help to provide more NN training mechanism into Sun, Coenen’s system. Therefore it would have been obvious to one having ordinary skill in the art before the effective filing data of the claimed invention was made that adjusting weights and activation functions in training NN model would help to improve accuracy of prediction precision and training efficiency.
In regard to claim 3, Coenen and Park, Sun disclose The electronic device of updating the neural network model according to claim 2,
Coenen disclose wherein the bit comprises at least one of a most significant bit and a least significant bit. ([0003]-[0007][0031]-[0047] the weight can be pruned and the pruned weight word is one of a most significant or a least significant byte (part, bit)
In regard to claim 8, Coenen and Park, Sun disclose The electronic device of updating the neural network model according to claim 1,
Coenen fail to explicitly disclose further comprising: a storage medium coupled to the processor and configured for storing the neural network model. ([0003]-[0007] [0024]-[0037] [0073]-[0076] a memory device via a processor data bus to store the encoded weights for the NN)
In regard to claims 10-12, 17, claims 10-12, 17 are method claims corresponding to the device claims 1-3, 8 above and, therefore, are rejected for the same reasons set forth in the rejections of claims 1-3, 8.
Claims 4-5, 13-14 are rejected under 35 U.S.C. 103 as being unpatentable over Coenen et al. (Coenen) US 2021/0287074 and Park US 2019/0212981 and Sun et al. (Sun) 2021/0064985 as applied to claim 1, further in view of Hara US 2022/0366303
In regard to claim 4, Coenen and Park, Sun disclose The electronic device of updating the neural network model according to claim 1, wherein the processor is further configured to execute:
Coenen disclose deleting one bit of the second estimated value to generate the quantized second estimated value; ([0003]-[0007][0031]-[0047] the weight can be pruned and the pruned weight word is one of a most significant or a least significant byte (part, bit) and to generate the output value)
But Coenen and Park, Sun fail to explicitly disclose “calculating a difference between the first estimated value and the quantized second estimated value; and training a downstream neuron of the second neuron with an output of the fourth neuron in response to the difference being less than a threshold.”
Hara disclose calculating a difference between the first estimated value and the quantized second estimated value; and training a downstream neuron of the second neuron with an output of the fourth neuron in response to the difference being less than a threshold. (Fig. 10, [0009]-[0012] the difference between the weights are calculated and training the downstream if the weight different is less than a threshold which means it is not trained)
It would have been obvious to one having ordinary skill in the art before the effective filing data of the claimed invention was made to incorporate Hara‘s ML model training into Sun, Park and Coenen’s invention as they are related to the same field endeavor of model training and learning. The motivation to combine these arts, as proposed above, at least because Hara‘s ML training with training criteria would help to provide more NN training mechanism into Sun, Park and Coenen’s system. Therefore it would have been obvious to one having ordinary skill in the art before the effective filing data of the claimed invention was made that providing training criteria in training NN model would help to improve accuracy of prediction precision and training efficiency.
In regard to claim 5, Coenen and Park, Sun The electronic device of updating the neural network model according to claim 4,
But Coenen and Park, Sun fail to explicitly disclose “wherein the processor is further configured to execute: training the downstream neuron with an output of the second neuron in response to the difference being greater than or equal to the threshold.”
Hara disclose wherein the processor is further configured to execute: training the downstream neuron with an output of the second neuron in response to the difference being greater than or equal to the threshold (Fig. 10, [0009]-[0012] [0014] [0046]-[0052] training the downstream with output of second neuron when the difference is equal to or larger than the threshold)
It would have been obvious to one having ordinary skill in the art before the effective filing data of the claimed invention was made to incorporate Hara‘s ML model training into Sun, Park and Coenen’s invention as they are related to the same field endeavor of model training and learning. The motivation to combine these arts, as proposed above, at least because Hara‘s ML training with training criteria would help to provide more NN training mechanism into Sun, Park and Coenen’s system. Therefore it would have been obvious to one having ordinary skill in the art before the effective filing data of the claimed invention was made that providing training criteria in training NN model would help to improve accuracy of prediction precision and training efficiency.
In regard to claims 13-14, claims 13-14 are method claims corresponding to the device claims 4-5 above and, therefore, are rejected for the same reasons set forth in the rejections of claims 4-5.
Claims 7, 16 are rejected under 35 U.S.C. 103 as being unpatentable over Coenen et al. (Coenen) US 2021/0287074 and Park US 2019/0212981 and Sun et al. (Sun) 2021/0064985 as applied to claim 1, further in view of Biryukova et al. (Biryukova) US 2022/0138562
In regard to claim 7, Coenen and Park, Sun disclose The electronic device of updating the neural network model according to claim 1,
But Coenen and Park, Sun fail to explicitly disclose “wherein the first activation function is a piecewise function.”
Biryukova disclose wherein the first activation function is a piecewise function.
([0184]–[0189][0226]-[0227] the activation function is a piecewise function)
It would have been obvious to one having ordinary skill in the art before the effective filing data of the claimed invention was made to incorporate Biryukova’s NN training into Sun, Park and Coenen’s invention as they are related to the same field endeavor of model training and learning. The motivation to combine these arts, as proposed above, at least because Biryukova’s NN training with piecewise activation function would help to provide more NN training mechanism into Sun, Park and Coenen’s system. Therefore it would have been obvious to one having ordinary skill in the art before the effective filing data of the claimed invention was made that training with piecewise activation function would help to improve accuracy of prediction precision and training efficiency.
In regard to claim 16, claim 16 is a method claim corresponding to the device claim 7 above and, therefore, is rejected for the same reasons set forth in the rejections of claim 7.
Claims 9, 18 are rejected under 35 U.S.C. 103 as being unpatentable over Coenen et al. (Coenen) US 2021/0287074 and Park US 2019/0212981 Sun et al. (Sun) 2021/0064985 as applied to claim 1, further in view of O’Connor et al. US 2022/0121927
In regard to claim 9, Coenen and Park, Sun disclose The electronic device of updating the neural network model according to claim 1, further comprising:
Coenen and Park, Sun fail to explicitly disclose “a storage medium coupled to the processor for storing a threshold of a number of iteration times, wherein the processor determines whether to stop updating the neural network model according to the threshold of the number of the iteration times.”
O’Connor disclose a storage medium coupled to the processor for storing a threshold of a number of iteration times, wherein the processor determines whether to stop updating the neural network model according to the threshold of the number of the iteration times. ([0071]-[0073][0092]-[0099] claim 4, the repeating is performed for a predetermined number of iterations and the parameter of the network is stored at the storage device)
It would have been obvious to one having ordinary skill in the art before the effective filing data of the claimed invention was made to incorporate O’Connor’s NN training into Sun, Park and Coenen’s invention as they are related to the same field endeavor of model training and learning. The motivation to combine these arts, as proposed above, at least because O’Connor’s NN training with training stop criteria would help to provide more NN training mechanism into Sun, Park and Coenen’s system. Therefore it would have been obvious to one having ordinary skill in the art before the effective filing data of the claimed invention was made that training with stopping criteria would help to improve accuracy of prediction precision and training efficiency.
In regard to claim 18, claim 18 is a method claim corresponding to the device claim 9 above and, therefore, is rejected for the same reasons set forth in the rejections of claim 9.
Response to Arguments
Applicant’s arguments with respect to claims 1-5, 7-14, 16-18 filed on 12/04/2025 have been considered but are moot because the arguments do not apply to the current rejection.
Conclusion
The prior art made of record and not relied upon is considered pertinent to Applicant's disclosure.
U.S. Patent Documents PATENT DATE INVENTOR(S) TITLE
US 20200134460 2020-04-30 Du et al.
PROCESSING METHOD AND ACCELERATING DEVICE
Du et al. disclose he present disclosure provides a processing device including: a coarse-grained pruning unit configured to perform coarse-grained pruning on a weight of a neural network to obtain a pruned weight, an operation unit configured to train the neural network according to the pruned weight. The coarse-grained pruning unit is specifically configured to select M weights from the weights of the neural network through a sliding window, and when the M weights meet a preset condition, all or part of the M weights may be set to 0. The processing device can reduce the memory access while reducing the amount of computation, thereby obtaining an acceleration ratio and reducing energy consumption… see abstract.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to XUYANG XIA whose telephone number is (571)270-3045. The examiner can normally be reached Monday-Friday 8am-4pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Welch can be reached at 571-272-7212. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
XUYANG XIA
Primary Examiner
Art Unit 2143
/XUYANG XIA/Primary Examiner, Art Unit 2143