DETAILED ACTION
Claims 1-20 are presented for examination.
This office action is in response to submission of application on 05/05/2023.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 04/29/2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Objections
Applicant is advised that should claim 12 be found allowable, claim 13 will be objected to under 37 CFR 1.75 as being a substantial duplicate thereof. When two claims in an application are duplicates or else are so close in content that they both cover the same thing, despite a slight difference in wording, it is proper after allowing one claim to object to the other as being a substantial duplicate of the allowed claim. See MPEP § 608.01(m).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 10-15 are rejected under 35 U.S.C. 103 as being unpatentable over Case et al. (Pub. No.: US 2020/0380369 A1), hereafter Case, in view of Bernstein et al. ("“CONVERGENCE RATE OF SIGN STOCHASTIC GRADIENT DESCENT FOR NON-CONVEX FUNCTIONS"), hereafter Bernstein.
Regarding claim 1, Case discloses:
A method of training a neural network configured to perform a machine learning task by processing a network input in accordance with a set of weights of the neural network to generate a network output for the machine learning task, the method comprising (Fig. 1 and ¶[0043]):
maintaining, for each of the weights of the neural network, a respective estimated first moment of a gradient of a loss function for the machine learning task with respect to the weight (Fig. 1 and ¶[0043] teaches maintaining estimated moments, i.e. momentum, of a gradient of a loss function for the neural network with respect to weight),
repeatedly performing operations comprising: performing, using a plurality of training examples, a training step to obtain respective current gradients of the loss function with respect to each of the weights of the neural network (¶[0061] and ¶[0065] teaches using a training set as training examples to perform a training step that obtains current gradients of the loss function L with respect to each weight of the neural network),
determining, for each weight, a respective temporary estimated first moment from the respective estimated first moment of the gradient for the weight and the respective current gradient of the loss function with respect to the weight (Examiner’s Note: BRI of a “temporary” moment value is any moment that will undergo a change, including moment values that will be iteratively updated) (¶[0043] discloses, using the equation Vt+1 = μ Vt – α∇L(Wt), determining a temporary estimated first moment, Vt+1, from the respective estimated first moment of the gradient for the weight, i.e. Vt, and the respective current gradient of the loss function with respect to the weight, ∇L(Wt)).
Case does not disclose: determining, for each weight, an output of a sign function applied to the respective temporary estimated first moment for the weight; determining, for each weight, a respective update from the output of the sign function applied to the respective temporary estimated first moment for the weight; and updating each weight using the respective update for the weight.
Bernstein discloses:
determining, for each weight, an output of a sign function applied to the respective temporary estimated first moment for the weight (page 11, equations 11 and 13 teach an output of a sign function for signSGD applied to a temporary estimated first moment for a weight in equation, i.e. momentum mk+1),
determining, for each weight, a respective update from the output of the sign function applied to the respective temporary estimated first moment for the weight; and updating each weight using the respective update for the weight (page 11, equation 13 teaches determining the update of weights from the output of the sign function and updating each weight) .
Case and Bernstein are analogous art because they are from the same field of endeavor, machine learning and momentum.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Case to include determining, for each weight, an output of a sign function applied to the respective temporary estimated first moment for the weight; determining, for each weight, a respective update from the output of the sign function applied to the respective temporary estimated first moment for the weight; and updating each weight using the respective update for the weight based on the teachings of Bernstein. One of ordinary skill in the art would have been motivated to make this modification in order to improve gradient descent, as suggested by Bernstein (page 3, second paragraph, line 1).
Regarding claim 2, Case, in view of Bernstein, discloses the method of claim 1 (and thus the rejection of claim 1 is incorporated). Case further discloses:
wherein the operations further comprise: updating, for each weight, the respective estimated first moment using the respective current gradient of the loss function with respect to the weight (Figs. 4-5, ¶[0043], and ¶[0065] teaches updating, for each weight, the respective estimated first moment using the respective current gradient of the loss function with respect to the weight).
Regarding claim 10, Case, in view of Bernstein, discloses the method of claim 1 determining, for each weight, a respective update from the output of the sign function applied to the respective temporary estimated first moment for the weight (and thus the rejection of claim 1 is incorporated). Bernstein further discloses:
determining the respective update from (i) the output of the sign function applied to the respective temporary estimated first moment for the weight, (ii) a respective weight decay factor for the weight, (iii) the weight, and (iv) a respective learning rate for the weight
(page 11, first paragraph, lines 2-3 “We tuned over {weight decay, momentum, initial learning rate for optimisers in {SGD, signSGD, Adam).” and equations 11 and 13 teach an update from an output of a sign function for signSGD applied to a temporary estimated first moment for a weight in equation, i.e. momentum mk+1, a respective weight decay factor, the weight, and the initial learning rate weight).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Case to include determining the respective update from (i) the output of the sign function applied to the respective temporary estimated first moment for the weight, (ii) a respective weight decay factor for the weight, (iii) the weight, and (iv) a respective learning rate for the weight based on the teachings of Bernstein. One of ordinary skill in the art would have been motivated to make this modification in order to improve gradient descent, as suggested by Bernstein (page 3, second paragraph, line 1).
Regarding claim 11, Case, in view of Bernstein, discloses the method of claim 10 (and thus the rejection of claim 10 is incorporated). Bernstein further discloses:
wherein updating each weight using the respective update for the weight comprises: subtracting the respective update from the weight (page 11, equation 13).
Claims 12 and 13 are substantially similar to claim 1, and thus are rejected on the same basis as claim 1.
Claims 14-15 are substantially similar to claims 1-2, and thus are rejected on the same basis as claims 1-2.
Claims 3 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Case et al. (Pub. No.: US 2020/0380369 A1), hereafter Case, in view of Bernstein et al. ("“CONVERGENCE RATE OF SIGN STOCHASTIC GRADIENT DESCENT FOR NON-CONVEX FUNCTIONS"), hereafter Bernstein, in further view of Zou et al. ("Weighted AdaGrad with Unified Momentum "), hereafter Zou.
Regarding claim 3, Case, in view of Bernstein, discloses the method of claim 2 updating, for each weight, the respective estimated first moment using the respective current gradient of the loss function with respect to the weight (and thus the rejection of claim 2 is incorporated). Case, in view of Bernstein, does not disclose:
computing, in accordance with a second interpolation weight, a second interpolation between (i) the respective estimated first moment of the gradient for the weight and (ii) the respective current gradient of the loss function with respect to the weight.
Zou discloses:
computing, in accordance with a second interpolation weight, a second interpolation between (i) the respective estimated first moment of the gradient for the weight and (ii) the respective current gradient of the loss function with respect to the weight (page 3, equation 5 discloses an interpolation between an estimated first moment of a gradient for a weight and a respective current gradient of a loss function with respect to the weight).
Case, Bernstein, and Zou are analogous art because they are from the same field of endeavor, Machine learning and momentum.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Case, in view of Bernstein, to include computing, in accordance with a second interpolation weight, a second interpolation between (i) the respective estimated first moment of the gradient for the weight and (ii) the respective current gradient of the loss function with respect to the weight based on the teachings of Zou. One of ordinary skill in the art would have been motivated to make this modification in order to improve convergence speed, as suggested by Zou (page 2, first paragraph, lines 2-3).
Claims 16 is substantially similar to claim 3, and thus is rejected on the same basis as claim 3.
Claims 4-9, and 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over Case et al. (Pub. No.: US 2020/0380369 A1), hereafter Case, in view of Bernstein et al. ("“CONVERGENCE RATE OF SIGN STOCHASTIC GRADIENT DESCENT FOR NON-CONVEX FUNCTIONS"), hereafter Bernstein, in further view of Zou et al. ("Weighted AdaGrad with Unified Momentum "), hereafter Zou, in further view of Ziyin ("LaProp: Separating Momentum and Adaptivity in Adam"), hereafter Ziyin.
Regarding claim 4, Case, in view of Bernstein, in further view of Zou, discloses the method of claim 3 (and thus the rejection of claim 3 is incorporated).
Case, in view of Bernstein, discloses determining, for each weight, a respective temporary estimated first moment from the respective estimated first moment of the gradient for the weight and the respective current gradient of the loss function with respect to the weight in claim 1. Case, in view of Bernstein, in further view of Zou, does not disclose computing, in accordance with a first interpolation weight, a first interpolation between (i) a bias corrected version of the respective estimated first moment of the gradient for the weight and (ii) the respective current gradient of the loss function with respect to the weight
Ziyin discloses:
computing, in accordance with a first interpolation weight, a first interpolation between (i) a bias corrected version of the respective estimated first moment of the gradient for the weight and (ii) the respective current gradient of the loss function with respect to the weight
(page 2, paragraph 4, line 3 “a momentum mt”, page 4, penultimate paragraph, lines 3-4 “
PNG
media_image1.png
20
396
media_image1.png
Greyscale
” and Algorithm 1 discloses bias correction factors cn and cm, and
PNG
media_image2.png
32
256
media_image2.png
Greyscale
for a first interpolation between adaptive and signed gradient method, in accordance with a first interpolation weight μ, between a bias corrected version of the respective estimated first moment of the gradient for the weight and the respective current gradient of the loss function ∇l with respect to the weight).
Case, Bernstein, Zou and Ziyin are analogous art because they are from the same field of endeavor, Machine learning and momentum.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Case, in view of Bernstein, in further view of Zou, to include computing, in accordance with a first interpolation weight, a first interpolation between (i) a bias corrected version of the respective estimated first moment of the gradient for the weight and (ii) the respective current gradient of the loss function with respect to the weight, based on the teachings of Ziyin. One of ordinary skill in the art would have been motivated to make this modification in order to adaptively optimize with better stability and flexibility, as suggested by Ziyin (page 2, second paragraph, lines 1-2).
Regarding claim 5, Case, in view of Bernstein, in further view of Zou, discloses the method of claim 3 (and thus the rejection of claim 3 is incorporated).
Case, in view of Bernstein, discloses determining, for each weight, a respective temporary estimated first moment from the respective estimated first moment of the gradient for the weight and the respective current gradient of the loss function with respect to the weight in claim 1. Case, in view of Bernstein, in further view of Zou, does not disclose computing, in accordance with a first interpolation weight, a first interpolation between (i) the respective estimated first moment of the gradient for the weight and (ii) the respective current gradient of the loss function with respect to the weight.
Ziyin discloses:
computing, in accordance with a first interpolation weight, a first interpolation between (i) the respective estimated first moment of the gradient for the weight and (ii) the respective current gradient of the loss function with respect to the weight (Algorithm 1, and page 4, penultimate paragraph, lines 1-4 “
PNG
media_image3.png
155
892
media_image3.png
Greyscale
” teaches a first interpolation weight μ to compute a first interpolation between the respective estimated first moment of the gradient for the weight and the respective current gradient of the loss function with respect to the weight).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Case, in view of Bernstein, in further view of Zou, to include computing, in accordance with a first interpolation weight, a first interpolation between (i) the respective estimated first moment of the gradient for the weight and (ii) the respective current gradient of the loss function with respect to the weight, based on the teachings of Ziyin. One of ordinary skill in the art would have been motivated to make this modification in order to adaptively optimize with better stability and flexibility, as suggested by Ziyin (page 2, second paragraph, lines 1-2).
Regarding claim 6, Case, in view of Bernstein, in further view of Zou, in further view of Ziyin, discloses the method of claim 5 (and thus the rejection of claim 5 is incorporated). Ziyin further discloses:
wherein the first interpolation weight is different from the second interpolation weight (page 6, first paragraph, lines 3-4 “The default LaProp hyperparameters we recommend are … μ = 0.8 -0.9, v = 0.96 - 0.999…” teaches a second interpolation weight v, which is different from the first interpolation weight μ).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Case, in view of Bernstein, in further view of Zou, to include wherein the first interpolation weight is different from the second interpolation weight, based on the teachings of Ziyin. One of ordinary skill in the art would have been motivated to make this modification in order to adaptively optimize with better stability and flexibility, as suggested by Ziyin (page 2, second paragraph, lines 1-2).
Regarding claim 7, Case, in view of Bernstein, in further view of Zou, in further view of Ziyin, discloses the method of claim 6 (and thus the rejection of claim 6 is incorporated). Ziyin further discloses:
wherein the first interpolation weight is lower than the second interpolation weight (page 6, first paragraph, lines 3-4 “The default LaProp hyperparameters we recommend are … μ = 0.8 -0.9, v = 0.96 - 0.999…” teaches the first interpolation weight μ to be lower than v).
Regarding claim 8, Case, in view of Bernstein, in further view of Zou, in further view of Ziyin, discloses the method of claim 7 (and thus the rejection of claim 7 is incorporated). Ziyin further discloses:
wherein the first interpolation weight is equal to .9 (page 6, first paragraph, lines 3-4 “The default LaProp hyperparameters we recommend are … μ = 0.8 - 0.9…” teaches the first interpolation weight μ to be 0.9).
Regarding claim 9, Case, in view of Bernstein, in further view of Zou, in further view of Ziyin, discloses the method of claim 8 (and thus the rejection of claim 8 is incorporated). Ziyin further discloses:
wherein the second interpolation weight is equal to .99 (page 6, first paragraph, lines 3-4 “The default LaProp hyperparameters we recommend are … v = 0.96 - 0.999…” teaches the second interpolation weight v to be 0.9).
Claims 17-20 are substantially similar to claims 4-7, and thus are rejected on the same basis as claims 4-7.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's
disclosure.
U.S. Pub No. 11347997 B1: Song et al. teaches neural networks, gradient update and moment.
Berrada et al. (“Training Neural Networks for and by Interpolation”) teaches neural networks, interpolation and moment.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HUMAIRA ZAHIN MAUNI whose telephone number is (703)756-5654. The examiner can normally be reached Monday - Friday, 9 am - 5 pm (ET).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, MATT ELL can be reached at (571) 270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/H.Z.M./Examiner, Art Unit 2141
/MATTHEW ELL/Supervisory Patent Examiner, Art Unit 2141
Prosecution Insights