Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in response to the application and claims filed 05/20/2024. Claims 1-20 are pending and have been examined. Claims 1-20 are rejected.
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The present application claims foreign priority based on Chinese Application CN202410147440.3 filed Filing Date 02/01/2024. The examiner notes that a certified copy (in Chinese) of the above-noted application was retrieved on 07/07/2024. Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 06/25/2025 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Objections
Claim 9 objected to because of the following informalities:
“causes … to perform performing,” is grammatically redundant.
Appropriate correction is required.
Specification
The disclosure is objected to because of the following informalities:
[0032] "is the final global iteration, that is, the final global iteration" delete the duplicated phrase
[0073] "the asynchronous ALRReduce operation" replace "ALRReduce" with "ALLReduce"
[0076] "interior iteration" and "stale exterior momentum" replace with "internal iteration" and "stale external momentum"
[0088] "ARR represents asynchronous ALLReduce" replace "ARR" with "AAR" (FIG. 5 and the same paragraph use "Param.AAR")
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 1, 8, 15 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1, 8, 15 recites the limitation "the global iteration". There is insufficient antecedent basis for this limitation in the claim. Claims 2-7, 9-14, 16-20 are also rejected for being dependent on claim 1, 8, and 15 respectively.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Examiner’s Note: Some rejections will include an Examiner’s Note (labeled ‘EN’) to provide additional context or rationale explaining the basis for the rejection.
Claim 1:
Step 1: The claim recites a method; therefore, it is directed to the statutory category of processes.
Step 2A prong 1: If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the "Mental Processes" grouping of abstract ideas. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation by mathematical calculation but for the recitation of generic computer components, then it falls within the "Mathematical Concepts" grouping of abstract ideas. The claim recites the following abstract ideas:
"determining a first ALLReduce model parameter value of the current global iteration according to the first node model parameter value and the second node model parameter value," (This limitation falls within the mental processes and mathematical concepts groupings. The ALLReduce model parameter value is an average of the node model parameter values (see paragraph [0044]); a person mentally or with a pen and paper can average two values.)
"in response to the current global iteration being a non-first global iteration," (This limitation falls within the mental processes grouping because a person mentally or with a pen and paper can judge whether the current iteration is the first iteration.)
"performing the external iteration by using the second ALLReduce model parameter value to obtain a target model parameter value of the current global iteration," (This limitation falls within the mental processes and mathematical concepts groupings. The external iteration is a momentum update of the parameter value performed according to formulas (4) and (5) (see paragraph [0031]); a person mentally or with a pen and paper can evaluate these formulas to obtain an updated value.)
"wherein the target model parameter value of the current global iteration is configured as an initial model parameter value of a next global iteration or a model parameter value after training on the preset model is finished." (This limitation falls within the mental processes grouping because a person mentally or with a pen and paper can carry a calculated value over as the starting value of the next iteration, or keep it as the final result.)
Step 2A prong 2: This judicial exception is not integrated into a practical application. The claim further recites:
"A distributed model training method, applied to a first computing node in a distributed system, comprising:" (The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)). -- EN: The "first computing node" and the "distributed system" are generic computer components that merely limit the abstract idea to the technological environment of distributed model training.)
"through a computation process," "through the computation process," (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The "computation process" is a generic computer process recited at a high level of generality.)
"performing, (...) training of an internal iteration with a preset number of internal iterations on a preset model, to obtain a first node model parameter value of a current global iteration, wherein the preset model comprises a machine learning model, and the global iteration comprises the internal iteration and an external iteration;" (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The claim denotes generic training of a generic, off the shelf machine learning model with no additional details or limitations beyond repeating the generic training a preset number of times.)
"acquiring, through a communication process, a second node model parameter value of the current global iteration of a second computing node in the distributed system," (Insignificant extra-solution activity as the limitation amounts to receiving data (MPEP 2106.05(g)(3)). -- EN: The limitation amounts to receiving a parameter value from another computing node over a network; the "communication process" is a generic computer process recited at a high level of generality.)
"wherein the communication process runs in parallel with the computation process," (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: Running two generic computer processes concurrently is a generic computer function; the claim does not recite how the parallel execution is achieved.)
"the second computing node is a computing node in the distributed system except the first computing node," (The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)). -- EN: The limitation merely describes the technological environment, i.e., that the distributed system includes another generic computing node.)
"and the second node model parameter value is obtained after the second computing node performs training on the preset model with the preset number of internal iterations;" (Data Gathering - the limitation merely specifies the source of the gathered data, recited at a high level of generality, and thus is insignificant extra-solution activity (MPEP 2106.05(g)). -- EN: The limitation merely specifies the source of the received data, i.e., generic training on another generic computing node.)
"acquiring, (...) a second ALLReduce model parameter value of a last global iteration," (Insignificant extra-solution activity as the limitation amounts to receiving (retrieving) data (MPEP 2106.05(g)(3)). -- EN: The limitation amounts to retrieving a previously calculated value.)
Step 2B:
"A distributed model training method, applied to a first computing node in a distributed system, comprising:" (The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)). -- EN: The "first computing node" and the "distributed system" are generic computer components that merely limit the abstract idea to the technological environment of distributed model training.)
"through a computation process," "through the computation process," (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The "computation process" is a generic computer process recited at a high level of generality.)
"performing, (...) training of an internal iteration with a preset number of internal iterations on a preset model, to obtain a first node model parameter value of a current global iteration, wherein the preset model comprises a machine learning model, and the global iteration comprises the internal iteration and an external iteration;" (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The claim denotes generic training of a generic, off the shelf machine learning model with no additional details or limitations beyond repeating the generic training a preset number of times.)
"acquiring, through a communication process, a second node model parameter value of the current global iteration of a second computing node in the distributed system," (MPEP 2106.05(d)(II) indicates that receiving or transmitting data over a network is a well-understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed limitation is well-understood, routine, conventional activity is supported under Berkheimer.)
"wherein the communication process runs in parallel with the computation process," (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: Running two generic computer processes concurrently is a generic computer function; the claim does not recite how the parallel execution is achieved.)
"the second computing node is a computing node in the distributed system except the first computing node," (The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)). -- EN: The limitation merely describes the technological environment, i.e., that the distributed system includes another generic computing node.)
"and the second node model parameter value is obtained after the second computing node performs training on the preset model with the preset number of internal iterations;" (This falls under a well-understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). See MPEP 2106.05(d)(II). Thereby, a conclusion that the claimed limitation is well-understood, routine, conventional activity is supported under Berkheimer.)
"acquiring, (...) a second ALLReduce model parameter value of a last global iteration," (MPEP 2106.05(d)(II) indicates that merely gathering data, and storing and retrieving information in memory, are well-understood, routine, conventional functions when they are claimed in a merely generic manner (as they are in the present claim). Thereby, a conclusion that the claimed limitation is well-understood, routine, conventional activity is supported under Berkheimer.)
The additional elements considered individually or in combination do not amount to significantly more than the judicial exception. Therefore, the claim is not patent eligible.
Claim 2:
Step 1: A process, as above.
Step 2A prong 1: See the rejection of Claim 1 above, which claim 2 depends on. Claim 2 further recites:
"computing, (...) a target momentum of the current global iteration according to an initial momentum of the current global iteration and the second ALLReduce model parameter value, wherein the target momentum of the current global iteration is configured as an initial momentum of the next global iteration; and" (This limitation falls within the mathematical concepts grouping because it involves calculating a momentum value from a previous momentum value and a parameter value according to a formula, and carrying the result over to the next iteration.)
"computing, (...) the target model parameter value of the current global iteration according to the target momentum of the current global iteration and an initial model parameter value of the current global iteration." (This limitation falls within the mathematical concepts grouping because it involves calculating an updated parameter value from the momentum value and the initial parameter value according to a formula.)
Step 2A prong 2: This judicial exception is not integrated into a practical application. The claim further recites:
"wherein performing, through the computation process, the external iteration by using the second ALLReduce model parameter value to obtain the target model parameter value of the current global iteration comprises:" (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The claim recites the generic "computation process" as a tool to perform the recited mathematical calculations.)
"through the computation process," "through the computation process," (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The "computation process" is a generic computer process recited at a high level of generality.)
Step 2B:
"wherein performing, through the computation process, the external iteration by using the second ALLReduce model parameter value to obtain the target model parameter value of the current global iteration comprises:" (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The claim recites the generic "computation process" as a tool to perform the recited mathematical calculations.)
"through the computation process," "through the computation process," (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The "computation process" is a generic computer process recited at a high level of generality.)
The additional elements considered individually or in combination do not amount to significantly more than the judicial exception. Therefore, the claim is not patent eligible.
Claim 3:
Step 1: A process, as above.
Step 2A prong 1: See the rejection of Claim 2 above, which claim 3 depends on. Claim 3 further recites:
"determining, (...) a delay penalty amount of the current global iteration; and" (This is interpreted as a mental process and recitation of mathematical concepts. A person mentally or with a pen and paper can determine a penalty amount by evaluating a formula (see formula (7) at paragraph [0075]).)
"computing, (...) the target momentum of the current global iteration according to the initial momentum of the current global iteration, the delay penalty amount of the current global iteration, and the second ALLReduce model parameter value." (This limitation falls within the mathematical concepts grouping because it involves calculating a momentum value from a previous momentum value, a penalty amount, and a parameter value according to a formula.)
Step 2A prong 2: This judicial exception is not integrated into a practical application. The claim further recites:
"wherein computing, through the computation process, the target momentum of the current global iteration according to the initial momentum of the current global iteration and the second ALLReduce model parameter value comprises:" (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The claim recites the generic "computation process" as a tool to perform the recited mathematical calculations.)
"through the computation process," "through the computation process," (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The "computation process" is a generic computer process recited at a high level of generality.)
Step 2B:
"wherein computing, through the computation process, the target momentum of the current global iteration according to the initial momentum of the current global iteration and the second ALLReduce model parameter value comprises:" (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The claim recites the generic "computation process" as a tool to perform the recited mathematical calculations.)
"through the computation process," "through the computation process," (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The "computation process" is a generic computer process recited at a high level of generality.)
The additional elements considered individually or in combination do not amount to significantly more than the judicial exception. Therefore, the claim is not patent eligible.
Claim 4:
Step 1: A process, as above.
Step 2A prong 1: See the rejection of Claim 3 above, which claim 4 depends on. Claim 4 further recites:
"determining, (...) a variation amplitude of the initial model parameter value of the current global iteration compared with an initial model parameter value of the last global iteration;" (This limitation falls within the mathematical concepts grouping because it involves calculating the difference between two parameter values.)
"determining, (...) a maximum distance between model parameter values from all model parameter values that are capable of being traversed in an internal iteration of the last global iteration; and" (This is interpreted as a mental process and recitation of mathematical concepts. A person mentally or with a pen and paper can compare parameter values and identify the largest distance between them.)
"determining, (...) the delay penalty amount of the current global iteration according to a quotient of the variation amplitude and the maximum distance;" (This limitation falls within the mathematical concepts grouping because it involves dividing one value by another.)
"computing, (...) a first difference value between the initial model parameter value of the last global iteration and the second ALLReduce model parameter value;" (This limitation falls within the mathematical concepts grouping because it involves subtracting one value from another.)
"computing, (...) a first product of a reciprocal of the delay penalty amount of the current global iteration and the first difference value; and" (This limitation falls within the mathematical concepts grouping because it involves taking a reciprocal and multiplying it by another value.)
"computing, (...) a second product of the initial momentum of the current global iteration and a preset momentum factor, and computing a sum of the second product and the first product to obtain the target momentum of the current global iteration." (This limitation falls within the mathematical concepts grouping because it involves multiplying two values and adding the result to another value.)
Step 2A prong 2: This judicial exception is not integrated into a practical application. The claim further recites:
"wherein determining, through the computation process, the delay penalty amount of the current global iteration comprises:" "wherein computing, through the computation process, the target momentum of the current global iteration according to the initial momentum of the current global iteration, the delay penalty amount of the current global iteration, and the second ALLReduce model parameter value comprises:" (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The claim recites the generic "computation process" as a tool to perform the recited mathematical calculations.)
"through the computation process," "through the computation process," "through the computation process," "through the computation process," "through the computation process," "through the computation process," (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The "computation process" is a generic computer process recited at a high level of generality.)
Step 2B:
"wherein determining, through the computation process, the delay penalty amount of the current global iteration comprises:" "wherein computing, through the computation process, the target momentum of the current global iteration according to the initial momentum of the current global iteration, the delay penalty amount of the current global iteration, and the second ALLReduce model parameter value comprises:" (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The claim recites the generic "computation process" as a tool to perform the recited mathematical calculations.)
"through the computation process," "through the computation process," "through the computation process," "through the computation process," "through the computation process," "through the computation process," (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The "computation process" is a generic computer process recited at a high level of generality.)
The additional elements considered individually or in combination do not amount to significantly more than the judicial exception. Therefore, the claim is not patent eligible.
Claim 5:
Step 1: A process, as above.
Step 2A prong 1: See the rejection of Claim 2 above, which claim 5 depends on. Claim 5 further recites:
"performing, (...) momentum clipping on the target momentum of the current global iteration so that a value of the momentum-clipped target momentum is in a preset interval; and" (This is interpreted as a mental process and recitation of mathematical concepts. A person mentally or with a pen and paper can compare a value with the limits of an interval and replace it with the limit when it falls outside the interval.)
"computing, (...) a third product of the momentum-clipped target momentum of the current global iteration and a preset external iteration learning rate, and computing a difference value between the initial model parameter value of the current global iteration and the third product to obtain the target model parameter value of the current global iteration." (This limitation falls within the mathematical concepts grouping because it involves multiplying two values and subtracting the result from another value.)
Step 2A prong 2: This judicial exception is not integrated into a practical application. The claim further recites:
"wherein computing, through the computation process, the target model parameter value of the current global iteration according to the target momentum of the current global iteration and the initial model parameter value of the current global iteration comprises:" (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The claim recites the generic "computation process" as a tool to perform the recited mathematical calculations.)
"through the computation process," "through the computation process," (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The "computation process" is a generic computer process recited at a high level of generality.)
Step 2B:
"wherein computing, through the computation process, the target model parameter value of the current global iteration according to the target momentum of the current global iteration and the initial model parameter value of the current global iteration comprises:" (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The claim recites the generic "computation process" as a tool to perform the recited mathematical calculations.)
"through the computation process," "through the computation process," (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The "computation process" is a generic computer process recited at a high level of generality.)
The additional elements considered individually or in combination do not amount to significantly more than the judicial exception. Therefore, the claim is not patent eligible.
Claim 6:
Step 1: A process, as above.
Step 2A prong 1: See the rejection of Claim 3 above, which claim 6 depends on. Claim 6 further recites:
"performing, (...) momentum clipping on the target momentum of the current global iteration so that a value of the momentum-clipped target momentum is in a preset interval; and" (This is interpreted as a mental process and recitation of mathematical concepts. A person mentally or with a pen and paper can compare a value with the limits of an interval and replace it with the limit when it falls outside the interval.)
"computing, (...) a third product of the momentum-clipped target momentum of the current global iteration and a preset external iteration learning rate, and computing a difference value between the initial model parameter value of the current global iteration and the third product to obtain the target model parameter value of the current global iteration." (This limitation falls within the mathematical concepts grouping because it involves multiplying two values and subtracting the result from another value.)
Step 2A prong 2: This judicial exception is not integrated into a practical application. The claim further recites:
"wherein computing, through the computation process, the target model parameter value of the current global iteration according to the target momentum of the current global iteration and the initial model parameter value of the current global iteration comprises:" (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The claim recites the generic "computation process" as a tool to perform the recited mathematical calculations.)
"through the computation process," "through the computation process," (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The "computation process" is a generic computer process recited at a high level of generality.)
Step 2B:
"wherein computing, through the computation process, the target model parameter value of the current global iteration according to the target momentum of the current global iteration and the initial model parameter value of the current global iteration comprises:" (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The claim recites the generic "computation process" as a tool to perform the recited mathematical calculations.)
"through the computation process," "through the computation process," (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The "computation process" is a generic computer process recited at a high level of generality.)
The additional elements considered individually or in combination do not amount to significantly more than the judicial exception. Therefore, the claim is not patent eligible.
Claim 7:
Step 1: A process, as above.
Step 2A prong 1: See the rejection of Claim 4 above, which claim 7 depends on. Claim 7 further recites:
"performing, (...) momentum clipping on the target momentum of the current global iteration so that a value of the momentum-clipped target momentum is in a preset interval; and" (This is interpreted as a mental process and recitation of mathematical concepts. A person mentally or with a pen and paper can compare a value with the limits of an interval and replace it with the limit when it falls outside the interval.)
"computing, (...) a third product of the momentum-clipped target momentum of the current global iteration and a preset external iteration learning rate, and computing a difference value between the initial model parameter value of the current global iteration and the third product to obtain the target model parameter value of the current global iteration." (This limitation falls within the mathematical concepts grouping because it involves multiplying two values and subtracting the result from another value.)
Step 2A prong 2: This judicial exception is not integrated into a practical application. The claim further recites:
"wherein computing, through the computation process, the target model parameter value of the current global iteration according to the target momentum of the current global iteration and the initial model parameter value of the current global iteration comprises:" (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The claim recites the generic "computation process" as a tool to perform the recited mathematical calculations.)
"through the computation process," "through the computation process," (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The "computation process" is a generic computer process recited at a high level of generality.)
Step 2B:
"wherein computing, through the computation process, the target model parameter value of the current global iteration according to the target momentum of the current global iteration and the initial model parameter value of the current global iteration comprises:" (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The claim recites the generic "computation process" as a tool to perform the recited mathematical calculations.)
"through the computation process," "through the computation process," (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The "computation process" is a generic computer process recited at a high level of generality.)
The additional elements considered individually or in combination do not amount to significantly more than the judicial exception. Therefore, the claim is not patent eligible.
Claim 8:
Step 1: The claim recites an electronic device comprising at least one processor and a memory; therefore, it is directed to the statutory category of machine.
Step 2A prong 1: If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the "Mental Processes" grouping of abstract ideas. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation by mathematical calculation but for the recitation of generic computer components, then it falls within the "Mathematical Concepts" grouping of abstract ideas. The claim recites the following abstract ideas:
"determining a first ALLReduce model parameter value of the current global iteration according to the first node model parameter value and the second node model parameter value," (This limitation falls within the mental processes and mathematical concepts groupings. The ALLReduce model parameter value is an average of the node model parameter values (see paragraphs [0029] and [0044]); a person mentally or with a pen and paper can average two values.)
"in response to the current global iteration being a non-first global iteration," (This limitation falls within the mental processes grouping because a person mentally or with a pen and paper can judge whether the current iteration is the first iteration.)
"performing the external iteration by using the second ALLReduce model parameter value to obtain a target model parameter value of the current global iteration," (This limitation falls within the mental processes and mathematical concepts groupings. The external iteration is a momentum update of the parameter value performed according to formulas (4) and (5) (see paragraph [0031]); a person mentally or with a pen and paper can evaluate these formulas to obtain an updated value.)
"wherein the target model parameter value of the current global iteration is configured as an initial model parameter value of a next global iteration or a model parameter value after training on the preset model is finished." (This limitation falls within the mental processes grouping because a person mentally or with a pen and paper can carry a calculated value over as the starting value of the next iteration, or keep it as the final result.)
Step 2A prong 2: This judicial exception is not integrated into a practical application. The claim further recites:
"An electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program, when executed by the at least one processor, causes the at least one processor to perform:" (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The claim recites a generic off the shelf processor and memory as tools to perform the recited abstract ideas.)
"by a first computing node in a distributed system" (The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)). -- EN: The "first computing node" and the "distributed system" are generic computer components that merely limit the abstract idea to the technological environment of distributed model training.)
"through a computation process," "through the computation process," (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The "computation process" is a generic computer process recited at a high level of generality.)
"performing, (...) training of an internal iteration with a preset number of internal iterations on a preset model, to obtain a first node model parameter value of a current global iteration, wherein the preset model comprises a machine learning model, and the global iteration comprises the internal iteration and an external iteration;" (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The claim denotes generic training of a generic, off the shelf machine learning model with no additional details or limitations beyond repeating the generic training a preset number of times.)
"acquiring, through a communication process, a second node model parameter value of the current global iteration of a second computing node in the distributed system," (Insignificant extra-solution activity as the limitation amounts to receiving data (MPEP 2106.05(g)(3)). -- EN: The limitation amounts to receiving a parameter value from another computing node over a network; the "communication process" is a generic computer process recited at a high level of generality.)
"wherein the communication process runs in parallel with the computation process," (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: Running two generic computer processes concurrently is a generic computer function; the claim does not recite how the parallel execution is achieved.)
"the second computing node is a computing node in the distributed system except the first computing node," (The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)). -- EN: The limitation merely describes the technological environment, i.e., that the distributed system includes another generic computing node.)
"and the second node model parameter value is obtained after the second computing node performs training on the preset model with the preset number of internal iterations;" (Data Gathering - the limitation merely specifies the source of the gathered data, recited at a high level of generality, and thus is insignificant extra-solution activity (MPEP 2106.05(g)). -- EN: The limitation merely specifies the source of the received data, i.e., generic training on another generic computing node.)
"acquiring, (...) a second ALLReduce model parameter value of a last global iteration," (Insignificant extra-solution activity as the limitation amounts to receiving (retrieving) data (MPEP 2106.05(g)(3)). -- EN: The limitation amounts to retrieving a previously calculated value.)
Step 2B:
"An electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program, when executed by the at least one processor, causes the at least one processor to perform:" (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The claim recites a generic off the shelf processor and memory as tools to perform the recited abstract ideas.)
"by a first computing node in a distributed system" (The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)). -- EN: The "first computing node" and the "distributed system" are generic computer components that merely limit the abstract idea to the technological environment of distributed model training.)
"through a computation process," "through the computation process," (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The "computation process" is a generic computer process recited at a high level of generality.)
"performing, (...) training of an internal iteration with a preset number of internal iterations on a preset model, to obtain a first node model parameter value of a current global iteration, wherein the preset model comprises a machine learning model, and the global iteration comprises the internal iteration and an external iteration;" (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The claim denotes generic training of a generic, off the shelf machine learning model with no additional details or limitations beyond repeating the generic training a preset number of times.)
"acquiring, through a communication process, a second node model parameter value of the current global iteration of a second computing node in the distributed system," (MPEP 2106.05(d)(II) indicates that receiving or transmitting data over a network is a well-understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed limitation is well-understood, routine, conventional activity is supported under Berkheimer.)
"wherein the communication process runs in parallel with the computation process," (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: Running two generic computer processes concurrently is a generic computer function; the claim does not recite how the parallel execution is achieved.)
"the second computing node is a computing node in the distributed system except the first computing node," (The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)). -- EN: The limitation merely describes the technological environment, i.e., that the distributed system includes another generic computing node.)
"and the second node model parameter value is obtained after the second computing node performs training on the preset model with the preset number of internal iterations;" (This falls under a well-understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). See MPEP 2106.05(d)(II). Thereby, a conclusion that the claimed limitation is well-understood, routine, conventional activity is supported under Berkheimer.)
"acquiring, (...) a second ALLReduce model parameter value of a last global iteration," (MPEP 2106.05(d)(II) indicates that merely gathering data, and storing and retrieving information in memory, are well-understood, routine, conventional functions when they are claimed in a merely generic manner (as they are in the present claim). Thereby, a conclusion that the claimed limitation is well-understood, routine, conventional activity is supported under Berkheimer.)
The additional elements considered individually or in combination do not amount to significantly more than the judicial exception. Therefore, the claim is not patent eligible.
Claim 9:
Claim 9 is an electronic device claim that recites substantially the same limitations as claim 2. Therefore, claim 9 is rejected under the same rationale as claim 2.
Claim 10:
Claim 10 is an electronic device claim that recites substantially the same limitations as claim 3. Therefore, claim 10 is rejected under the same rationale as claim 3.
Claim 11:
Claim 11 is an electronic device claim that recites substantially the same limitations as claim 4. Therefore, claim 11 is rejected under the same rationale as claim 4.
Claim 12:
Claim 12 is an electronic device claim that recites substantially the same limitations as claim 5. Therefore, claim 12 is rejected under the same rationale as claim 5.
Claim 13:
Claim 13 is an electronic device claim that recites substantially the same limitations as claim 6. Therefore, claim 13 is rejected under the same rationale as claim 6.
Claim 14:
Claim 14 is an electronic device claim that recites substantially the same limitations as claim 7. Therefore, claim 14 is rejected under the same rationale as claim 7.
Claim 15:
Step 1: The claim recites a non-transitory computer-readable storage medium; therefore, it is directed to the statutory category of manufacture.
Step 2A prong 1: If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the "Mental Processes" grouping of abstract ideas. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation by mathematical calculation but for the recitation of generic computer components, then it falls within the "Mathematical Concepts" grouping of abstract ideas. The claim recites the following abstract ideas:
"determining a first ALLReduce model parameter value of the current global iteration according to the first node model parameter value and the second node model parameter value," (This limitation falls within the mental processes and mathematical concepts groupings. The ALLReduce model parameter value is an average of the node model parameter values (see paragraphs [0029] and [0044]); a person mentally or with a pen and paper can average two values.)
"in response to the current global iteration being a non-first global iteration," (This limitation falls within the mental processes grouping because a person mentally or with a pen and paper can judge whether the current iteration is the first iteration.)
"performing the external iteration by using the second ALLReduce model parameter value to obtain a target model parameter value of the current global iteration," (This limitation falls within the mental processes and mathematical concepts groupings. The external iteration is a momentum update of the parameter value performed according to formulas (4) and (5) (see paragraph [0031]); a person mentally or with a pen and paper can evaluate these formulas to obtain an updated value.)
"wherein the target model parameter value of the current global iteration is configured as an initial model parameter value of a next global iteration or a model parameter value after training on the preset model is finished." (This limitation falls within the mental processes grouping because a person mentally or with a pen and paper can carry a calculated value over as the starting value of the next iteration, or keep it as the final result.)
Step 2A prong 2: This judicial exception is not integrated into a practical application. The claim further recites:
"A non-transitory computer-readable storage medium, storing a computer instruction, wherein the computer instruction is configured to, when executed by a processor, implement:" (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The claim recites a generic storage medium and a generic processor as tools to perform the recited abstract ideas.)
"by a first computing node in a distributed system" (The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)). -- EN: The "first computing node" and the "distributed system" are generic computer components that merely limit the abstract idea to the technological environment of distributed model training.)
"through a computation process," "through the computation process," (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The "computation process" is a generic computer process recited at a high level of generality.)
"performing, (...) training of an internal iteration with a preset number of internal iterations on a preset model, to obtain a first node model parameter value of a current global iteration, wherein the preset model comprises a machine learning model, and the global iteration comprises the internal iteration and an external iteration;" (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The claim denotes generic training of a generic, off the shelf machine learning model with no additional details or limitations beyond repeating the generic training a preset number of times.)
"acquiring, through a communication process, a second node model parameter value of the current global iteration of a second computing node in the distributed system," (Insignificant extra-solution activity as the limitation amounts to receiving data (MPEP 2106.05(g)(3)). -- EN: The limitation amounts to receiving a parameter value from another computing node over a network; the "communication process" is a generic computer process recited at a high level of generality.)
"wherein the communication process runs in parallel with the computation process," (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: Running two generic computer processes concurrently is a generic computer function; the claim does not recite how the parallel execution is achieved.)
"the second computing node is a computing node in the distributed system except the first computing node," (The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)). -- EN: The limitation merely describes the technological environment, i.e., that the distributed system includes another generic computing node.)
"and the second node model parameter value is obtained after the second computing node performs training on the preset model with the preset number of internal iterations;" (Data Gathering - the limitation merely specifies the source of the gathered data, recited at a high level of generality, and thus is insignificant extra-solution activity (MPEP 2106.05(g)). -- EN: The limitation merely specifies the source of the received data, i.e., generic training on another generic computing node.)
"acquiring, (...) a second ALLReduce model parameter value of a last global iteration," (Insignificant extra-solution activity as the limitation amounts to receiving (retrieving) data (MPEP 2106.05(g)(3)). -- EN: The limitation amounts to retrieving a previously calculated value.)
Step 2B:
"A non-transitory computer-readable storage medium, storing a computer instruction, wherein the computer instruction is configured to, when executed by a processor, implement:" (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The claim recites a generic storage medium and a generic processor as tools to perform the recited abstract ideas.)
"by a first computing node in a distributed system" (The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)). -- EN: The "first computing node" and the "distributed system" are generic computer components that merely limit the abstract idea to the technological environment of distributed model training.)
"through a computation process," "through the computation process," (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The "computation process" is a generic computer process recited at a high level of generality.)
"performing, (...) training of an internal iteration with a preset number of internal iterations on a preset model, to obtain a first node model parameter value of a current global iteration, wherein the preset model comprises a machine learning model, and the global iteration comprises the internal iteration and an external iteration;" (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: The claim denotes generic training of a generic, off the shelf machine learning model with no additional details or limitations beyond repeating the generic training a preset number of times.)
"acquiring, through a communication process, a second node model parameter value of the current global iteration of a second computing node in the distributed system," (MPEP 2106.05(d)(II) indicates that receiving or transmitting data over a network is a well-understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed limitation is well-understood, routine, conventional activity is supported under Berkheimer.)
"wherein the communication process runs in parallel with the computation process," (Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). -- EN: Running two generic computer processes concurrently is a generic computer function; the claim does not recite how the parallel execution is achieved.)
"the second computing node is a computing node in the distributed system except the first computing node," (The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)). -- EN: The limitation merely describes the technological environment, i.e., that the distributed system includes another generic computing node.)
"and the second node model parameter value is obtained after the second computing node performs training on the preset model with the preset number of internal iterations;" (This falls under a well-understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). See MPEP 2106.05(d)(II). Thereby, a conclusion that the claimed limitation is well-understood, routine, conventional activity is supported under Berkheimer.)
"acquiring, (...) a second ALLReduce model parameter value of a last global iteration," (MPEP 2106.05(d)(II) indicates that merely gathering data, and storing and retrieving information in memory, are well-understood, routine, conventional functions when they are claimed in a merely generic manner (as they are in the present claim). Thereby, a conclusion that the claimed limitation is well-understood, routine, conventional activity is supported under Berkheimer.)
The additional elements considered individually or in combination do not amount to significantly more than the judicial exception. Therefore, the claim is not patent eligible.
Claim 16:
Claim 16 is a storage medium claim that recites substantially the same limitations as claim 2. Therefore, claim 16 is rejected under the same rationale as claim 2.
Claim 17:
Claim 17 is a storage medium claim that recites substantially the same limitations as claim 3. Therefore, claim 17 is rejected under the same rationale as claim 3.
Claim 18:
Claim 18 is a storage medium claim that recites substantially the same limitations as claim 4. Therefore, claim 18 is rejected under the same rationale as claim 4.
Claim 19:
Claim 19 is a storage medium claim that recites substantially the same limitations as claim 5. Therefore, claim 19 is rejected under the same rationale as claim 5.
Claim 20:
Claim 20 is a storage medium claim that recites substantially the same limitations as claim 6. Therefore, claim 20 is rejected under the same rationale as claim 6.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Examiner’s Note: Some rejections will include an Examiner’s Note (labeled ‘EN’) to provide additional context or rationale explaining the basis for the rejection.
Claim(s) 1 and 2 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al., "SlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentum," (hereinafter SlowMo) in view of Wang et al., "Overlap Local-SGD: An Algorithmic Approach to Hide Communication Delays in Distributed SGD," (hereinafter Overlap).
Regarding claim 1, SlowMo teaches:
A distributed model training method, applied to a first computing node in a distributed system, comprising (Abstract, p. 1, "Distributed optimization is essential for training large models on large datasets."; Section 2, p. 2, "using m worker nodes ... available at the ith worker" -- EN: the m worker nodes denote the computing nodes of the distributed system and worker i denotes the first computing node):
performing, through a computation process, training of an internal iteration with a preset number of internal iterations on a preset model (Section 2, p. 2, "Within each outer iteration, workers first take τ steps of the base optimizer" -- EN: the τ base optimizer steps denote the internal iteration with the preset number τ of internal iterations; Algorithm 1, p. 3, "Inner loop steps τ ... 3 for k ∈ {0, 1, . . . , τ − 1} do 4 Base optimizer step: xt,k+1(i) = xt,k(i) − γt dt,k(i) 5 end" -- EN: the base optimizer steps computed at worker i denote the computation process), to obtain a first node model parameter value of a current global iteration (Section 2, p. 2, "Each worker maintains a local copy of the parameters, xt,k(i) at worker i after the kth inner step of the tth outer iteration" -- EN: the outer iteration t denotes the global iteration, and xt,τ(i), worker i's parameters after its τ inner steps, denotes the first node model parameter value), wherein the preset model comprises a machine learning model (Section 1, p. 2, "training ResNets on CIFAR-10 and ImageNet, and training a transformer on WMT'16 En-De" -- EN: the ResNet and transformer models denote the machine learning model), and the global iteration comprises the internal iteration and an external iteration (Section 2, p. 2, "SlowMo builds on top of a base optimization algorithm and has a nested loop structure shown in Algorithm 1" -- EN: the nested loops denote the internal iteration and the external iteration within one global iteration; Section 2, p. 3, "After the τ base optimizer steps, the workers calculate the average ... using AllReduce (line 6), and then they perform a slow momentum update" -- EN: the slow momentum update of lines 7 and 8 denotes the external iteration);
-- EN: the specification's external iteration denotes any update of the model parameters performed after the internal iterations of a global iteration, the way it’s done is open ended. (instant specification Para. [0031], "a specific processing manner of the external iteration is not limited, for example, the external iteration may be performed in a momentum updating manner"). Therefore, SlowMo's slow momentum update of lines 7 and 8 corresponds to such an update.
acquiring, through a communication process, a second node model parameter value of the current global iteration of a second computing node in the distributed system (Section 1, p. 2, "Periodically, after taking some number τ of base algorithm steps, workers average their parameters using AllReduce and perform a momentum update" -- EN: the AllReduce denotes the communication process; Algorithm 1, p. 3, "6 Exact-Average: xt,τ = (1/m) Σi=1m xt,τ(i)" -- EN: the parameters xt,τ(j) of any other worker j denote the second node model parameter value of a second computing node), and determining a first ALLReduce model parameter value of the current global iteration according to the first node model parameter value and the second node model parameter value (Algorithm 1, p. 3, "6 Exact-Average: xt,τ = (1/m) Σi=1m xt,τ(i)" -- EN: the average xt,τ over worker i's parameters and the other workers' parameters denotes the first ALLReduce model parameter value of the current global iteration; Section 2, p. 3, "After the τ base optimizer steps, the workers calculate the average xt,τ = xt,0 − (γt/m) Σi=1m Σk=0τ−1 dt,k(i) using AllReduce (line 6)" -- EN: calculating the average with AllReduce corresponds to the claimed determining),
-- EN: the specification's acquiring of the second node model parameter value through the communication process denotes the ALLReduce communication operation, initiated after the internal iterations, in which the other nodes' parameter values are obtained and then averaged (instant specification, Para. [0029], "After the internal iteration in the current global iteration has performed the preset number of internal iterations, the ALLReduce communication operation may be initiated"; [0029], "After the communication process acquires the second node model parameter value of the current global iteration, the ALLReduce computation is performed ... The computation manner may be averaging"). SlowMo's Exact-Average of line 6, computed using AllReduce, corresponds to that operation.
the second computing node is a computing node in the distributed system except the first computing node (Section 2, p. 2, "using m worker nodes, where the loss function term Fi and samples ξi from the distribution Di are available at the ith worker" -- EN: each worker j other than worker i denotes the second computing node; Section 4, p. 5, "we train a ResNet-18 (He et al., 2016) using 32 V100 GPUs, located on 32 different worker nodes" -- EN: the worker nodes denote the computing nodes of the distributed system), and the second node model parameter value is obtained after the second computing node performs training on the preset model with the preset number of internal iterations (Algorithm 1, p. 3, "1 for t ∈ {0, 1, . . . , T − 1} at worker i in parallel do ... 3 for k ∈ {0, 1, . . . , τ − 1} do 4 Base optimizer step ... 6 Exact-Average: xt,τ = (1/m) Σi=1m xt,τ(i)" -- EN: every worker's xt,τ(j) enters the Exact-Average of line 6 only after that worker's own τ base optimizer steps, which corresponds to the second node model parameter value being obtained after the preset number of internal iterations); and
SlowMo discloses performing the external iteration … to obtain a target model parameter value of the current global iteration (Algorithm 1, p. 3, "7 Update slow momentum: ut+1 = βut + (1/γt)(xt,0 − xt,τ) 8 Update outer iterates: xt+1,0 = xt,0 − αγt ut+1" -- EN: the slow momentum update of lines 7 and 8 denotes the external iteration, and xt+1,0 denotes the target model parameter value of the current global iteration), wherein the target model parameter value of the current global iteration is configured as an initial model parameter value of a next global iteration or a model parameter value after training on the preset model is finished (Algorithm 1, p. 3, "8 Update outer iterates: xt+1,0 = xt,0 − αγt ut+1" -- EN: xt+1,0 denotes the initial model parameter value of the next outer iteration t + 1; Section 2, p. 2, "Each worker maintains a local copy of the parameters, xt,k(i) at worker i after the kth inner step of the tth outer iteration" -- EN: xt+1,0 is the value at inner step 0 of outer iteration t + 1, before any inner step of that outer iteration has run, which corresponds to the initial model parameter value of a next global iteration).
Lines 7 and 8 use the average xt,τ. SlowMo forms that average at line 6 of the same outer iteration t (Algorithm 1, p. 3, "6 Exact-Average: xt,τ = (1/m) Σi=1m xt,τ(i)" -- EN: the average formed at line 6 of outer iteration t is the average used at lines 7 and 8 of the same outer iteration t). Line 6 is an AllReduce, and the AllReduce is blocking, so each worker stops and waits for the average of the current outer iteration before it runs lines 7 and 8 (Section 1, p. 1, "aggregate these using a blocking communication primitive, AllReduce" -- EN: the AllReduce of line 6 is blocking; Abstract, p. 1, "AllReduce-based methods, which use blocking communication before every update" -- EN: the blocking communication runs before the update of lines 7 and 8, not alongside it; Section 2, p. 3, "the values of xt,0, xt,τ, and hence ut+1 and xt+1,0 are always identical across all workers, since they follow the AllReduce in line 6" -- EN: lines 7 and 8 run after the AllReduce of line 6 has finished and use its result). SlowMo does this in every outer iteration, the first one and each later one alike. The averaged value that the outer update of SlowMo uses is therefore always the average of the current outer iteration, and the outer update waits for that average to arrive. Waiting for the average of the current outer iteration and using it is not acquiring, through the computation process, a second ALLReduce model parameter value of a last global iteration in response to the current global iteration being a non-first global iteration, and it is not performing the external iteration by using the second ALLReduce model parameter value.
SlowMo therefore does not explicitly teach:
wherein the communication process runs in parallel with the computation process,
in response to the current global iteration being a non-first global iteration, acquiring, through the computation process, a second … [ALLReduce] model parameter value of a last global iteration,
and performing the external iteration by using the second … [ALLReduce] model parameter value.
(EN: SlowMo teaches the specific ALLReduce operation, but does not teach the parallel/stale-result arrangement.)
However, Overlap teaches:
wherein the communication process runs in parallel with the computation process (Section 1, p. 1, "After each round of local updates, the anchor model use another thread/process to synchronize. Thus, the communication and computation are decoupled and happen in parallel" -- EN: the synchronization thread denotes the communication process and the local-update thread denotes the computation process; Section 2, p. 2, "Meanwhile, another thread (or process) on each node will synchronize the current local models in parallel and store the average value into the anchor model" -- EN: the synchronization running in parallel with the local updates corresponds to the claimed running in parallel; Fig. 3, p. 2, "There is an extra communication thread on each worker node to perform communication and update anchor models" -- EN: the extra communication thread denotes the communication process; Section 2, p. 2, "This is because the communication operations are non-blocking" -- EN: non-blocking communication does not stop the computation process),
in response to the current global iteration being a non-first global iteration (Section 2, p. 2, "From the update rules (3) to (5), one can observe that the anchor model zaτ, a = 1, 2, 3, . . . will only be used when updating x(a+1)τ(i)" -- EN: each round of τ local updates denotes a global iteration, the anchor of round a is used only in round a + 1, and the average of a preceding round is therefore acquired only from the second round on, which corresponds to the claimed non-first global iteration), acquiring, through the computation process, a second … [ALLReduce] model parameter value of a last global iteration (Section 2, p. 2, "an additional anchor model z, which can be considered as a stale version of the averaged local model" -- EN: the stale anchor z, the averaged model of the preceding round, denotes the model parameter value of a last global iteration; Section 2, p. 2, Eq. (5), "zk+1 = (1/m) Σi=1m xk+1(i) if (k + 1) mod τ = 0" -- EN: the anchor holds the average of all nodes' models formed at the end of a round, SlowMo teaching the ALLReduce as mapped above; Section 2, p. 2, "the updates (3) and (4) do not involve any communication, because each node has one local copy of the anchor model" -- EN: the local-update thread reading the local copy of the anchor denotes the acquiring through the computation process),
and performing the external iteration by using the second … [ALLReduce] model parameter value (Section 2, p. 2, "after every τ local updates, the updated local model x(i) will be pulled towards the anchor model" -- EN: the update closing a round uses the stale anchor, which corresponds to using the last round's value in the external iteration; Section 2, p. 2, Eq. (4), "xk+1(i) = xk+1/2(i) − α(xk+1/2(i) − zk) if (k + 1) mod τ = 0" -- EN: zk, the anchor of the preceding round, denotes the last global iteration's value used in the update, SlowMo teaching the ALLReduce as mapped above).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of SlowMo to include a separate communication thread on each worker that forms the exact average of line 6 while the workers take their next τ base optimizer steps, and an anchor on each worker holding the preceding outer iteration's exact average for use in the present outer update, as taught by Overlap, in order to let the outer update run without waiting on the average (Section 2, p. 2). Overlap itself builds its anchor momentum on the momentum scheme of SlowMo, which it cites as its reference [18] (Section 2, p. 2).
Motivation would eliminate the idle time in which SlowMo's workers wait for the blocking AllReduce to complete because each worker keeps the prior exact average of line 6 locally as its anchor, uses that older value in the slow momentum update of lines 7 and 8 for the present outer step, and lets the separate communication thread finish the current exact average in the background for the next outer step, as taught by Overlap where the benefit is disclosed (Section 2, p. 2 and Fig. 3, p. 2).
Regarding claim 2, SlowMo in view of Overlap teaches all the limitations of claim 1, SlowMo further teaches:
wherein performing, through the computation process, the external iteration by using the second ALLReduce model parameter value to obtain the target model parameter value of the current global iteration comprises:
computing, through the computation process, a target momentum of the current global iteration according to an initial momentum of the current global iteration (SlowMo Algorithm 1, p. 3, "7 Update slow momentum: ut+1 = βut + (1/γt)(xt,0 − xt,τ)" -- EN: ut+1 denotes the target momentum and ut denotes the initial momentum of the current global iteration) and the second ALLReduce model parameter value (SlowMo Algorithm 1, p. 3, "7 Update slow momentum: ut+1 = βut + (1/γt)(xt,0 − xt,τ)" -- EN: in the combination set out for claim 1, the average that line 7 consumes at the close of outer iteration t is the anchor holding the preceding iteration's average, which corresponds to the second ALLReduce model parameter value), wherein the target momentum of the current global iteration is configured as an initial momentum of the next global iteration (SlowMo Algorithm 1, p. 3, "7 Update slow momentum: ut+1 = βut + (1/γt)(xt,0 − xt,τ)" -- EN: the ut+1 written at outer iteration t is the ut read by line 7 at outer iteration t + 1; Section 2, p. 2, "the framework also uses a slow momentum buffer ut which is initialized to u0 = 0; although each worker stores a copy of ut locally, these are always synchronized across all nodes" -- EN: the slow momentum buffer carried from one outer iteration to the next denotes the initial momentum of the next global iteration); and
computing, through the computation process, the target model parameter value of the current global iteration according to the target momentum of the current global iteration (SlowMo Algorithm 1, p. 3, "8 Update outer iterates: xt+1,0 = xt,0 − αγt ut+1" -- EN: xt+1,0 denotes the target model parameter value, computed from the target momentum ut+1) and an initial model parameter value of the current global iteration (SlowMo Algorithm 1, p. 3, "8 Update outer iterates: xt+1,0 = xt,0 − αγt ut+1" -- EN: xt,0 denotes the initial model parameter value of the current global iteration; Section 2, p. 3, "The outer update in line 8 uses the product αγt of the slow and fast learning rates" -- EN: the outer update of line 8 denotes the computing of the target model parameter value).
Claim(s) 3 and 4 is/are rejected under 35 U.S.C. 103 as being unpatentable over SlowMo in view of Overlap in view of Barkai et al., "Gap-Aware Mitigation of Gradient Staleness," (hereinafter Barkai).
Regarding claim 3, SlowMo in view of Overlap teaches all the limitations of claims 1 and 2 including “wherein computing, through the computation process, the target momentum of the current global iteration according to the initial momentum of the current global iteration and the second ALLReduce model parameter value”.
Barkai teaches:
comprises:
determining, through the computation process, a delay penalty amount of the current global iteration (Section 5.1, p. 4, Definition 1, "Gk, the Gap at the kth step, is defined as the minimal number of updates required to traverse the current distance between the master's and worker's parameters using the maximal learning rate and assuming all gradients have an average norm. Gk ∈ ℝ is defined as: Gk = ‖θk − θk−τk‖/C + 1" -- EN: the Gap Gk denotes the delay penalty amount; Abstract, p. 1, "In this paper we define the Gap as a measure of gradient staleness and propose Gap-Aware (GA), a novel asynchronous-distributed method that penalizes stale gradients linearly to the Gap" -- EN: the Gap measures the staleness of the stale contribution and sets its penalty, which corresponds to the claimed delay penalty amount; Algorithm 4, p. 4, "Calculate Gap: Gk = |θk − θk−τk|/C + 1d" -- EN: the master computing the Gap at each step denotes the determining through the computation process; Section 5.1, p. 4, "This implies that ‖θk − θk−τk‖ is a valid (and easily calculated) measure of the gradient staleness" -- EN: the distance between the current parameters θk and the stale parameters θk−τk is the staleness that the Gap measures); and
-- EN: the instant specification's delay penalty amount denotes a quantity measuring the staleness of the parameter version in the external momentum update, by which the stale contribution is penalized (Instant Specification, Para. [0055], "a difference between different parameter versions during the updating of the external momentum is penalized by introducing a delay penalty"; Para. [0073], "a suitable and easy-to-acquire metric for measuring the staleness difference is recorded as Λ, i. e., the delay penalty amount"). Barkai's Gap corresponds to that quantity.
computing, through the computation process, the target momentum of the current global iteration according to the initial momentum of the current global iteration (Algorithm 4, p. 4, "Update momentum vk+1 ← γvk + (1/Gk) ⊙ gki" -- EN: vk+1 denotes the target momentum and vk denotes the initial momentum), the delay penalty amount of the current global iteration (Algorithm 4, p. 4, "Update momentum vk+1 ← γvk + (1/Gk) ⊙ gki" -- EN: the stale contribution gki entering the momentum is divided by the Gap Gk, the delay penalty amount, which corresponds to computing the target momentum according to the delay penalty amount; Section 5.1, p. 5, "To mitigate the gradient staleness, while eliminating the over-penalization and under-penalization, we divide the gradients themselves by their respective Gap. We refer to this method as Gap-Aware (GA)" -- EN: dividing by the Gap is how the delay penalty amount enters the momentum), and the second ALLReduce model parameter value (SlowMo in view of Overlap teaches, as discussed with respect to claim 2, computing the target momentum using the initial momentum and the second ALLReduce model parameter value. Barkai further teaches modifying a momentum update according to a calculated Gap Gk, which measures the staleness of the delayed contribution, by scaling that stale contribution according to 1/Gk (Algorithm 4, p. 4, "Update momentum vk+1 ← γvk + (1/Gk) ⊙ gki"; Section 5.1, page 4-5). Thus, in the proposed combination, Barkai's Gap-based penalty would be applied to the stale contribution derived from the second ALLReduce model parameter value in the SlowMo/Overlap momentum computation, such that the target momentum is computed according to the initial momentum, the delay penalty amount, and the second ALLReduce model parameter value).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of SlowMo in view of Overlap to include a Gap for each outer iteration, equal to the distance the parameters have moved from the version at which the stale difference was formed divided by the maximal distance they can travel between two outer updates plus one, as taught by Barkai, in order to penalize the stale difference in proportion to how far the parameters have moved since it was formed (Section 5.1, page 4-5).
Motivation would improve the final test accuracy of the trained model because the stale contribution is divided by its Gap, which is a penalty that grows with the distance the parameters have actually traveled since the contribution was formed and that eliminates the over-penalization and under-penalization of the stale contribution, as taught by Barkai where the benefit is disclosed (Abstract, p. 1; Section 1, p. 2; and Section 5.1, p. 5).
Regarding claim 4, SlowMo in view of Overlap in view of Barkai teaches all the limitations of claims 1, 2, and 3 including “wherein determining, through the computation process, the delay penalty amount of the current global iteration”.
Barkai further teaches:
comprises:
determining, through the computation process, a variation amplitude of the initial model parameter value of the current global iteration compared with an initial model parameter value of the last global iteration (Barkai Section 5.1, p. 4, "‖θk − θk−τk‖ is a valid (and easily calculated) measure of the gradient staleness" -- EN: the distance ‖θk − θk−τk‖ between the current parameters θk and the parameter version θk−τk from which the stale contribution was formed denotes the variation amplitude; Section 5.1, p. 4, Definition 1, "the minimal number of updates required to traverse the current distance between the master's and worker's parameters" -- EN: the master computing that distance denotes the determining through the computation process; Algorithm 4, p. 4, "Calculate Gap: Gk = |θk − θk−τk|/C + 1d" -- EN: with the Gap taken per outer iteration as set out for claim 3, θk corresponds to the initial model parameter value of the current global iteration and θk−τk to the initial model parameter value of the last global iteration, the version from which the stale difference was formed);
determining, through the computation process, a maximum distance between model parameter values from all model parameter values that are capable of being traversed in an internal iteration of the last global iteration (Barkai Section 5.1, p. 4, "Where C = ηmax Ek[‖∇f(θk−τk)‖] is a constant representing the maximal distance the parameters can travel in a single update, given the gradient's norm is the average gradient norm" -- EN: C denotes the maximum distance the parameters can travel in one update, and with the Gap taken per outer iteration as set out for claim 3 that update is the internal iteration of the last global iteration; Section 5.3, p. 6, "Where C ∈ ℝd is also calculated element-wise" -- EN: C is a computed quantity, which corresponds to the determining); and
determining, through the computation process, the delay penalty amount of the current global iteration according to a quotient of the variation amplitude and the maximum distance (Barkai Section 5.1, p. 4, Definition 1, "Gk = ‖θk − θk−τk‖/C + 1" -- EN: the Gap is the quotient of the variation amplitude and the maximum distance C, plus one, which corresponds to determining the delay penalty amount according to the quotient; Section 5.3, p. 6, Eq. (13), "Every element in Gk is calculated and applied per-element: Gk = |θk − θk−τk|/C + 1d" -- EN: the per-element Gap denotes the delay penalty amount determined for each model parameter);
SlowMo in view of Overlap in view of Barkai teaches, as set out above for claim 3, “wherein computing, through the computation process, the target momentum of the current global iteration according to the initial momentum of the current global iteration, the delay penalty amount of the current global iteration, and the second ALLReduce model parameter value”; SlowMo and Barkai further teach:
comprises:
computing, through the computation process, a first difference value between the initial model parameter value of the last global iteration and the second ALLReduce model parameter value (SlowMo Algorithm 1, p. 3, "7 Update slow momentum: ut+1 = βut + (1/γt)(xt,0 − xt,τ)" - EN: x(t,0) is where outer iteration t starts and x(t,τ) is the average formed from it. In the combination for claim 1 the average is one outer iteration late, so line 7 uses the previous iteration's difference, x(t−1,0) − x(t−1,τ). x(t−1,0) is the initial model parameter value of the last global iteration, and x(t−1,τ), the stale average, is the second ALLReduce model parameter value. Both terms come from the same iteration because SlowMo defines the difference as that iteration's own movement (Section 2, p. 3, "x(t,τ) = x(t,0) − (γ(t)/m) ΣΣ d(t,k)"). This difference is the first difference value, Section 2, p. 3, "Note that the difference xt,0 − xt,τ is scaled by 1/γt in (2) to make the slow momentum buffer invariant to the fast learning rate γt" -- EN: the difference xt,0 − xt,τ is formed as its own quantity before the momentum takes it in, which corresponds to computing the first difference value);
computing, through the computation process, a first product of a reciprocal of the delay penalty amount of the current global iteration and the first difference value (Barkai Algorithm 4, p. 4, "Update momentum vk+1 ← γvk + (1/Gk) ⊙ gki" -- EN: (1/Gk) ⊙ gki denotes the product of the reciprocal of the delay penalty amount Gk and the stale contribution gki, which corresponds to the first product, the stale contribution corresponding to the first difference value as mapped for claim 3; Section 5.1, p. 5, "we divide the gradients themselves by their respective Gap" -- EN: dividing by the Gap denotes multiplying by the reciprocal of the delay penalty amount); and
computing, through the computation process, a second product of the initial momentum of the current global iteration and a preset momentum factor (Barkai Algorithm 4, p. 4, "Update momentum vk+1 ← γvk + (1/Gk) ⊙ gki" -- EN: γvk denotes the second product of the initial momentum vk and the momentum factor γ; Section 3, p. 3, "the momentum iterative update rule uses an exponentially-weighted moving average of gradients called the update vector: vk+1 = γvk + ∇f(θk)" -- EN: γ is the fixed coefficient of the moving average, which corresponds to the preset momentum factor), and computing a sum of the second product and the first product to obtain the target momentum of the current global iteration (Barkai Algorithm 4, p. 4, "Update momentum vk+1 ← γvk + (1/Gk) ⊙ gki" -- EN: vk+1, the sum of the second product γvk and the first product (1/Gk) ⊙ gki, denotes the target momentum).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of SlowMo in view of Overlap to include a Gap for each outer iteration, equal to the distance the parameters have moved from the version at which the stale difference was formed divided by the maximal distance they can travel between two outer updates plus one, as taught by Barkai, in order to penalize the stale difference in proportion to how far the parameters have moved since it was formed (Section 5.1, page 4-5).
Motivation would improve the final test accuracy of the trained model because the stale contribution is divided by its Gap, which is a penalty that grows with the distance the parameters have actually traveled since the contribution was formed and that eliminates the over-penalization and under-penalization of the stale contribution, as taught by Barkai where the benefit is disclosed (Abstract, p. 1; Section 1, p. 2; and Section 5.1, p. 5).
Claim(s) 5 is/are rejected under 35 U.S.C. 103 as being unpatentable over SlowMo in view of Overlap in view of Liu et al., "Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training," (hereinafter Liu).
Regarding claim 5, SlowMo in view of Overlap teaches all the limitations of claims 1 and 2 including “wherein computing, through the computation process, the target model parameter value of the current global iteration according to the target momentum of the current global iteration and the initial model parameter value of the current global iteration”.
Liu teaches:
comprises:
performing, through the computation process, momentum clipping on the target momentum of the current global iteration (Section 2.2, p. 6, "Let mt be the EMA of gradients, mt ← β1 mt−1 + (1−β1)gt, which is the numerator of the update" -- EN: the exponential moving average mt denotes the momentum; Section 2.2, p. 6, Eq. (6), "θt+1 ← θt − ηt · clip(mt / max{γ · ht, ε}, 1)" -- EN: the clip applied to the momentum-based update denotes the momentum clipping, the claim does not exclude Liu's coordinate-wise scaling of mt before the clip; Abstract, p. 1, "The update is the moving average of the gradients divided by the moving average of the estimated Hessian, followed by element-wise clipping" -- EN: the optimizer performs the element-wise clipping) so that a value of the momentum-clipped target momentum is in a preset interval (Section 2.2, p. 6, "For a clipping threshold ρ > 0, let the clipping function be clip(z, ρ) = max{min{z, ρ}, −ρ} where all operations are applied coordinate-wise" -- EN: the range from −ρ to ρ, with ρ preset and set to 1 in Eq. (6), denotes the preset interval in which each clipped coordinate lies); and
computing, through the computation process, a third product of the momentum-clipped target momentum of the current global iteration and a preset external iteration learning rate (Section 2.2, p. 6, Eq. (6), "θt+1 ← θt − ηt · clip(mt / max{γ · ht, ε}, 1)" -- EN: ηt · clip(mt / max{γ · ht, ε}, 1) denotes the third product, with ηt as the preset learning rate; Algorithm 3, p. 3, "θt+1 = θt − ηt · clip(mt / max{γ · ht, ε}, 1)" -- EN: the optimizer performs this computation), and computing a difference value between the initial model parameter value of the current global iteration and the third product to obtain the target model parameter value of the current global iteration (Section 2.2, p. 6, Eq. (6), "θt+1 ← θt − ηt · clip(mt / max{γ · ht, ε}, 1)" -- EN: θt minus the third product denotes the difference value and θt+1 denotes the target model parameter value).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the SlowMo/Overlap system by applying Liu's coordinate-wise clipping technique to the slow momentum before the slow/external learning rate is applied, as taught by Liu, in order to bound the size of every outer update in every parameter dimension (Section 2.2, page 6-7).
Motivation would improve the stability of the training because the clipping holds the worst-case size of the update in all parameter dimensions to at most ρ, as taught by Liu where the benefit is disclosed (Section 2.2, page 6-7).
Claim(s) 6 and 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over SlowMo in view of Overlap in view of Barkai in view of Liu.
Regarding claim 6, SlowMo in view of Overlap in view of Barkai teaches all the limitations of claims 1, 2, and 3 including “wherein computing, through the computation process, the target model parameter value of the current global iteration according to the target momentum of the current global iteration and the initial model parameter value of the current global iteration”.
Liu teaches:
comprises:
performing, through the computation process, momentum clipping on the target momentum of the current global iteration (Section 2.2, p. 6, "Let mt be the EMA of gradients, mt ← β1 mt−1 + (1−β1)gt, which is the numerator of the update" -- EN: the exponential moving average mt denotes the momentum; Section 2.2, p. 6, Eq. (6), "θt+1 ← θt − ηt · clip(mt / max{γ · ht, ε}, 1)" -- EN: the clip applied to the momentum-based update denotes the momentum clipping, the claim does not exclude Liu's coordinate-wise scaling of mt before the clip; Abstract, p. 1, "The update is the moving average of the gradients divided by the moving average of the estimated Hessian, followed by element-wise clipping" -- EN: the optimizer performs the element-wise clipping) so that a value of the momentum-clipped target momentum is in a preset interval (Section 2.2, p. 6, "For a clipping threshold ρ > 0, let the clipping function be clip(z, ρ) = max{min{z, ρ}, −ρ} where all operations are applied coordinate-wise" -- EN: the range from −ρ to ρ, with ρ preset and set to 1 in Eq. (6), denotes the preset interval in which each clipped coordinate lies); and
computing, through the computation process, a third product of the momentum-clipped target momentum of the current global iteration and a preset external iteration learning rate (Section 2.2, p. 6, Eq. (6), "θt+1 ← θt − ηt · clip(mt / max{γ · ht, ε}, 1)" -- EN: ηt · clip(mt / max{γ · ht, ε}, 1) denotes the third product, with ηt as the preset learning rate; Algorithm 3, p. 3, "θt+1 = θt − ηt · clip(mt / max{γ · ht, ε}, 1)" -- EN: the optimizer performs this computation), and computing a difference value between the initial model parameter value of the current global iteration and the third product to obtain the target model parameter value of the current global iteration (Section 2.2, p. 6, Eq. (6), "θt+1 ← θt − ηt · clip(mt / max{γ · ht, ε}, 1)" -- EN: θt minus the third product denotes the difference value and θt+1 denotes the target model parameter value).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of SlowMo in view of Overlap in view of Barkai to include a coordinate-wise clip of the slow momentum to a preset threshold ρ, applied before the slow learning rate multiplies the momentum, as taught by Liu, in order to bound the size of every outer update in every parameter dimension (Section 2.2, page 6-7).
Motivation would improve the stability of the training because the clipping holds the worst-case size of the update in all parameter dimensions to at most ρ, as taught by Liu where the benefit is disclosed (Section 2.2, page 6-7).
Regarding claim 7, SlowMo in view of Overlap in view of Barkai teaches all the limitations of claims 1, 2, 3, and 4 including “wherein computing, through the computation process, the target model parameter value of the current global iteration according to the target momentum of the current global iteration and the initial model parameter value of the current global iteration”.
Liu teaches:
comprises:
performing, through the computation process, momentum clipping on the target momentum of the current global iteration (Section 2.2, p. 6, "Let mt be the EMA of gradients, mt ← β1 mt−1 + (1−β1)gt, which is the numerator of the update" -- EN: the exponential moving average mt denotes the momentum; Section 2.2, p. 6, Eq. (6), "θt+1 ← θt − ηt · clip(mt / max{γ · ht, ε}, 1)" -- EN: the clip applied to the momentum-based update denotes the momentum clipping, the claim does not exclude Liu's coordinate-wise scaling of mt before the clip; Abstract, p. 1, "The update is the moving average of the gradients divided by the moving average of the estimated Hessian, followed by element-wise clipping" -- EN: the optimizer performs the element-wise clipping) so that a value of the momentum-clipped target momentum is in a preset interval (Section 2.2, p. 6, "For a clipping threshold ρ > 0, let the clipping function be clip(z, ρ) = max{min{z, ρ}, −ρ} where all operations are applied coordinate-wise" -- EN: the range from −ρ to ρ, with ρ preset and set to 1 in Eq. (6), denotes the preset interval in which each clipped coordinate lies); and
computing, through the computation process, a third product of the momentum-clipped target momentum of the current global iteration and a preset external iteration learning rate (Section 2.2, p. 6, Eq. (6), "θt+1 ← θt − ηt · clip(mt / max{γ · ht, ε}, 1)" -- EN: ηt · clip(mt / max{γ · ht, ε}, 1) denotes the third product, with ηt as the preset learning rate; Algorithm 3, p. 3, "θt+1 = θt − ηt · clip(mt / max{γ · ht, ε}, 1)" -- EN: the optimizer performs this computation), and computing a difference value between the initial model parameter value of the current global iteration and the third product to obtain the target model parameter value of the current global iteration (Section 2.2, p. 6, Eq. (6), "θt+1 ← θt − ηt · clip(mt / max{γ · ht, ε}, 1)" -- EN: θt minus the third product denotes the difference value and θt+1 denotes the target model parameter value).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of SlowMo in view of Overlap in view of Barkai to include a coordinate-wise clip of the slow momentum to a preset threshold ρ, applied before the slow learning rate multiplies the momentum, as taught by Liu, in order to bound the size of every outer update in every parameter dimension (Section 2.2, page 6-7).
Motivation would improve the stability of the training because the clipping holds the worst-case size of the update in all parameter dimensions to at most ρ, as taught by Liu where the benefit is disclosed (Section 2.2, page 6-7).
Claim(s) 8, 9, 15, and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over SlowMo in view of Overlap in view of Chen et al. (US 2019/0050743 A1) (hereinafter Chen).
Regarding claim 8, it is an electronic device claim that recites substantially the same limitations as claim 1, and is therefore rejected under the same rationale as claim 1. SlowMo in view of Overlap do not explicitly teach:
An electronic device, comprising:
at least one processor; and
a memory communicatively connected to the at least one processor;
wherein the memory stores a computer program executable by the at least one processor, and the computer program, when executed by the at least one processor, causes the at least one processor to perform: the operations set out above for claim 1.
However, Chen teaches:
An electronic device, comprising: ([0077], "FIG. 5 illustrates a device for training a learning machine according to an embodiment of the disclosure. The device 500 shown in FIG. 5 may implement the master node 21, for example" -- EN: the device 500 denotes the electronic device)
at least one processor; and ([0078], "the device 500 may comprise one or more processors 510" -- EN: the one or more processors 510 denote the at least one processor)
a memory communicatively connected to the at least one processor; ([0078], "and a memory 520 for storing computer-executable instructions that, when executed, cause the one or more processors 510 to perform acts included in the method 300" -- EN: the memory 520, whose stored instructions the processors 510 execute, denotes the memory communicatively connected to the at least one processor)
wherein the memory stores a computer program executable by the at least one processor ([0078], "a memory 520 for storing computer-executable instructions that, when executed, cause the one or more processors 510 to perform acts included in the method 300" -- EN: the computer-executable instructions stored in the memory 520 denote the computer program executable by the at least one processor), and the computer program, when executed by the at least one processor, causes the at least one processor to perform: the operations set out above for claim 1 ([0078], "In an aspect, the modules included in the apparatus 400 may be embodied as the computer-executable instructions stored in the memory 520" -- EN: the acts of the training method performed by the processors 510 when the instructions execute correspond to the claimed operations, which in the combination are the operations set out above for claim 1 performed at each of SlowMo's worker nodes).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of SlowMo in view of Overlap to include, at each worker node, one or more processors and a memory holding the training method as computer-executable instructions, as taught by Chen, in order to carry out the method on a programmed computing device ([0078]).
Motivation would allow the training method to be carried out by general-purpose computing hardware because the acts of the method are embodied as computer-executable instructions stored in the memory and executed by the one or more processors, as taught by Chen where the benefit is disclosed ([0076] and [0078]).
Regarding claim 9, it is an electronic device claim of independent claim 8 and recites substantially the same limitations as claim 2, and is therefore rejected under the same rationale as claim 2.
Regarding claim 15, it is a non-transitory computer-readable storage medium claim that recites substantially the same limitations as claim 1, and is therefore rejected under the same rationale as claim 1. SlowMo in view of Overlap do not explicitly teach:
A non-transitory computer-readable storage medium, storing a computer instruction, wherein the computer instruction is configured to, when executed by a processor, implement: the operations set out above for claim 1.
However, Chen teaches:
A non-transitory computer-readable storage medium, storing a computer instruction ([0079], "embodiments of the disclosure may also provide a computer-readable medium having thereon computer-executable instructions" -- EN: the computer-readable medium having the instructions thereon denotes the storage medium storing a computer instruction; [0078], "a memory 520 for storing computer-executable instructions" -- EN: the memory 520 storing the instructions denotes a non-transitory storage medium), wherein the computer instruction is configured to, when executed by a processor, implement: the operations set out above for claim 1 ([0079], "computer-executable instructions that are executable to cause one or more processors to perform acts included in the method 300" -- EN: the acts of the training method performed when the instructions execute correspond to the claimed operations, which in the combination are the operations set out above for claim 1 performed at each of SlowMo's worker nodes).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of SlowMo in view of Overlap to include, at each worker node, a computer-readable medium holding the training method as computer-executable instructions, as taught by Chen, in order to carry out the method on a programmed computing device ([0079]).
Motivation would allow the training method to be carried out by general-purpose computing hardware because the acts of the method are embodied as computer-executable instructions stored on the medium and executed by the one or more processors, as taught by Chen where the benefit is disclosed ([0076], [0078], and [0079]).
Regarding claim 16, it is a non-transitory computer-readable storage medium claim of independent claim 15 and recites substantially the same limitations as claim 2, and is therefore rejected under the same rationale as claim 2.
Claim(s) 10, 11, 17, and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over SlowMo in view of Overlap in view of Chen in view of Barkai.
Regarding claim 10, SlowMo in view of Overlap in view of Chen teaches all the limitations of claims 8 and 9 including “wherein the computer program, when executed by the at least one processor, causes the at least one processor to perform computing, through the computation process, the target momentum of the current global iteration according to the initial momentum of the current global iteration and the second ALLReduce model parameter value”.
Barkai teaches:
by:
determining, through the computation process, a delay penalty amount of the current global iteration (Section 5.1, p. 4, Definition 1, "Gk, the Gap at the kth step, is defined as the minimal number of updates required to traverse the current distance between the master's and worker's parameters using the maximal learning rate and assuming all gradients have an average norm. Gk ∈ ℝ is defined as: Gk = ‖θk − θk−τk‖/C + 1" -- EN: the Gap Gk denotes the delay penalty amount; Abstract, p. 1, "In this paper we define the Gap as a measure of gradient staleness and propose Gap-Aware (GA), a novel asynchronous-distributed method that penalizes stale gradients linearly to the Gap" -- EN: the Gap measures the staleness of the stale contribution and sets its penalty, which corresponds to the claimed delay penalty amount; Algorithm 4, p. 4, "Calculate Gap: Gk = |θk − θk−τk|/C + 1d" -- EN: the master computing the Gap at each step denotes the determining through the computation process; Section 5.1, p. 4, "This implies that ‖θk − θk−τk‖ is a valid (and easily calculated) measure of the gradient staleness" -- EN: the distance between the current parameters θk and the stale parameters θk−τk is the staleness that the Gap measures); and
-- EN: the instant specification's delay penalty amount denotes a quantity measuring the staleness of the parameter version in the external momentum update, by which the stale contribution is penalized (Instant Specification, Para. [0055], "a difference between different parameter versions during the updating of the external momentum is penalized by introducing a delay penalty"; Para. [0073], "a suitable and easy-to-acquire metric for measuring the staleness difference is recorded as Λ, i. e., the delay penalty amount"). Barkai's Gap corresponds to that quantity.
computing, through the computation process, the target momentum of the current global iteration according to the initial momentum of the current global iteration (Algorithm 4, p. 4, "Update momentum vk+1 ← γvk + (1/Gk) ⊙ gki" -- EN: vk+1 denotes the target momentum and vk denotes the initial momentum), the delay penalty amount of the current global iteration (Algorithm 4, p. 4, "Update momentum vk+1 ← γvk + (1/Gk) ⊙ gki" -- EN: the stale contribution gki entering the momentum is divided by the Gap Gk, the delay penalty amount, which corresponds to computing the target momentum according to the delay penalty amount; Section 5.1, p. 5, "To mitigate the gradient staleness, while eliminating the over-penalization and under-penalization, we divide the gradients themselves by their respective Gap. We refer to this method as Gap-Aware (GA)" -- EN: dividing by the Gap is how the delay penalty amount enters the momentum), and the second ALLReduce model parameter value (SlowMo in view of Overlap teaches, as discussed with respect to claims 2 and 9, computing the target momentum using the initial momentum and the second ALLReduce model parameter value. Barkai further teaches modifying a momentum update according to a calculated Gap Gk, which measures the staleness of the delayed contribution, by scaling that stale contribution according to 1/Gk (Algorithm 4, p. 4, "Update momentum vk+1 ← γvk + (1/Gk) ⊙ gki"; Section 5.1, page 4-5). Thus, in the proposed combination, Barkai's Gap-based penalty would be applied to the stale contribution derived from the second ALLReduce model parameter value in the SlowMo/Overlap momentum computation, such that the target momentum is computed according to the initial momentum, the delay penalty amount, and the second ALLReduce model parameter value).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of SlowMo in view of Overlap in view of Chen to include a Gap for each outer iteration, equal to the distance the parameters have moved from the version at which the stale difference was formed divided by the maximal distance they can travel between two outer updates plus one, as taught by Barkai, in order to penalize the stale difference in proportion to how far the parameters have moved since it was formed (Section 5.1, page 4-5).
Motivation would improve the final test accuracy of the trained model because the stale contribution is divided by its Gap, which is a penalty that grows with the distance the parameters have actually traveled since the contribution was formed and that eliminates the over-penalization and under-penalization of the stale contribution, as taught by Barkai where the benefit is disclosed (Abstract, p. 1; Section 1, p. 2; and Section 5.1, p. 5).
Regarding claim 11, SlowMo in view of Overlap in view of Chen in view of Barkai teaches all the limitations of claims 8, 9, and 10 including “wherein the computer program, when executed by the at least one processor, causes the at least one processor to perform determining, through the computation process, the delay penalty amount of the current global iteration”.
Barkai further teaches:
by:
determining, through the computation process, a variation amplitude of the initial model parameter value of the current global iteration compared with an initial model parameter value of the last global iteration (Barkai Section 5.1, p. 4, "‖θk − θk−τk‖ is a valid (and easily calculated) measure of the gradient staleness" -- EN: the distance ‖θk − θk−τk‖ between the current parameters θk and the parameter version θk−τk from which the stale contribution was formed denotes the variation amplitude; Section 5.1, p. 4, Definition 1, "the minimal number of updates required to traverse the current distance between the master's and worker's parameters" -- EN: the master computing that distance denotes the determining through the computation process; Algorithm 4, p. 4, "Calculate Gap: Gk = |θk − θk−τk|/C + 1d" -- EN: with the Gap taken per outer iteration as set out for claim 10, θk corresponds to the initial model parameter value of the current global iteration and θk−τk to the initial model parameter value of the last global iteration, the version from which the stale difference was formed);
determining, through the computation process, a maximum distance between model parameter values from all model parameter values that are capable of being traversed in an internal iteration of the last global iteration (Barkai Section 5.1, p. 4, "Where C = ηmax Ek[‖∇f(θk−τk)‖] is a constant representing the maximal distance the parameters can travel in a single update, given the gradient's norm is the average gradient norm" -- EN: C denotes the maximum distance the parameters can travel in one update, and with the Gap taken per outer iteration as set out for claim 10 that update is the internal iteration of the last global iteration; Section 5.3, p. 6, "Where C ∈ ℝd is also calculated element-wise" -- EN: C is a computed quantity, which corresponds to the determining); and
determining, through the computation process, the delay penalty amount of the current global iteration according to a quotient of the variation amplitude and the maximum distance (Barkai Section 5.1, p. 4, Definition 1, "Gk = ‖θk − θk−τk‖/C + 1" -- EN: the Gap is the quotient of the variation amplitude and the maximum distance C, plus one, which corresponds to determining the delay penalty amount according to the quotient; Section 5.3, p. 6, Eq. (13), "Every element in Gk is calculated and applied per-element: Gk = |θk − θk−τk|/C + 1d" -- EN: the per-element Gap denotes the delay penalty amount determined for each model parameter);
SlowMo in view of Overlap in view of Chen in view of Barkai teaches, as set out above for claim 10, “wherein the computer program, when executed by the at least one processor, causes the at least one processor to perform computing, through the computation process, the target momentum of the current global iteration according to the initial momentum of the current global iteration, the delay penalty amount of the current global iteration, and the second ALLReduce model parameter value”; SlowMo and Barkai further teach:
by:
computing, through the computation process, a first difference value between the initial model parameter value of the last global iteration and the second ALLReduce model parameter value (SlowMo Algorithm 1, p. 3, "7 Update slow momentum: ut+1 = βut + (1/γt)(xt,0 − xt,τ)" - EN: x(t,0) is where outer iteration t starts and x(t,τ) is the average formed from it. In the combination for claim 1 the average is one outer iteration late, so line 7 uses the previous iteration's difference, x(t−1,0) − x(t−1,τ). x(t−1,0) is the initial model parameter value of the last global iteration, and x(t−1,τ), the stale average, is the second ALLReduce model parameter value. Both terms come from the same iteration because SlowMo defines the difference as that iteration's own movement (Section 2, p. 3, "x(t,τ) = x(t,0) − (γ(t)/m) ΣΣ d(t,k)"). This difference is the first difference value. ; Section 2, p. 3, "Note that the difference xt,0 − xt,τ is scaled by 1/γt in (2) to make the slow momentum buffer invariant to the fast learning rate γt" -- EN: the difference xt,0 − xt,τ is formed as its own quantity before the momentum takes it in, which corresponds to computing the first difference value);
computing, through the computation process, a first product of a reciprocal of the delay penalty amount of the current global iteration and the first difference value (Barkai Algorithm 4, p. 4, "Update momentum vk+1 ← γvk + (1/Gk) ⊙ gki" -- EN: (1/Gk) ⊙ gki denotes the product of the reciprocal of the delay penalty amount Gk and the stale contribution gki, which corresponds to the first product, the stale contribution corresponding to the first difference value as mapped for claim 10; Section 5.1, p. 5, "we divide the gradients themselves by their respective Gap" -- EN: dividing by the Gap denotes multiplying by the reciprocal of the delay penalty amount); and
computing, through the computation process, a second product of the initial momentum of the current global iteration and a preset momentum factor (Barkai Algorithm 4, p. 4, "Update momentum vk+1 ← γvk + (1/Gk) ⊙ gki" -- EN: γvk denotes the second product of the initial momentum vk and the momentum factor γ; Section 3, p. 3, "the momentum iterative update rule uses an exponentially-weighted moving average of gradients called the update vector: vk+1 = γvk + ∇f(θk)" -- EN: γ is the fixed coefficient of the moving average, which corresponds to the preset momentum factor), and computing a sum of the second product and the first product to obtain the target momentum of the current global iteration (Barkai Algorithm 4, p. 4, "Update momentum vk+1 ← γvk + (1/Gk) ⊙ gki" -- EN: vk+1, the sum of the second product γvk and the first product (1/Gk) ⊙ gki, denotes the target momentum).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of SlowMo in view of Overlap in view of Chen to include a Gap for each outer iteration, equal to the distance the parameters have moved from the version at which the stale difference was formed divided by the maximal distance they can travel between two outer updates plus one, as taught by Barkai, in order to penalize the stale difference in proportion to how far the parameters have moved since it was formed (Section 5.1, page 4-5).
Motivation would improve the final test accuracy of the trained model because the stale contribution is divided by its Gap, which is a penalty that grows with the distance the parameters have actually traveled since the contribution was formed and that eliminates the over-penalization and under-penalization of the stale contribution, as taught by Barkai where the benefit is disclosed (Abstract, p. 1; Section 1, p. 2; and Section 5.1, p. 5).
Regarding claim 17, it is a non-transitory computer-readable storage medium claim of independent claim 15 and recites substantially the same limitations as claim 10, and is therefore rejected under the same rationale as claim 10.
Regarding claim 18, it is a non-transitory computer-readable storage medium claim of independent claim 15 and recites substantially the same limitations as claim 11, and is therefore rejected under the same rationale as claim 11.
Claim(s) 12 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over SlowMo in view of Overlap in view of Chen in view of Liu.
Regarding claim 12, SlowMo in view of Overlap in view of Chen teaches all the limitations of claims 8 and 9 including “wherein the computer program, when executed by the at least one processor, causes the at least one processor to perform computing, through the computation process, the target model parameter value of the current global iteration according to the target momentum of the current global iteration and the initial model parameter value of the current global iteration”.
Liu teaches:
by:
performing, through the computation process, momentum clipping on the target momentum of the current global iteration (Section 2.2, p. 6, "Let mt be the EMA of gradients, mt ← β1 mt−1 + (1−β1)gt, which is the numerator of the update" -- EN: the exponential moving average mt denotes the momentum; Section 2.2, p. 6, Eq. (6), "θt+1 ← θt − ηt · clip(mt / max{γ · ht, ε}, 1)" -- EN: the clip applied to the momentum-based update denotes the momentum clipping, the claim does not exclude Liu's coordinate-wise scaling of mt before the clip; Abstract, p. 1, "The update is the moving average of the gradients divided by the moving average of the estimated Hessian, followed by element-wise clipping" -- EN: the optimizer performs the element-wise clipping) so that a value of the momentum-clipped target momentum is in a preset interval (Section 2.2, p. 6, "For a clipping threshold ρ > 0, let the clipping function be clip(z, ρ) = max{min{z, ρ}, −ρ} where all operations are applied coordinate-wise" -- EN: the range from −ρ to ρ, with ρ preset and set to 1 in Eq. (6), denotes the preset interval in which each clipped coordinate lies); and
computing, through the computation process, a third product of the momentum-clipped target momentum of the current global iteration and a preset external iteration learning rate (Section 2.2, p. 6, Eq. (6), "θt+1 ← θt − ηt · clip(mt / max{γ · ht, ε}, 1)" -- EN: ηt · clip(mt / max{γ · ht, ε}, 1) denotes the third product, with ηt as the preset learning rate; Algorithm 3, p. 3, "θt+1 = θt − ηt · clip(mt / max{γ · ht, ε}, 1)" -- EN: the optimizer performs this computation), and computing a difference value between the initial model parameter value of the current global iteration and the third product to obtain the target model parameter value of the current global iteration (Section 2.2, p. 6, Eq. (6), "θt+1 ← θt − ηt · clip(mt / max{γ · ht, ε}, 1)" -- EN: θt minus the third product denotes the difference value and θt+1 denotes the target model parameter value).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the SlowMo/Overlap/Chen system by applying Liu's coordinate-wise clipping technique to the slow momentum before the slow/external learning rate is applied, as taught by Liu, in order to bound the size of every outer update in every parameter dimension (Section 2.2, page 6-7).
Motivation would improve the stability of the training because the clipping holds the worst-case size of the update in all parameter dimensions to at most ρ, as taught by Liu where the benefit is disclosed (Section 2.2, page 6-7).
Regarding claim 19, it is a non-transitory computer-readable storage medium claim of independent claim 15 and recites substantially the same limitations as claim 12, and is therefore rejected under the same rationale as claim 12.
Claim(s) 13, 14, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over SlowMo in view of Overlap in view of Chen in view of Barkai in view of Liu.
Regarding claim 13, SlowMo in view of Overlap in view of Chen in view of Barkai teaches all the limitations of claims 8, 9, and 10 including “wherein the computer program, when executed by the at least one processor, causes the at least one processor to perform computing, through the computation process, the target model parameter value of the current global iteration according to the target momentum of the current global iteration and the initial model parameter value of the current global iteration”.
Liu teaches:
by:
performing, through the computation process, momentum clipping on the target momentum of the current global iteration (Section 2.2, p. 6, "Let mt be the EMA of gradients, mt ← β1 mt−1 + (1−β1)gt, which is the numerator of the update" -- EN: the exponential moving average mt denotes the momentum; Section 2.2, p. 6, Eq. (6), "θt+1 ← θt − ηt · clip(mt / max{γ · ht, ε}, 1)" -- EN: the clip applied to the momentum-based update denotes the momentum clipping, the claim does not exclude Liu's coordinate-wise scaling of mt before the clip; Abstract, p. 1, "The update is the moving average of the gradients divided by the moving average of the estimated Hessian, followed by element-wise clipping" -- EN: the optimizer performs the element-wise clipping) so that a value of the momentum-clipped target momentum is in a preset interval (Section 2.2, p. 6, "For a clipping threshold ρ > 0, let the clipping function be clip(z, ρ) = max{min{z, ρ}, −ρ} where all operations are applied coordinate-wise" -- EN: the range from −ρ to ρ, with ρ preset and set to 1 in Eq. (6), denotes the preset interval in which each clipped coordinate lies); and
computing, through the computation process, a third product of the momentum-clipped target momentum of the current global iteration and a preset external iteration learning rate (Section 2.2, p. 6, Eq. (6), "θt+1 ← θt − ηt · clip(mt / max{γ · ht, ε}, 1)" -- EN: ηt · clip(mt / max{γ · ht, ε}, 1) denotes the third product, with ηt as the preset learning rate; Algorithm 3, p. 3, "θt+1 = θt − ηt · clip(mt / max{γ · ht, ε}, 1)" -- EN: the optimizer performs this computation), and computing a difference value between the initial model parameter value of the current global iteration and the third product to obtain the target model parameter value of the current global iteration (Section 2.2, p. 6, Eq. (6), "θt+1 ← θt − ηt · clip(mt / max{γ · ht, ε}, 1)" -- EN: θt minus the third product denotes the difference value and θt+1 denotes the target model parameter value).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of SlowMo in view of Overlap in view of Chen in view of Barkai to include a coordinate-wise clip of the slow momentum to a preset threshold ρ, applied before the slow learning rate multiplies the momentum, as taught by Liu, in order to bound the size of every outer update in every parameter dimension (Section 2.2, page 6-7).
Motivation would improve the stability of the training because the clipping holds the worst-case size of the update in all parameter dimensions to at most ρ, as taught by Liu where the benefit is disclosed (Section 2.2, page 6-7).
Regarding claim 14, SlowMo in view of Overlap in view of Chen in view of Barkai teaches all the limitations of claims 8, 9, 10, and 11 including “wherein the computer program, when executed by the at least one processor, causes the at least one processor to perform computing, through the computation process, the target model parameter value of the current global iteration according to the target momentum of the current global iteration and the initial model parameter value of the current global iteration”.
Liu teaches:
by:
performing, through the computation process, momentum clipping on the target momentum of the current global iteration (Section 2.2, p. 6, "Let mt be the EMA of gradients, mt ← β1 mt−1 + (1−β1)gt, which is the numerator of the update" -- EN: the exponential moving average mt denotes the momentum; Section 2.2, p. 6, Eq. (6), "θt+1 ← θt − ηt · clip(mt / max{γ · ht, ε}, 1)" -- EN: the clip applied to the momentum-based update denotes the momentum clipping, the claim does not exclude Liu's coordinate-wise scaling of mt before the clip; Abstract, p. 1, "The update is the moving average of the gradients divided by the moving average of the estimated Hessian, followed by element-wise clipping" -- EN: the optimizer performs the element-wise clipping) so that a value of the momentum-clipped target momentum is in a preset interval (Section 2.2, p. 6, "For a clipping threshold ρ > 0, let the clipping function be clip(z, ρ) = max{min{z, ρ}, −ρ} where all operations are applied coordinate-wise" -- EN: the range from −ρ to ρ, with ρ preset and set to 1 in Eq. (6), denotes the preset interval in which each clipped coordinate lies); and
computing, through the computation process, a third product of the momentum-clipped target momentum of the current global iteration and a preset external iteration learning rate (Section 2.2, p. 6, Eq. (6), "θt+1 ← θt − ηt · clip(mt / max{γ · ht, ε}, 1)" -- EN: ηt · clip(mt / max{γ · ht, ε}, 1) denotes the third product, with ηt as the preset learning rate; Algorithm 3, p. 3, "θt+1 = θt − ηt · clip(mt / max{γ · ht, ε}, 1)" -- EN: the optimizer performs this computation), and computing a difference value between the initial model parameter value of the current global iteration and the third product to obtain the target model parameter value of the current global iteration (Section 2.2, p. 6, Eq. (6), "θt+1 ← θt − ηt · clip(mt / max{γ · ht, ε}, 1)" -- EN: θt minus the third product denotes the difference value and θt+1 denotes the target model parameter value).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of SlowMo in view of Overlap in view of Chen in view of Barkai to include a coordinate-wise clip of the slow momentum to a preset threshold ρ, applied before the slow learning rate multiplies the momentum, as taught by Liu, in order to bound the size of every outer update in every parameter dimension (Section 2.2, page 6-7).
Motivation would improve the stability of the training because the clipping holds the worst-case size of the update in all parameter dimensions to at most ρ, as taught by Liu where the benefit is disclosed (Section 2.2, page 6-7).
Regarding claim 20, it is a non-transitory computer-readable storage medium claim of independent claim 15 and recites substantially the same limitations as claim 13, and is therefore rejected under the same rationale as claim 13.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NAYMUR RAHMAN ALI whose telephone number is (571)272-0007. The examiner can normally be reached Mon-Fri. 9:30-6:30 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alexey Shmatov can be reached at (571)270-3428. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NAYMUR RAHMAN ALI/Examiner, Art Unit 2123
/ALEXEY SHMATOV/Supervisory Patent Examiner, Art Unit 2123
Prosecution Insights