DETAILED ACTION
This action is in response to the Applicant Response filed 10 August 2026 for application 18/298,791 filed 11 April 2023.
Claim(s) 21, 23-24, 27-28, 30-31, 36, 38-39 is/are currently amended.
Claim(s) 1-20, 22, 29, 37 is/are cancelled.
Claim(s) 21, 23-28, 30-36, 38-40 is/are pending.
Claim(s) 21, 23-28, 30-36, 38-40 is/are rejected.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments regarding the 35 U.S.C. 101 rejection of the claims are based on the newly amended subject matter. All arguments are addressed in the 35 U.S.C. 101 rejection of the claims below.
Claim Objections
Claim(s) 36, 38-40 is/are objected to because of the following informalities:
Claims 36, line 12, a second momentum configured to should read “a second momentum decompressor configured to”
Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim(s) 21, 23-28, 30-36, 38-40 is/are rejected under 35 U.S.C. 101, because the claim(s) is/are directed to an abstract idea, and because the claim elements, whether considered individually or in combination, do not amount to significantly more than the abstract idea, see Alice Corporation Pty. Ltd. V. CLS Bank International et al., 573 US 208 (2014).
Regarding claim 21, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 21 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method for training a neural network model.
The limitation of computing gradients associated with parameters of a layer of the neural network model, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses calculating gradients.
The limitation of ... decompressing the first set of compressed momentum values to generate a first set of decompressed momentum values, the first set of decompressed momentum values being in a training format that is higher precision than the selected storage format, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses converting a numerical precision.
The limitation of ... decompressing the second set of compressed momentum values to generate a second set of decompressed momentum values in the training format, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses converting a numerical precision.
The limitation of … generating a first set of new momentum values based on the first set of decompressed momentum values and the gradients, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses calculating momentum values.
The limitation of … generating a second set of new momentum values based on the second set of decompressed values and the gradients, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses calculating momentum values.
The limitation of updating parameters of the layer of the neural network model using at least the first set of new momentum values and the second set of new momentum values ..., as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of ... compressing the first set of new momentum values into the selected storage format associated with the memory, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses converting a numerical precision.
The limitation of ... compressing the second set of new momentum values into the selected storage format associated with the memory, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses converting a numerical precision.
The limitation of decompressing and reusing the first set of new momentum values and the second set of new momentum values in a subsequent training iteration of the neural network model, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses repeating mathematical calculations and conversions.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. If a claim limitation, under its broadest reasonable interpretation, covers performance of mathematical concepts, then it falls within the "Mathematical Concepts" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – memory, first momentum decompressor, second momentum decompressor, first compressor, second compressor. The additional element(s) is/are recited at a high-level of generality (i.e., as generic computer components performing generic computer functions of executing instructions on the computers) such that it amounts to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b)).
The claim recites additional element(s) – neural network model, dual-momentum algorithm, dual-momentum gradient optimizer. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
The claim recites retrieving from memory a first set of compressed momentum values associated with a first momentum of a dual-momentum algorithm and a second set of compressed momentum values associated with a second momentum of the dual-momentum algorithm, the first set of compressed momentum values and the second set of compressed momentum values corresponding to a layer of the neural network model and being stored in a selected storage format; storing the first set of new momentum values and the second set of new momentum values in the selected storage format in the memory, which is simply storing and retrieving data recited at a high level of generality. This is nothing more than insignificant extra-solution activity (MPEP 2106.05(g)).
The claim recites a neural network model comprising L layers, wherein L is an integer greater than one; ... wherein the dual-momentum gradient optimizer is configured to operate in a burst mode in which gradients associated with successive layers of the neural network model are streamed through the dual-momentum gradient optimizer and are sequentially used to generate the first and second sets of new momentum values which is simply additional information regarding the model, and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
memory, first momentum decompressor, second momentum decompressor, first compressor, second compressor amount(s) to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b))
storing and retrieving data amount(s) to no more than insignificant extra-solution activity (MPEP 2106.05(g)), wherein the insignificant extra-solution activity is the well-understood routine and conventional activit(y/ies) of storing and retrieving information in memory (MPEP 2016.05(d))
neural network model, dual-momentum algorithm, dual-momentum gradient optimizer amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
additional information regarding the model do(es) not apply the exception in a meaningful way (MPEP 2106.05(e))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 23, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 23 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method for training a neural network model.
The limitation of padding extra bits to form single-precision format decompressed momentum values or double-precision format decompressed momentum values, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses converting a numerical precision.
If a claim limitation, under its broadest reasonable interpretation, covers performance of mathematical concepts, then it falls within the "Mathematical Concepts" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites wherein the training format comprises a single-precision format or a double-precision format which is simply additional information regarding the data precision, and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
additional information regarding the data precision do(es) not apply the exception in a meaningful way (MPEP 2106.05(e))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 24, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 24 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method for training a neural network model.
The limitation of wherein the 8-bit compressed format is converted into the 32-bit precision format by inserting one zero bit between a most significant bit of an exponent and padding twenty-three bits right after a least significant bit of the exponent, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses converting a numerical precision.
If a claim limitation, under its broadest reasonable interpretation, covers performance of mathematical concepts, then it falls within the "Mathematical Concepts" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites wherein the training format comprises a 32-bit precision format and wherein each of the first set of compressed momentum values and the second set of compressed momentum values correspond to an 8-bit compressed format which is simply additional information regarding the data precision, and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
additional information regarding the data precision do(es) not apply the exception in a meaningful way (MPEP 2106.05(e))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 25, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 25 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method for training a neural network model. The Step 2A Prong One Analysis for claim 21 is applicable here since claim 25 carries out the method of claim 21 but for the recitation of additional element(s) of wherein the selected storage format comprises a reduced single-precision format.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the data precision and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the data precision do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 26, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 26 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method for training a neural network model.
The limitation of wherein values corresponding to the reduced single-precision format are formed by removing a most significant bit of each of the first set of new momentum values and by truncating all mantissa bits of each of the first set of new momentum values, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses converting a numerical precision.
The limitation of wherein values corresponding to the reduced single-precision format are formed by removing the most significant bit of each of the second set of new momentum values and by truncating all mantissa bits of each of the second set of new momentum values, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses converting a numerical precision.
If a claim limitation, under its broadest reasonable interpretation, covers performance of mathematical concepts, then it falls within the "Mathematical Concepts" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated
into a practical application. The claim does not recite any additional elements which integrate the
abstract idea into a practical application and, therefore, does not impose any meaningful limits on
practicing the abstract idea. Therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to
significantly more than the judicial exception. As discussed above with respect to the integration of the
abstract idea into a practical application, the claim does not recite any additional elements which
provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 27, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 27 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method for training a neural network model. The Step 2A Prong One Analysis for claim 21 is applicable here since claim 27 carries out the method of claim 21 but for the recitation of additional element(s) of wherein the dual-momentum gradient optimizer is configured to implement an adaptive moment estimation algorithm.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites wherein the dual-momentum gradient optimizer is configured to implement an adaptive moment estimation algorithm which is simply additional information regarding the model, and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)).
The claim recites additional element(s) – adaptive moment estimation algorithm. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
adaptive moment estimation algorithm amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
additional information regarding the model do(es) not apply the exception in a meaningful way (MPEP 2106.05(e))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 28, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 28 is directed to a system with a processor, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) system.
The limitation of ... compute gradients associated with parameters of a layer of the plurality of layers, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses calculating gradients.
The limitation of decompress the first set of compressed momentum values to generate a first set of decompressed momentum values in a training format that is higher precision than the selected storage format, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses converting a numerical precision.
The limitation of decompress the second set of compressed momentum values to generate a second set of decompressed momentum values in the training format, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses converting a numerical precision.
The limitation of generate a first set of new momentum values based on the first set of decompressed momentum values and the gradients, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses calculating momentum values.
The limitation of generate a second set of new momentum values based on the second set of decompressed values and the gradients ..., as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses calculating momentum values.
The limitation of ... compress the first set of new momentum values into the selected storage format, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses converting a numerical precision.
The limitation of ... compress the second set of new momentum values into the selected storage format, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses converting a numerical precision.
The limitation of ... decompress and use the first set of new momentum values and the second set of new momentum values in a subsequent training iteration of the neural network model, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses repeating mathematical calculations and conversions.
If a claim limitation, under its broadest reasonable interpretation, covers performance of mathematical concepts, then it falls within the "Mathematical Concepts" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – system, first momentum decompressor, memory, second momentum decompressor, first compressor, second compressor, storage component. The additional element(s) is/are recited at a high-level of generality (i.e., as generic computer components performing generic computer functions of executing instructions on the computers) such that it amounts to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b)).
The claim recites additional element(s) – neural network model, dual-momentum algorithm, dual-momentum gradient optimizer. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
The claim recites receive from memory a first set of compressed momentum values stored in a selected storage format, the first set of compressed momentum values associated with a first momentum of a dual-momentum algorithm; receive from the memory a second set of compressed momentum values stored in the selected storage format, the second set of compressed momentum values associated with a second momentum of the dual-momentum algorithm different from the first momentum; a storage component that stores the first set of new momentum values and the second set of new momentum values in the selected storage format in the memory ..., which is simply storing and retrieving data recited at a high level of generality. This is nothing more than insignificant extra-solution activity (MPEP 2106.05(g)).
The claim recites a neural network model including a plurality of layers ...; ... wherein the dual-momentum gradient optimizer is configured to operate in a burst mode in which gradients associated with successive layers of the neural network model are streamed through the dual-momentum gradient optimizer and are sequentially used to generate the first and second sets of new momentum values which is simply additional information regarding the model, and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
system, first momentum decompressor, memory, second momentum decompressor, first compressor, second compressor, storage component amount(s) to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b))
storing and retrieving data amount(s) to no more than insignificant extra-solution activity (MPEP 2106.05(g)), wherein the insignificant extra-solution activity is the well-understood routine and conventional activit(y/ies) of storing and retrieving information in memory (MPEP 2016.05(d))
neural network model, dual-momentum algorithm, dual-momentum gradient optimizer amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
additional information regarding the model do(es) not apply the exception in a meaningful way (MPEP 2106.05(e))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 30, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 30 is directed to a system [For the purposes of compact prosecution, the system is interpreted to comprise appropriate structure], which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) system.
The limitation of pad extra bits to form single-precision format decompressed momentum values or double-precision format decompressed momentum values, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses converting a numerical precision.
If a claim limitation, under its broadest reasonable interpretation, covers performance of mathematical concepts, then it falls within the "Mathematical Concepts" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites wherein the training format comprises a single-precision format or a double-precision format which is simply additional information regarding the data precision, and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
additional information regarding the data precision do(es) not apply the exception in a meaningful way (MPEP 2106.05(e))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 31, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 31 is directed to a system [For the purposes of compact prosecution, the system is interpreted to comprise appropriate structure], which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) system.
The limitation of wherein the 8-bit compressed format is converted into the 32-bit precision format by inserting one zero bit between a most significant bit of an exponent and padding twenty-three bits right after a least significant bit of the exponent, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses converting a numerical precision.
If a claim limitation, under its broadest reasonable interpretation, covers performance of mathematical concepts, then it falls within the "Mathematical Concepts" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites wherein the training format comprises a 32-bit precision format, wherein each of the first set of compressed momentum values and the second set of compressed momentum values correspond to an 8-bit compressed format which is simply additional information regarding the data precision, and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
additional information regarding the data precision do(es) not apply the exception in a meaningful way (MPEP 2106.05(e))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 32, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 32 is directed to a system [For the purposes of compact prosecution, the system is interpreted to comprise appropriate structure], which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) system. The Step 2A Prong One Analysis for claim 28 is applicable here since claim 32 carries out the system of claim 28 but for the recitation of additional element(s) of wherein the selected storage format comprises a reduced single-precision format.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the data precision and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the data precision do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 33, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 33 is directed to a system [For the purposes of compact prosecution, the system is interpreted to comprise appropriate structure], which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) system.
The limitation of … forms values corresponding to the reduced single-precision format by removing a most significant bit of each of the first set of new momentum values and by truncating all mantissa bits of each of the first set of new momentum values, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses converting a numerical precision.
The limitation of … forms values corresponding to the reduced single-precision format by removing the most significant bit of each of the second set of new momentum values and by truncating all mantissa bits of each of the second set of new momentum values, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses converting a numerical precision.
If a claim limitation, under its broadest reasonable interpretation, covers performance of mathematical concepts, then it falls within the "Mathematical Concepts" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated
into a practical application. The claim does not recite any additional elements which integrate the
abstract idea into a practical application and, therefore, does not impose any meaningful limits on
practicing the abstract idea. Therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to
significantly more than the judicial exception. As discussed above with respect to the integration of the
abstract idea into a practical application, the claim does not recite any additional elements which
provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 34, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 34 is directed to a system [For the purposes of compact prosecution, the system is interpreted to comprise appropriate structure], which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) system. The Step 2A Prong One Analysis for claim 28 is applicable here since claim 34 carries out the system of claim 28 but for the recitation of additional element(s) of wherein the dual-gradient optimizer is configured to implement an adaptive moment estimation algorithm.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites wherein the dual-gradient optimizer is configured to implement an adaptive moment estimation algorithm which is simply additional information regarding the model, and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)).
The claim recites additional element(s) – adaptive moment estimation algorithm. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
adaptive moment estimation algorithm amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
additional information regarding the model do(es) not apply the exception in a meaningful way (MPEP 2106.05(e))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 35, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 35 is directed to a system [For the purposes of compact prosecution, the system is interpreted to comprise appropriate structure], which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) system. The Step 2A Prong One Analysis for claim 28 is applicable here since claim 35 carries out the system of claim 28 but for the recitation of additional element(s) of wherein the dual-gradient gradient optimizer is implemented using a field programmable gate array (FPGA).
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites wherein the dual-gradient gradient optimizer is implemented using a field programmable gate array (FPGA) which is simply additional information regarding the model, and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)).
The claim recites additional element(s) – field programmable gate array (FPGA). The additional element(s) is/are recited at a high-level of generality (i.e., as generic computer components performing generic computer functions of executing instructions on the computers) such that it amounts to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
field programmable gate array (FPGA) amount(s) to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b))
additional information regarding the model do(es) not apply the exception in a meaningful way (MPEP 2106.05(e))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 36, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 36 is directed to a system with a processor, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) system for training a neural network model.
The limitation of ... compute gradients associated with parameters of the neural network model during training, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses calculating gradients.
The limitation of decompress the first set of compressed momentum values to generate a first set of decompressed momentum values, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses converting a numerical precision.
The limitation of decompress the second set of compressed momentum values to generate a second set of decompressed momentum values, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses converting a numerical precision.
The limitation of generate a first set of new momentum values based on the first set of decompressed momentum values and the gradients, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses calculating momentum values.
The limitation of generate a second set of new momentum values based on the second set of decompressed values and the gradients..., as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses calculating momentum values.
The limitation of ... updates parameters of the neural network model using the first set of new momentum values and the second set of new momentum values, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of ... compress the first set of new momentum values into a selected storage format associated with the memory, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses converting a numerical precision.
The limitation of ... compress the second set of new momentum values into the selected storage format associated with the memory, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses converting a numerical precision.
The limitation of decompressing and using the first set of new momentum values and the second set of new momentum values during a subsequent training iteration of the neural network model, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses repeating mathematical calculations and conversions.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. If a claim limitation, under its broadest reasonable interpretation, covers performance of mathematical concepts, then it falls within the "Mathematical Concepts" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – system, first momentum decompressor, memory, second momentum [decompressor], first compressor, second compressor. The additional element(s) is/are recited at a high-level of generality (i.e., as generic computer components performing generic computer functions of executing instructions on the computers) such that it amounts to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b)).
The claim recites additional element(s) – neural network model, dual-momentum adaptive moment estimation algorithm, dual-momentum gradient optimizer. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
The claim recites receive from memory a first set of compressed momentum values associated with a first momentum of a dual-momentum adaptive moment estimation algorithm; receive from the memory a second set of compressed momentum values associated with a second momentum of the dual-momentum adaptive moment estimation algorithm different from the first momentum, which is simply retrieving data recited at a high level of generality. This is nothing more than insignificant extra-solution activity (MPEP 2106.05(g)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
system, first momentum decompressor, memory, second momentum [decompressor], first compressor, second compressor amount(s) to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b))
retrieving data amount(s) to no more than insignificant extra-solution activity (MPEP 2106.05(g)), wherein the insignificant extra-solution activity is the well-understood routine and conventional activit(y/ies) of storing and retrieving information in memory (MPEP 2016.05(d))
neural network model, dual-momentum adaptive moment estimation algorithm, dual-momentum gradient optimizer amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 38, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 38 is directed to a system with a processor, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) system for training a neural network model.
The limitation of pad extra bits to form single-precision format decompressed momentum values or double-precision format decompressed momentum values, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses converting a numerical precision.
If a claim limitation, under its broadest reasonable interpretation, covers performance of mathematical concepts, then it falls within the "Mathematical Concepts" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites wherein the training format comprises a single-precision format or a double-precision format which is simply additional information regarding the data precision, and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
additional information regarding the data precision do(es) not apply the exception in a meaningful way (MPEP 2106.05(e))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 39, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 39 is directed to a system with a processor, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) system for training a neural network model. The Step 2A Prong One Analysis for claim 36 is applicable here since claim 39 carries out the system of claim 36 but for the recitation of additional element(s) of wherein the selected storage format comprises a reduced single-precision format.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the data precision and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the data precision do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 40, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 40 is directed to a system with a processor, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) system for training a neural network model. The Step 2A Prong One Analysis for claim 39 is applicable here since claim 40 carries out the system of claim 39 but for the recitation of additional element(s) of wherein the dual-gradient optimizer is configured to implement an adaptive moment estimation algorithm.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites wherein the dual-gradient optimizer is configured to implement an adaptive moment estimation algorithm which is simply additional information regarding the model, and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)).
The claim recites additional element(s) – adaptive moment estimation algorithm. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
adaptive moment estimation algorithm amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
additional information regarding the model do(es) not apply the exception in a meaningful way (MPEP 2106.05(e))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communication from the examiner should be directed to MARSHALL WERNER whose telephone number is (469) 295-9143. The examiner can normally be reached on Monday – Thursday 7:30 AM – 4:30 PM ET.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kamran Afshar, can be reached at (571) 272-7796. The fax number for the organization where this application or proceeding is assigned is (571) 273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MARSHALL L WERNER/ Primary Examiner, Art Unit 2125
