DETAILED ACTION
This action is responsive to the Application filed on 03/07/2023. Claims 1-20 are pending in the case. Claims 1, 9, and 16 are independent claims.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55. The date used in determinations of prior art is 11/30/2022.
Drawings
The drawings are objected to because:
In Fig. 1, Ref. No. 142 is defined as the “host physical machine set” in the specification, but is labeled as the “hot physical machine set” in the drawing. The current label should be corrected in accordance with the specification.
In Fig. 2, Ref. No. 103 is labeled “system” despite being defined as the “end user device” in the specification. A correct label should be provided to the element in accordance with its intended title from the specification. See the objection to the specification below.
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specification
The disclosure is objected to because of the following informalities:
In paragraph [0018], Ref. No. 103 is defined as the “end user device” in relation to its appearance in Fig. 1, but there is no description of Ref. No. 103 in relation to its appearance in Fig. 2. If these elements are meant to represent the same thing, then this should be mentioned in paragraph [0036] with the introduction of the elements of Fig. 2. Additionally, the label for Ref. No. 103 in Fig. 2 should be corrected in accordance with its intended title.
Appropriate correction is required.
Claim Objections
Claims 1-20 objected to because of the following informalities:
In claims 1, 9, and 16 the “most computationally expensive operation” is referred to as the “most expensive operation.” Although these elements are understood to be the same thing, the examiner requires that all references to this element are amended to include the word “computationally” so as to prevent misconstruction of the limitations.
In claims 8 and 15, the “most computationally expensive operation” is again referred to as the “most expensive operation.” Although these elements are understood to be the same thing, the examiner requires that all references to this element are amended to include the word “computationally” so as to prevent misconstruction of the limitations.
Claims 2-8, 10-15, and 17-20 inherit the objections of the claims upon which they depend
Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed towards an abstract idea with significantly more.
Step 1: Claims 1-8 are directed towards a method, claims 9-15 are directed towards a machine, and claims 16-20 are directed towards an article of manufacture. Therefore, claims 1-20 are rejected are directed towards one of 4 statutory categories; process, machine, manufacture, or composition of matter.
With respect to claim 1:
Step 2A Prong 1: The claim is directed to a judicial exception.
computing a solution to the system of linear equations by a flexible iterative algorithm, wherein the computing includes, for each iteration… (Mental Process and Mathematical Concept: One could compute the solution to the system of linear equations by applying mathematical concepts and following a flexible iterative algorithm, mentally or using pen and paper)
determining a most computationally expensive operation of the iteration (Mental Process and Mathematical Concept: One could determine a most computationally expensive operation in computing the solution to the system of linear equations using mathematical concepts, mentally or using pen and paper)
mapping the most expensive operation to a low precision format, and performing the most expensive operation according to a low precision (Mental Process and Mathematical Concept: One could map the most expensive operation to a low precision format and perform the operation, mathematically, according to that precision, mentally or using pen and paper)
performing other operations of the iteration according to a high precision (Mental Process and Mathematical Concept: One could perform the other operations for computing a solution to a system of linear equations using mathematical concepts, according to a high precision, mentally or using pen and paper)
Step 2A Prong 2: The judicial exceptions as a whole are not integrated into a practical
application.
Additional Elements:
receiving, by a requesting device, a system of linear equations (Amounts to necessary data gathering. Insignificant extra-solution activity, as discussed in MPEP § 2106.05(g))
returning the solution to the requesting device (Amounts to necessary data output. Insignificant extra-solution activity, as discussed in MPEP § 2106.05(g))
Step 2B: The claim does not include additional elements that amount to significantly more than the judicial exception.
Re-evaluation of Insignificant Extra-Solution Activities:
receiving, by a requesting device, a system of linear equations (“Receiving or transmitting data over a network” is a well-understood, routine, conventional activity when claimed in a merely generic manner (as it is in the present claim), as discussed in MPEP § 2106.05(d)(II))
returning the solution to the requesting device (“Receiving or transmitting data over a network” is a well-understood, routine, conventional activity when claimed in a merely generic manner (as it is in the present claim), as discussed in MPEP § 2106.05(d)(II))
With respect to claim 2:
Step 2A Prong 1: The claim is directed to a judicial exception, including those inherited from claim 1 via dependency.
Step 2A Prong 2: The judicial exceptions as a whole are not integrated into a practical
application.
Additional Elements:
The method of claim 1 (See above)
wherein the solution is returned in a high precision format (Amounts to necessary data output, regardless of the format of the solution returned. Insignificant extra-solution activity, as discussed in MPEP § 2106.05(g))
Step 2B: The claim does not include additional elements that amount to significantly more than the judicial exception.
Re-evaluation of Insignificant Extra-Solution Activities:
wherein the solution is returned in a high precision format (“Receiving or transmitting data over a network” is a well-understood, routine, conventional activity when claimed in a merely generic manner (as it is in the present claim), as discussed in MPEP § 2106.05(d)(II))
Additional Elements:
The method of claim 1 (See above)
With respect to claim 3:
Step 2A Prong 1: The claim is directed to a judicial exception, including those inherited from claim 1 via dependency.
wherein the most computationally expensive operation is a preconditioning operation (Mental Process and Mathematical Concept: One could determine the most computationally expensive operation is the mathematical concept of preconditioning, mentally or using pen and paper)
Step 2A Prong 2: The judicial exceptions as a whole are not integrated into a practical
application.
Additional Elements:
The method of claim 1 (See above)
Step 2B: The claim does not include additional elements that amount to significantly more than the judicial exception.
Additional Elements:
The method of claim 1 (See above)
With respect to claim 4:
Step 2A Prong 1: The claim is directed to a judicial exception, including those inherited from claim 3 via dependency.
where the flexible iterative algorithm is configured to apply the preconditioning operation using one or more preconditioners having one or more variable parameters (Mental Process and Mathematical Concept: One could solve the system of equations using a flexible iterative algorithm that includes the mathematical concept of a preconditioning operation and preconditioners with variable parameters, mentally or using pen and paper)
Step 2A Prong 2: The judicial exceptions as a whole are not integrated into a practical
application.
Additional Elements:
The method of claim 3 (See above)
Step 2B: The claim does not include additional elements that amount to significantly more than the judicial exception.
Additional Elements:
The method of claim 3 (See above)
With respect to claim 5:
Step 2A Prong 1: The claim is directed to a judicial exception, including those inherited from claim 4 via dependency.
wherein the flexible iterative algorithm is a flexible Generalized Minimal Residual (GMRES) algorithm (Mathematical Concept: Naming the type of iterative algorithm used merely limits the algorithm to a certain set of mathematical processes which, as above, one could perform mentally or using pen and paper)
Step 2A Prong 2: The judicial exceptions as a whole are not integrated into a practical
application.
Additional Elements:
The method of claim 4 (See above)
Step 2B: The claim does not include additional elements that amount to significantly more than the judicial exception.
Additional Elements:
The method of claim 4 (See above)
With respect to claim 6:
Step 2A Prong 1: The claim is directed to a judicial exception, including those inherited from claim 1 via dependency.
wherein the low precision is selected from at least one of a 4-bit precision and an 8-bit precision (Mental Process: One could select the type of low precision used from 4-bit and 8-bit precision. The mapping and performing of an operation according to the selected low precision could still be done mentally, as above)
Step 2A Prong 2: The judicial exceptions as a whole are not integrated into a practical
application.
Additional Elements:
The method of claim 1 (See above)
Step 2B: The claim does not include additional elements that amount to significantly more than the judicial exception.
Additional Elements:
The method of claim 1 (See above)
With respect to claim 7:
Step 2A Prong 1: The claim is directed to a judicial exception, including those inherited from claim 1 via dependency.
wherein the high precision is selected from at least one of a 16-bit precision, a 32-bit precision and a 64-bit precision (Mental Process: One could select the type of high precision used from 32-bit and 64-bit precision. The performing of operations according to the selected high precision could still be done mentally, as above)
Step 2A Prong 2: The judicial exceptions as a whole are not integrated into a practical
application.
Additional Elements:
The method of claim 1 (See above)
Step 2B: The claim does not include additional elements that amount to significantly more than the judicial exception.
Additional Elements:
The method of claim 1 (See above)
With respect to claim 8:
Step 2A Prong 1: The claim is directed to a judicial exception, including those inherited from claim 1 via dependency.
the most expensive operation is performed… (Mental Process: One could perform the most expensive operation in computing a solution to the system of linear equations, mentally or using pen and paper)
the other operations are performed… (Mental Process: One could perform the rest of the operations in computing a solution to the system of linear equations, mentally or using pen and paper)
Step 2A Prong 2: The judicial exceptions as a whole are not integrated into a practical
application.
Additional Elements:
The method of claim 1 (See above)
…using a low precision hardware device (Performing the most expensive operation on a low precision hardware device merely indicates the technological environment in which to apply a judicial exception, as discussed in MPEP § 2106.05(h))
…using a high precision hardware device (Performing the rest of the operations on a high precision hardware device merely indicates the technological environment in which to apply a judicial exception, as discussed in MPEP § 2106.05(h))
Step 2B: The claim does not include additional elements that amount to significantly more than the judicial exception.
Additional Elements:
The method of claim 1 (See above)
…using a low precision hardware device (Performing the most expensive operation on a low precision hardware device merely indicates the technological environment in which to apply a judicial exception, as discussed in MPEP § 2106.05(h))
…using a high precision hardware device (Performing the rest of the operations on a high precision hardware device merely indicates the technological environment in which to apply a judicial exception, as discussed in MPEP § 2106.05(h))
With respect to claim 9:
Step 2A Prong 1: The claim is directed to a judicial exception.
computing a solution to the system of linear equations by a flexible iterative algorithm, wherein the computing includes, for each iteration… (Mental Process and Mathematical Concept: One could compute the solution to the system of linear equations by applying mathematical concepts and following a flexible iterative algorithm, mentally or using pen and paper)
determining a most computationally expensive operation of the iteration (Mental Process and Mathematical Concept: One could determine a most computationally expensive operation in computing the solution to the system of linear equations using mathematical concepts, mentally or using pen and paper)
mapping the most expensive operation to a low precision format, and performing the most expensive operation according to a low precision (Mental Process and Mathematical Concept: One could map the most expensive operation to a low precision format and perform the operation, mathematically, according to that precision, mentally or using pen and paper)
performing other operations of the iteration according to a high precision (Mental Process and Mathematical Concept: One could perform the other operations for computing a solution to a system of linear equations using mathematical concepts, according to a high precision, mentally or using pen and paper)
Step 2A Prong 2: The judicial exceptions as a whole are not integrated into a practical
application.
Additional Elements:
A system comprising: a memory device; and one or more processing units coupled with the memory device, the one or more processing units are configured to perform a method of computation, the method comprising… (Adding generic computer components to perform the method is not sufficient. Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f))
receiving, by a requesting device, a system of linear equations (Amounts to necessary data gathering. Insignificant extra-solution activity, as discussed in MPEP § 2106.05(g))
returning the solution to the requesting device (Amounts to necessary data output. Insignificant extra-solution activity, as discussed in MPEP § 2106.05(g))
Step 2B: The claim does not include additional elements that amount to significantly more than the judicial exception.
Re-evaluation of Insignificant Extra-Solution Activities:
receiving, by a requesting device, a system of linear equations (“Receiving or transmitting data over a network” is a well-understood, routine, conventional activity when claimed in a merely generic manner (as it is in the present claim), as discussed in MPEP § 2106.05(d)(II))
returning the solution to the requesting device (“Receiving or transmitting data over a network” is a well-understood, routine, conventional activity when claimed in a merely generic manner (as it is in the present claim), as discussed in MPEP § 2106.05(d)(II))
Additional Elements:
A system comprising: a memory device; and one or more processing units coupled with the memory device, the one or more processing units are configured to perform a method of computation, the method comprising… (Adding generic computer components to perform the method is not sufficient. Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f))
With respect to claims 10-15:
See the rejections for claims 2-8 above. Note that the only difference between claims 10-15 and claims 2-8 is that claims 10-15 are directed to the machine that performs the method, introduced in claim 9, whereas claims 2-8 are directed towards the method, introduced in claim 1. See the rejection for claim 9 above which incorporates the system and its components into the rejection. Additionally, note that claim 14 incorporates the limitations of claims 6 and 7 into a single claim; in understanding the rejection for claim 14, see the rejections for both claims 6 and 7.
With respect to claim 16:
Step 2A Prong 1: The claim is directed to a judicial exception.
computing a solution to the system of linear equations by a flexible iterative algorithm, wherein the computing includes, for each iteration… (Mental Process and Mathematical Concept: One could compute the solution to the system of linear equations by applying mathematical concepts and following a flexible iterative algorithm, mentally or using pen and paper)
determining a most computationally expensive operation of the iteration (Mental Process and Mathematical Concept: One could determine a most computationally expensive operation in computing the solution to the system of linear equations using mathematical concepts, mentally or using pen and paper)
mapping the most expensive operation to a low precision format, and performing the most expensive operation according to a low precision (Mental Process and Mathematical Concept: One could map the most expensive operation to a low precision format and perform the operation, mathematically, according to that precision, mentally or using pen and paper)
performing other operations of the iteration according to a high precision (Mental Process and Mathematical Concept: One could perform the other operations for computing a solution to a system of linear equations using mathematical concepts, according to a high precision, mentally or using pen and paper)
Step 2A Prong 2: The judicial exceptions as a whole are not integrated into a practical
application.
Additional Elements:
A computer program product comprising a computer-readable memory that has computer-executable instructions stored thereupon, the computer-executable instructions when executed by a processor cause the processor to perform operations comprising… (Adding generic computer components to perform the method is not sufficient; a computer-readable memory is a generic computer component. Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f))
receiving, by a requesting device, a system of linear equations (Amounts to necessary data gathering. Insignificant extra-solution activity, as discussed in MPEP § 2106.05(g))
returning the solution to the requesting device (Amounts to necessary data output. Insignificant extra-solution activity, as discussed in MPEP § 2106.05(g))
Step 2B: The claim does not include additional elements that amount to significantly more than the judicial exception.
Re-evaluation of Insignificant Extra-Solution Activities:
receiving, by a requesting device, a system of linear equations (“Receiving or transmitting data over a network” is a well-understood, routine, conventional activity when claimed in a merely generic manner (as it is in the present claim), as discussed in MPEP § 2106.05(d)(II))
returning the solution to the requesting device (“Receiving or transmitting data over a network” is a well-understood, routine, conventional activity when claimed in a merely generic manner (as it is in the present claim), as discussed in MPEP § 2106.05(d)(II))
Additional Elements:
A computer program product comprising a computer-readable memory that has computer-executable instructions stored thereupon, the computer-executable instructions when executed by a processor cause the processor to perform operations comprising… (Adding generic computer components to perform the method is not sufficient; a computer-readable memory is a generic computer component. Adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f))
With respect to claims 17-20:
See the rejections for claims 2-7 above. Note that the only difference between claims 17-20 and claims 2-7 is that claims 17-20 are directed to the article of manufacture holding instructions for the method, introduced in claim 16, whereas claims 2-7 are directed towards the method, introduced in claim 1. See the rejection for claim 16 above which incorporates the computer-readable medium holding instructions into the rejection. Additionally, note that claim 18 incorporates the limitations of claims 3 and 4 into a single claim; in understanding the rejection for claim 18, see the rejections for both claims 3 and 4. Further, note that claim 20 incorporates the limitations of claims 6 and 7 into a single claim; in understanding the rejection for claim 20, see the rejections for both claims 6 and 7.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Castonguay et. al. (US 20140046993 A1) in view of Darvish Rouhani et. al. (US 20200193274 A1), further in view of Kniazev et. al. (US 20170123388 A1).
Regarding claim 1, Castonguay teaches a method (Fig. 2) of computation, comprising:
receiving… a system of linear equations (Fig. 2, Ref. No. 202; Paragraph [0042], “At block 202, data corresponding to a system of equations is accessed.” Note that receiving and accessing are understood to be synonymous actions for getting data which is, in this case, a system of equations. Further, note that the system of equations accessed in at ref. no. 202 can be a linear system, see the last line of paragraph [0042], “for the linear system of equations Ax=b”)
computing a solution to the system of linear equations by a… iterative algorithm (Paragraph [0077], “System 600 depicts components of a preconditioning module of an iterative solver for solving systems of equations… System 600 may be used with or as part of a variety of iterative solving methods including, but not limited to, Krylov subspace methods (e.g., Conjugate Gradient (CG), Bi-Conjugate Gradient (BiCG), Generalized Minimal Residual Method (GMRES)) and algebraic multigrid (AMG) methods.” Note that the solving methods mentioned are commonly understood to be both methods and algorithms. Further, see above, which denotes the system of equations can be linear), wherein the computing includes, for each iteration… and
…returning the solution (Fig. 2, Ref. No. 212; Paragraph [0047], “At block 212, a solution is output.” Note that returning the solution and outputting it are understood to be the same action)
Castonguay does not distinctly disclose:
receiving, by a requesting device…, determining a most computationally expensive operation of the iteration, mapping the most expensive operation to a low precision format, and performing the most expensive operation according to a low precision, performing other operations of the iteration according to a high precision, or …the requesting device
However, Darvish Rouhani teaches:
receiving, by a requesting device, a system… (Fig. 8, Ref. No. 810; Paragraph [0095], “At process block 810, an input tensor of a layer of a multi-layer neural network can be received… from an input/output interface of the neural network accelerator, an on-chip memory, an off-chip memory, or other storage location.” Note that with no definition of requesting device, it is understood to comprise any component in the method capable of holding and sending the system, thereby any of the options listed comprise a requesting device. Further, note that any tensor could potentially hold a system of something, including equations, the reference does not exclude this from the possible forms of the tensor)
determining a most computationally expensive operation of the iteration (Paragraph, [0091], “more computationally expensive operations such as vector-vector, vector-matrix, matrix-matrix, and convolution operations.” Note that the most computationally expensive operation is understood to be included in the more computationally expensive group of operations listed, given the whole of the reference disclosure. Further, see that listing the computationally expensive operations means that they were determined.)
mapping the most expensive operation to a low precision format… (Fig. 8, Ref. No. 820; Paragraph [0096], “At process block 820, the input tensor of the layer can be converted from a normal-precision floating-point format to a quantized-precision floating-point format.” Note that the input tensor used by the tensor operation, seen below, is understood to be mapped to a low precision format which, in this case, is a quantized-precision floating-point format, and therefore maps the operation to a low precision format. See paragraph [0035], which discusses the references definition of quantized, comprising low precision. See below for how the tensor input/operation is the most expensive operation),
and performing the most expensive operation according to a low precision (Fig. 8, Ref. No. 830; Paragraph [0098], “At process block 830, a tensor operation can be performed using the quantized-precision floating-point format of the converted input tensor as an input. For example, the tensor operation can be a vector-vector, vector-matrix, matrix-matrix, or convolution operation.” Note that as in the citation for paragraph [0091], the determined most expensive operations included those listed as potential tensor operations, making the tensor operation a most computationally expensive operation.);
performing other operations of the iteration according to a high precision (Fig. 8, Ref. No. 850; Paragraph [0101], “At optional process block 850, an operation can be performed using the converted result in the normal-precision floating-point format. For example, a scalar add (such as adding a bias value) or an activation function can be computed using the converted result in the normal-precision floating-point format.” Note that the other operations include, in this case, operations like scalar add or activation functions, which are performed according to a normal-precision floating point format. Given the specification, this normal-precision floating point format is considered a high precision format; See paragraph [0033], which explains the references definition of normal-precision, “Examples of normal-precision floating-point formats include, but are not limited to, IEEE 754 standard formats such as 16-bit, 32-bit, 64-bit, ” which are included in the applicant’s implicit definition of high precision.)
…to the requesting device (Paragraph [0095], “an input/output interface of the neural network accelerator, an on-chip memory, an off-chip memory, or other storage location.” Note that, as above, the requesting device can include any of the options listed in the citation above.)
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the method for computing a solution to a system of linear equations (including receiving the system of equations, computing the solution using an iterative algorithm, and returning the solution) as taught in Castonguay with the technique for determining and performing a most computationally expensive operation in a low precision format and performing other operations in a high precision format (including use of a requesting device for receiving a system of equations and returning the solution) as taught by Darvish Rouhani in order to reduce the computational complexity of expensive operations in the iterative algorithm and maintain or increase the accuracy of the determined solution, thereby improving the computational speed of the algorithm. (Darvish Rouhani, Paragraph [0062], “a mixed precision implementation can potentially increase an accuracy of some calculations while also providing the benefits of reduced complexity associated with quantized floating-point.”)
Further, the combination of Castonguay and Darvish Rouhani does not appear to distinctly disclose:
using a flexible iterative algorithm in the computation of a solution to the system of equations
However, Kniazev teaches:
computing a solution to the system of linear equations… (Paragraph [0025], “using an iterative method with a factorization of the preconditioner matrix to produce a solution vector of a system of equations.” Note that solution and solution vector are understood to be the same thing)
…by a flexible iterative algorithm (Paragraph [0089], an “alternative embodiment uses the recalculated or updated preconditioner in an iterative solver without the restart, leading to a variable preconditioner, i.e., the preconditioner that may change on every iteration. Thus, embodiments are modified to use flexible iterative methods that allow variable preconditioning, such as flexible generalized minimum residual.” Note that, as above, the solving method cited is understood to be method and/or algorithm)
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to modify the method for solving a system of equations as taught by Castonguay as modified by the technique for using mixed precision computation as taught by Darvish Rouhani to include the use of a flexible iterative algorithm as taught by Kniazev in order to simplify the computation of the iterative algorithm by not requiring the algorithm to restart when a preconditioner is updated. (Kniazev, Paragraph [0088], “Some embodiments stop the iterative method to update the preconditioner. For example, the iterative method can be stopped at some iterative step, and then restarted using the current iterative approximation to the solution as an initial approximation for the restarted method, with a newly recalculated or updated preconditioner” and Paragraph [0089], “Alternative embodiment uses the recalculated or updated preconditioner in an iterative solver without the restart, leading to a variable preconditioner… thus… [using] flexible iterative methods”)
Regarding claim 2, Castonguay as modified by Darvish Rouhani and further modified by Kniazev teaches all of the limitations of the method in claim 1 as cited above and Darvish Rouhani further teaches the limitation:
wherein the solution is returned in a high precision format (Paragraph [0109], “the output tensor is in normal-precision floating-point format.” Note that as combined above, the output/output tensor of Darvish Rouhani could be understood to represent the solution to a system of linear equations of Castonguay; the limitation here covers outputting a determined result in a high precision format, which could be accomplished for a solution to a system of linear equations via the combination of Castonguay and the techniques of Darvish Rouhani. Further, see the citation from paragraph [0033], which interprets normal-precision floating-point format as constituting a high precision format)
See the rationale for combining Castonguay and Darvish Rouhani in the rejection for claim 1 above.
Regarding claim 3, Castonguay as modified by Darvish Rouhani and further modified by Kniazev teaches all of the limitations of the method in claim 1 as cited above and Kniazev further teaches the limitation:
wherein the most computationally expensive operation is a preconditioning operation (Paragraph [0017], “The preconditioned iterative method requires matrix-vector multiplications.” Note that as combined above, the most computationally expensive operations or tensor operations in Darvish Rouhani, comprise vector-vector, vector-matrix, matrix-matrix, or convolution operations; thereby the preconditioned iterative method which comprises the preconditioning operation is a most computationally expensive operation and vice versa)
See the rationale for combining Castonguay and Darvish Rouhani with Kniazev in the rejection for claim 1 above. Note that an embodiment of the flexible iterative algorithm of Kniazev includes the preconditioned iterative method and, therefore, has the same rationale to combine.
Regarding claim 4, Castonguay as modified by Darvish Rouhani and further modified by Kniazev teaches all of the limitations of the method in claim 3 as cited above and Kniazev further teaches the limitation:
where the flexible iterative algorithm is configured to apply the preconditioning operation… (Paragraph [0089], “Thus, embodiments are modified to use flexible iterative methods that allow variable preconditioning.” Note that the flexible iterative methods are understood to be flexible iterative algorithms, as above. Further, note that these methods “allowing” variable preconditioning is understood to mean that they apply the preconditioning operation which, in this case, is variable)
…using one or more preconditioners having one or more variable parameters (Paragraph [0089], an “alternative embodiment uses the recalculated or updated preconditioner in an iterative solver without the restart, leading to a variable preconditioner, i.e., the preconditioner that may change on every iteration.” Note that with no explicit definition of variable parameters in the specification, they can be understood to comprise any sort of variable that alters the preconditioner; the mention of the preconditioner that may change on every iteration (variable preconditioner) is understood to represent a preconditioner with variable parameters. See paragraph [0112] which discusses one potential modification to the preconditioner to reduce the condition number)
See the rationale for combining Castonguay and Darvish Rouhani with Kniazev in the rejection for claim 1 above. Note that an embodiment of the flexible iterative algorithm of Kniazev includes the preconditioning operation with a variable preconditioner and, therefore, has the same rationale to combine.
Regarding claim 5, Castonguay as modified by Darvish Rouhani and further modified by Kniazev teaches all of the limitations of the method in claim 4 as cited above and Kniazev further teaches the limitation:
wherein the flexible iterative algorithm is a flexible Generalized Minimal Residual (GMRES) algorithm (Paragraph [0089], “Thus, embodiments are modified to use flexible iterative methods that allow variable preconditioning, such as flexible generalized minimum residual.” Note that, as above, the iterative methods mentioned are commonly understood as algorithms or methods. Also, note that flexible generalized minimal residual is understood to be the same thing as flexible generalized minimum residual.)
See the rationale for combining Castonguay and Darvish Rouhani with Kniazev in the rejection for claim 1 above.
Regarding claim 6, Castonguay as modified by Darvish Rouhani and further modified by Kniazev teaches all of the limitations of the method in claim 1 as cited above and Darvish Rouhani further teaches the limitation:
wherein the low precision is selected from at least one of a 4-bit precision and an 8-bit precision (Paragraph [0023], “Numbers represented in normal-precision floating-point format (e.g., a floating-point number expresses in a 16-bit floating-point format, a 32-bit floating-point format, a 64-bit floating-point format, or an 80-bit floating-point format) can be converted to quantized-precision format numbers… Examples of lower-precision quantized formats include formats having a reduced bit width (including by reducing the number of bits used to represent a number's mantissa...” Note that, as above, quantized-precision format is understood as a low precision format. Additionally, note that the reference exemplifies quantized-precision format (a.k.a. low precision format) as having reduced bit width from the normal-precision format (a.k.a. high precision format) which includes 16-, 32-, and 64-bit formats, thereby quantized-precision format includes any bit widths lower than those mentioned, including 4-bit and 8-bit formats. In any embodiment of the reference, 4-bit and/or 8-bit precision could be “selected” as the low precision format.)
See the rationale for combining Castonguay and Darvish Rouhani in the rejection for claim 1 above. Note that the rationale for incorporating the mixed-precision technique of Darvish Rouhani with the method for solving a system of equations in Castonguay covers what Darvish Rouhani classifies as mixed-precision format which further includes what they classify as high and low precision.
Regarding claim 7, Castonguay as modified by Darvish Rouhani and further modified by Kniazev teaches all of the limitations of the method in claim 1 as cited above and Darvish Rouhani further teaches the limitation:
wherein the high precision is selected from at least one of a 16-bit precision, a 32-bit precision and a 64-bit precision (Paragraph [0033], “Examples of normal-precision floating-point formats include, but are not limited to, IEEE 754 standard formats such as 16-bit, 32-bit, 64-bit.” Note that in any embodiment of the reference, the normal-precision floating-point format (understood to be a high precision format, as above) could be “selected” from 16-bit, 32-bit, and 64-bit precision formats)
See the rationale for combining Castonguay and Darvish Rouhani in the rejection for claim 1 above. Note that the rationale for incorporating the mixed-precision technique of Darvish Rouhani with the method for solving a system of equations in Castonguay covers what Darvish Rouhani classifies as mixed-precision format which further includes what they classify as high and low precision.
Regarding claim 8, Castonguay as modified by Darvish Rouhani and further modified by Kniazev teaches all of the limitations of the method in claim 1 as cited above and Kniazev further teaches the limitation:
wherein the most expensive operation is performed using a low precision hardware device, and the other operations are performed using a high precision hardware device (Paragraph [0079], “having a conventional computer processing unit (CPU) with double arithmetic precision and a GPU with a single arithmetic precision, the calculation of the control can be performed on the CPU in double precision, while the preconditioner update or setup 450 can be performed on the GPU in single precision.” Note that, as interpreted above, the most expensive operation is the preconditioner operation because it comprises a matrix-vector operation (See the citation for Darvish Rouhani above which defines the most computationally expensive operations as including matrix-vector operations). This operation is performed on a GPU with single arithmetic precision; the examiner comes to interpret a single arithmetic precision GPU as a low precision hardware device. Therefore, the most expensive operation, a preconditioner update, is performed on a low precision hardware device. Further, the examiner comes to interpret the “calculation of the control” as an “other” operation. This operation is performed on a CPU with double arithmetic precision; the examiner comes to interpret a double arithmetic precision CPU as a high precision hardware device. Therefore, the other operations, e.g. calculation of a control, are performed using a high precision hardware device.)
Although the rationale for combining Castonguay and Darvish Rouhani with Kniazev in the rejection for claim 1 does not cover this additional feature, it can be further seen that, before the effective filing date of the claimed invention it would have been obvious to one of ordinary skill in the art to modify the method for solving a system of equations as taught by Castonguay as modified by the technique for using mixed precision computation as taught by Darvish Rouhani to include the use of different precision hardware devices for different operations as taught by Kniazev in order to allow the combined methods and techniques to be performed in a mixed-type computing environment, allowing for a wider range of potential hardware deployments of the described process. (Kniazev, Paragraph [0079], “For example, in a mixed-type computing environment…”)
Regarding claim 9, see the rejection for claim 1 above. Note that the only difference between claim 9 and claim 1 is that claim 9 is directed towards a machine that performs the method whereas claim 1 is directed towards the method. Further, see Castonguay, Paragraph [0020], “In general, computer system 100 comprises at least one CPU 101, a system memory 115, and at least one graphics processor unit (GPU) 110,” which covers the embodiment of the claimed invention as a system comprising a memory device (Ref. No. 115) and one or more processing units (Ref. No. 101 or 110) coupled to the memory device (Paragraph [0020], “The CPU 101 can be coupled to the system memory 115”… “The GPU(s) 110 is coupled to the CPU 101 and the system memory 115”) configured to perform the method of computation.
Regarding claim 10, see the rejection for claim 2 above. Note that the only difference between claim 10 and claim 2 is that claim 10 is directed towards a machine that performs the method whereas claim 2 is directed towards the method. See the rejection for claim 9 above which covers the alternate embodiment.
Regarding claim 11, see the rejection for claim 3 above. Note that the only difference between claim 11 and claim 3 is that claim 11 is directed towards a machine that performs the method whereas claim 3 is directed towards the method. See the rejection for claim 9 above which covers the alternate embodiment.
Regarding claim 12, see the rejection for claim 4 above. Note that the only difference between claim 12 and claim 4 is that claim 12 is directed towards a machine that performs the method whereas claim 4 is directed towards the method. See the rejection for claim 9 above which covers the alternate embodiment.
Regarding claim 13, see the rejection for claim 5 above. Note that the only difference between claim 13 and claim 5 is that claim 13 is directed towards a machine that performs the method whereas claim 5 is directed towards the method. See the rejection for claim 9 above which covers the alternate embodiment.
Regarding claim 14, see the rejections for claims 6 and 7 above. Note that the difference between claim 14 and claims 6 and 7 is that claim 14 is directed towards a machine that performs the method whereas claims 6 and 7 are directed towards the method. See the rejection for claim 9 above which covers the alternate embodiment. Additionally, note that claim 14 incorporates all of the limitations of claims 6 and 7 into a single claim; in understanding the rejection for claim 14 be sure to read the rejections for both claims 6 and 7.
Regarding claim 15, see the rejection for claim 8 above. Note that the only difference between claim 15 and claim 8 is that claim 15 is directed towards a machine that performs the method whereas claim 8 is directed towards the method. See the rejection for claim 9 above which covers the alternate embodiment.
Regarding claim 16, see the rejection for claim 1 above. Note that the only difference between claim 16 and claim 1 is that claim 16 is directed towards an article of manufacture that holds instructions for the method whereas claim 1 is directed towards the method, i.e. claim 16 is just an alternate embodiment of the method of claim 1 and is covered by the rejection for claim 1.
Regarding claim 17, see the rejection for claim 2 above. Note that the only difference between claim 17 and claim 2 is that claim 17 is directed towards an article of manufacture that holds instructions for the method whereas claim 2 is directed towards the method, i.e. claim 17 is just an alternate embodiment of the method of claim 2 and is covered by the rejection for claim 2.
Regarding claim 18, see the rejections for claims 3 and 4 above. Note that the only difference between claim 18 and claims 3 and 4 is that claim 18 is directed towards an article of manufacture that holds instructions for the method whereas claims 3 and 4 are directed towards the method, i.e. claim 18 is just an alternate embodiment of the methods of claims 3 and 4 and is covered by the rejections for claims 3 and 4. Additionally, note that claim 18 incorporates all of the limitations of claims 3 and 4 into a single claim; in understanding the rejection for claim 18 be sure to read the rejections for both claims 3 and 4.
Regarding claim 19, see the rejection for claim 5 above. Note that the only difference between claim 19 and claim 5 is that claim 19 is directed towards an article of manufacture that holds instructions for the method whereas claim 5 is directed towards the method, i.e. claim 19 is just an alternate embodiment of the method of claim 5 and is covered by the rejection for claim 5.
Regarding claim 20, see the rejections for claims 6 and 7 above. Note that the only difference between claim 20 and claims 6 and 7 is that claim 20 is directed towards an article of manufacture that holds instructions for the method whereas claims 6 and 7 are directed towards the method, i.e. claim 20 is just an alternate embodiment of the methods of claims 6 and 7 and is covered by the rejections for claims 6 and 7. Additionally, note that claim 20 incorporates all of the limitations of claims 6 and 7 into a single claim; in understanding the rejection for claim 20 be sure to read the rejections for both claims 6 and 7.
Citation of Pertinent Prior Art
The prior art made of record and no relied upon is considered pertinent to the applicant’s disclosure. Bekas (US 20180067720 A1) discusses a mixed-precision computation environment that can use hardware devices with different levels of precision. In an embodiment of this invention, an iterative algorithm, including a GMRES, can be used to solve systems of linear equations. The computationally intensive matrix-vector operations of the iterative algorithm can be done in a lower precision environment, while the less computationally intensive operations can be done in a high precision environment. Quirynen et. al. (US 20190243320 A1) discloses a system and method for solving a dynamic optimization problem using an iterative algorithm that has an iterative preconditioning operation. It further discusses the computational complexities of a preconditioner. Gonzalez et. al. (US 10305980 B1) teaches a mixed-precision approach to computation that looks for operations that can be computed with a lower precision than the native precision of a computer system and performs other operations at a higher precision. In some embodiments of the invention, systems of equations can be solved, potentially using iterative algorithms like GMRES, that split the computations among different GPUs which may potentially use different precisions.
Conclusion
The prior art made of record and not relied upon is considered pertinent to Applicant's
disclosure. Applicant is required under 37 C.F.R. § 1.111(c) to consider these references fully
when responding to this action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Zane A Rawlings whose telephone number is (571)270-3372. The examiner can normally be reached M-F, 8am to 5pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alexey Shmatov can be reached at (571) 270-3428. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Z.A.R./Examiner, Art Unit 2123
/ALEXEY SHMATOV/Supervisory Patent Examiner, Art Unit 2123
Prosecution Insights