DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Claims 1-20 are presented for examination.
Specification
The disclosure is objected to because of the following:
The specification lacks the section heading “Field of the Invention”.
Arrangement of the Specification
As provided in 37 CFR 1.77(b), the specification of a utility application should include the following sections in order. Each of the lettered items should appear in upper case, without underlining or bold type, as a section heading. If no text follows the section heading, the phrase “Not Applicable” should follow the section heading:
(a) TITLE OF THE INVENTION…
(e) BACKGROUND OF THE INVENTION.
(1) Field of the Invention…
Content of Specification…
(e) Background of the Invention: See MPEP § 608.01(c). The specification should set forth the Background of the Invention in two parts:
(1) Field of the Invention: A statement of the field of art to which the invention pertains. This statement may include a paraphrasing of the applicable U.S. patent classification definitions of the subject matter of the claimed invention. This item may also be titled "Technical Field."…
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Independent claim 1 Step 1: a method (process = 2019 PEG Step 1 = yes)
Independent claim 1 Step 2A, Prong One: Independent claim 1 recites:
for numerical simulations involving a partial differential equation to produce corresponding output data; training the neural network across the tensors of the multiple processors with the input data and the corresponding output data to produce a surrogate model of the partial differential equation… generating corresponding subsequent output data utilizing the surrogate model
Claim 1 is substantially drawn to mathematical concepts: mathematical relationships, formulas or equations, and calculations. Information and/or data also fall within the realm of abstract ideas because information and data are intangible. See Electric Power Group1 (Electric Power hereinafter): “Information… is an intangible”.
As to the limitations "training the neural network across the tensors of the multiple processors with the input data and the corresponding output data to produce a surrogate model of the partial differential equation", under its broadest reasonable interpretation, the training is a mathematical concept. The specification reads (underline emphasis added):
'[0085]… training an FNO-based surrogate model for simulating turbulent flow around a sphere by solving the 3D Navier-Stokes equation…
[0086] The training data is simulated with WaterLily.jl, an open-source Julia package for solving the 2D and 3D Navier-Stokes equations with the geometric multigrid method. A Julia function is implemented that takes the location of the sphere as input, solves the 3D Navier-Stokes equation with Waterlily, and outputs the scalar vorticity as a function of space and time (i.e., as a 4D tensor)…
[0092] The second example involves training an FNO for simulating CO2 flow in an industry-scale carbon capture scenario. For simulating the training data, the Sleipner 2019 benchmark model is utilized for simulating the training data. The Sleipner 2019 benchmark model is a real-world geological model for 3D numerical reservoir simulations'
As to the limitations "generating corresponding subsequent output data utilizing the surrogate model", under its broadest reasonable interpretation, the generating is a mathematical concept. The specification reads (underline emphasis added):
'[0085]… training an FNO-based surrogate model for simulating turbulent flow around a sphere by solving the 3D Navier-Stokes equation…
[0092] The second example involves training an FNO for simulating CO2 flow in an industry-scale carbon capture scenario. For simulating the training data, the Sleipner 2019 benchmark model is utilized for simulating the training data. The Sleipner 2019 benchmark model is a real-world geological model for 3D numerical reservoir simulations…
[0096]… Simulations with the trained FNO take around 0.12 seconds (on the ND96amsr VM), whereas the average simulation time with OPM on the E8s VM is 6.8 hours… The FNO provides a fast and low-cost surrogate model for optimizing well placement or uncertainty quantification'
If a claim limitation, under its broadest reasonable interpretation, covers abstract ideas, then it falls within groupings of abstract ideas (2019 PEG Step 2A, Prong One: Abstract Idea Grouping? = Yes).
Independent claim 1 Step 2A Prong two: As to the limitations “obtaining input data relating to a physical system… receiving subsequent input data", they describe the concept of “mere data gathering”, which corresponds to the concepts identified as abstract ideas by the courts. Data gathering, including when limited to particular content does not change its character as information, is also within the realm of abstract ideas. Data gathering has not been held by the courts to be enough to qualify as “significantly more”. See Electric Power.
As to the limitations "partitioning tensors of a neural network across multiple processors; distributing the input data across multiple parallel cloud processing resources", they represent no more than just “apply it” limitations, because they invoke computers or other machinery merely as a tool to perform an existing process.
This judicial exception is not integrated into a practical application of the exception (2019 PEG Step 2A, Prong Two: Additional elements that integrate the Judicial exception/Abstract idea into a practical application? = NO).
Independent claim 1 Step 2B: As discussed with respect to Step 2A, the claim recites data gathering, these limitations are recited at a high level of generality; and therefore, remain insignificant extra-solution activity even upon reconsideration.
As discussed with respect to Step 2A, Prong two, limitations invoking computers or other machinery merely as a tool to perform an existing process are just “apply it” limitations. See MPEP 2106.05(f)(2). As to the limitations "partitioning tensors of a neural network across multiple processors", see for example in the Specification (underline emphasis added):
"[0017]… Partitioning the tensors (e.g., partitioning the neural network model across processors on different physical or virtual machines) so that no single machine has to store the entire neural network model can be termed ‘model parallelism’"
Thus, taken alone the individual additional elements do not amount to significantly more than the above-identified judicial exception (the abstract idea). Looking at the additional elements as an ordered combination adds nothing that is not already present when looking at the additional elements taken individually. There is no indication that their combination improves the functioning of a computer itself or improves any other technology (underline emphasis added). Therefore, the claim does not amount to significantly more than the abstract idea itself (2019 PEG Step 2B: NO).
Independent claim 4 Step 1: a device (system = 2019 PEG Step 1 = yes)
Independent claim 4 Step 2A, Prong One: Independent claim 4 recites:
simulate the physical system by employing partial differential equations on the other computing resources to produce corresponding output data… to train the neural network across the tensors of the multiple parallel processors with the input data and the corresponding output data to produce a trained surrogate model of the partial differential equations… generate corresponding subsequent output data utilizing the trained surrogate model
Claim 4 is substantially drawn to mathematical concepts. (See Independent claim 1, Step 2A Prong One above).
As to the limitations "simulate the physical system by employing partial differential equations on the other computing resources to produce corresponding output data", under its broadest reasonable interpretation, the simulation is a mathematical concept. The specification reads (underline emphasis added):
'[00102]… simulate the physical system by employing a partial differential equation on computing resources to produce corresponding output data. The partial differential equation can entail a single partial differential equation or multiple partial differential equations'.
If a claim limitation, under its broadest reasonable interpretation, covers abstract ideas, then it falls within groupings of abstract ideas (2019 PEG Step 2A, Prong One: Abstract Idea Grouping? = Yes).
Independent claim 4 Step 2A Prong two: As to the limitations "a communication component configured to communicate with other computing resources; a hybrid parallelism manager configured to", they are recited as performing generic computer functions routinely used in computer applications.
As to the limitations “obtain input data relating to a physical system… receive subsequent input data", they describe the concept of “mere data gathering”. (See Independent claim 1, Step 2A Prong two above).
As to the limitations "partition tensors of a neural network across multiple parallel processors of the other computing resources; distribute the input data and the corresponding output data across the multiple parallel processors", they represent no more than just “apply it” limitations, because they invoke computers or other machinery merely as a tool to perform an existing process.
This judicial exception is not integrated into a practical application of the exception (2019 PEG Step 2A, Prong Two: Additional elements that integrate the Judicial exception/Abstract idea into a practical application? = NO).
Independent claim 4 Step 2B: As discussed with respect to Step 2A, Prong two, the claim recites the limitations "a communication component configured to communicate with other computing resources; a hybrid parallelism manager configured to", they are interpreted as drawn to a generic computer. They are recited at a high level of generality and as performing generic computer functions routinely used in computer applications. Generic computer components recited as performing generic computer functions that are well-understood, routine and conventional activities amount to no more than implementing the abstract idea with a computerized system. The use of a computer to implement the abstract idea of a mathematical or mental algorithm has not been held by the courts to be enough to qualify as “significantly more”. Their collective functions merely provide conventional computer implementation, which is described in the specification as (underline emphasis added):
'[00113] Computing devices 1502 can include a communication component 1508, a processor 1510, storage resources (e.g., storage) 1512, and/or hybrid parallelism manager…
[00118]… each of computing devices 1502 can have an instance of the hybrid parallelism manager…
[00119] The term "device," "computer," or "computing device" as used herein can mean any type of device that has some amount of processing capability and/or storage capability'
As discussed with respect to Step 2A, the claim recites data gathering, these limitations are recited at a high level of generality; and therefore, remain insignificant extra-solution activity even upon reconsideration.
As discussed with respect to Step 2A, Prong two, limitations invoking computers or other machinery merely as a tool to perform an existing process are just “apply it” limitations. (See Independent claim 1, Step 2B above).
Thus, taken alone the individual additional elements do not amount to significantly more than the above-identified judicial exception (the abstract idea). Looking at the additional elements as an ordered combination adds nothing that is not already present when looking at the additional elements taken individually. There is no indication that their combination improves the functioning of a computer itself or improves any other technology (underline emphasis added). Therefore, the claim does not amount to significantly more than the abstract idea itself (2019 PEG Step 2B: NO).
Independent claim 9 Step 1: a system (system = 2019 PEG Step 1 = yes)
Independent claim 9 Step 2A, Prong One: Independent claim 9 recites:
for numerical simulations involving partial differential equations to produce corresponding output data; train the neural network across the tensors of the multiple parallel processors with the input data and the corresponding output data to produce a surrogate model of the partial differential equations
Claim 9 is substantially drawn to mathematical concepts. (See Independent claim 1, Step 2A Prong One above). If a claim limitation, under its broadest reasonable interpretation, covers abstract ideas, then it falls within groupings of abstract ideas (2019 PEG Step 2A, Prong One: Abstract Idea Grouping? = Yes).
Independent claim 9 Step 2A Prong two: As to the limitations "a hardware processing unit; and a storage resource storing computer-readable instructions which, when executed by the hardware processing unit, cause the hardware processing unit to", they are recited as performing generic computer functions routinely used in computer applications.
As to the limitations “obtain input data relating to a physical system… receive subsequent input data", they describe the concept of “mere data gathering”. (See Independent claim 1, Step 2A Prong two above).
As to the limitations "partition tensors of a neural network across multiple parallel processors; distribute the input data across multiple parallel processing resources", they represent no more than just “apply it” limitations, because they invoke computers or other machinery merely as a tool to perform an existing process.
As to the limitations "generate corresponding subsequent output data utilizing the surrogate model without performing additional numerical simulations on the subsequent input data", they represent no more than just “apply it” limitations, because they recite only the idea of a solution or outcome, i.e., they fail to recite details of how a solution to a problem is accomplished.
This judicial exception is not integrated into a practical application of the exception (2019 PEG Step 2A, Prong Two: Additional elements that integrate the Judicial exception/Abstract idea into a practical application? = NO).
Independent claim 9 Step 2B: As discussed with respect to Step 2A, Prong two, the claim recites generic computer function limitations and are interpreted as drawn to a generic computer. (See Independent claim 4, Step 2B above).
As discussed with respect to Step 2A, the claim recites data gathering, these limitations are recited at a high level of generality; and therefore, remain insignificant extra-solution activity even upon reconsideration.
As discussed with respect to Step 2A, Prong two, limitations invoking computers or other machinery merely as a tool to perform an existing process are just “apply it” limitations. (See Independent claim 1, Step 2B above).
As discussed with respect to Step 2A, Prong two, limitations reciting only the idea of a solution or outcome are just “apply it” limitations, because they fail to recite details of how a solution to a problem is accomplished. See MPEP 2106.05(f)(1). As to the limitations "without performing additional numerical simulations", the limitations are so broad that little is known about how they are performed. The specification merely reads:
'[00132]… receive subsequent input data and generate corresponding subsequent output data utilizing the surrogate model without performing additional numerical simulations on the subsequent input data'
Thus, taken alone the individual additional elements do not amount to significantly more than the above-identified judicial exception (the abstract idea). Looking at the additional elements as an ordered combination adds nothing that is not already present when looking at the additional elements taken individually. There is no indication that their combination improves the functioning of a computer itself or improves any other technology (underline emphasis added). Therefore, the claim does not amount to significantly more than the abstract idea itself (2019 PEG Step 2B: NO).
Dependent claims Step 2A, Prong One: dependent claims further the abstract ideas of their independent claims. (See Independent claim 1, Step 2A, Prong One above).
As to the limitations "8… wherein the hybrid parallelism manager is configured to cause the further input data to be run on the individual trained surrogate model to produce corresponding further output data", "13… to generate additional output data from the surrogate model that reflects the different PDE coefficients", under their broadest reasonable interpretations, they are mathematical concepts. (See Independent claim 1, Step 2A Prong One above).
If a claim limitation, under its broadest reasonable interpretation, covers abstract ideas, then it falls within groupings of abstract ideas (2019 PEG Step 2A, Prong One: Abstract Idea Grouping? = Yes).
Dependent claims Step 2A Prong two: As to the limitations "2… wherein partitioning tensors of a neural network across multiple processors comprises partitioning the tensors across multiple graphics processing units (GPUs), or wherein partitioning tensors of a neural network across multiple processors comprises partitioning the tensors across multiple deep learning processors or tensor processing units", "3… wherein distributing the input data across multiple parallel cloud processing resources comprises distributing the input data across multiple virtual machines or physical machines", "5… wherein the hybrid parallelism manager includes an application program interface (API) for distributing the input data and the corresponding output data across the multiple parallel processors", "10… wherein the hardware processing unit is further configured to cause the surrogate model to be stored in a library", "11… wherein the library comprises a registry stored on a cloud object storage component configured for storing unstructured data", they represent no more than just “apply it” limitations, because they invoke computers or other machinery merely as a tool to perform an existing process.
As to the limitations "6… wherein the hybrid parallelism manager is configured to store the trained surrogate model in a library that includes trained surrogate models relating to various physical systems", "14… wherein the hardware processing unit comprises a central processing unit", "15… wherein the multiple parallel processing resources include the central processing unit or wherein the central processing unit is located on different processing resources", "16… wherein the multiple parallel processing resources include the multiple processors", "17… wherein the multiple processors comprise multiple processors on a single physical computing device or wherein the multiple processors span multiple physical computing devices", "18… wherein the multiple processors comprise multiple processors on a single virtual machine or wherein the multiple processors span multiple virtual machines", "19… wherein the multiple parallel processing resources include the multiple processors, or wherein the multiple parallel processing resources are distinct from the multiple processors", "20… wherein the system includes the multiple parallel processing resources and the multiple processors, or wherein the system communicates with the multiple parallel processing resources and the multiple processors", they are recited as performing generic computer functions routinely used in computer applications.
As to the limitations "7… wherein the hybrid parallelism manager is configured to generate a graphical user interface through which further input data relating to an individual physical system can be paired with an individual trained surrogate model in the library", they are recited as a GUI performing generic computer functions routinely used in computer applications.
As to the limitations “12… wherein the hardware processing unit is further configured to receive additional input data relating to the physical system that has different partial differential equation (PDE) coefficients that reflect material parameters of the physical system", "13… wherein the hardware processing unit is configured to retrieve the surrogate model from the library", they describe the concept of “mere data gathering”. (See Independent claim 1, Step 2A Prong two above).
This judicial exception is not integrated into a practical application of the exception (2019 PEG Step 2A, Prong Two: Additional elements that integrate the Judicial exception/Abstract idea into a practical application? = NO).
Dependent claims Step 2B: As discussed with respect to Step 2A, Prong two, limitations invoking computers or other machinery merely as a tool to perform an existing process are just “apply it” limitations. (See Independent claim 1, Step 2B above).
As discussed with respect to Step 2A, Prong two, claims reciting generic computer function limitations are interpreted as drawn to a generic computer. (See Independent claim 4, Step 2B above).
As discussed with respect to Step 2A, Prong two, the GUI limitations have been found by the courts as not adding an inventive component/concept to claims to render them patentable. A GUI is a well-known graphical modeling means, and it is well-understood, routine, and conventional in the art. A GUI is a well-known graphical modeling means, and it is well-understood, routine, and conventional in the art. See MPEP 2106.04(a)(2), 2106.05(a), 2106.05(f).
As discussed with respect to Step 2A, the claims recite data gathering, these limitations are recited at a high level of generality; and therefore, remain insignificant extra-solution activity even upon reconsideration.
Therefore, the claims do not amount to significantly more than the abstract idea itself (2019 PEG Step 2B: NO).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103(a) are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Examiner would like to point out that any reference to specific figures, columns and lines should not be considered limiting in any way, the entire reference is considered to provide disclosure relating to the claimed invention.
Claims 1-3 and 9-20 are rejected under 35 U.S.C. 103(a) as being unpatentable over Lior Horesh, (Horesh hereinafter), U.S. Patent 11531902, taken in view of Paris Georgios Perdikaris, (Perdikaris hereinafter), U.S. Pre–Grant publication 20230214661.
As to claim 1, Horesh discloses a method comprising: obtaining input data relating to a physical system (see "devices provide both input and output capabilities such as remote computer(s)" in col. 29, lines 51-53); partitioning tensors of a neural network across multiple processors (see "generating, training, and managing neural networks (e.g., deep tensor neural networks)… performed by, for example, the processor component, the network management component, the network component, and/or the network" in col. 26, lines 55-60; "processor… comprise… graphics processing units (GPUs)" in col. 25, lines 41-43); distributing the input data across multiple parallel (see "network and the network management component, in conjunction with the network component, can perform forward propagation of the input data (e.g., as formatted as tensor-formatted input data) to facilitate extracting the features of the input data, for example in accordance with the applicable equations (e.g., equations relating to forward propagation)… network management component and/or the network can extract the features of the input data from the tensor-formatted input data in parallel" in col. 27, lines 23-36) cloud processing resources (see "component can emulate an electronic component via a virtual machine, e.g., within a cloud computing system" in col. 33, lines 5-7) for numerical simulations (see "system… utilized… deep tensor neural network… simulation" in col. 4, lines 33-36) involving a partial differential equation to produce corresponding output data (see "employing the network management component 220 and network 204) can create a stable tensor network architecture for the network… consider Eq. (26) to be the explicit Euler discretization of the following system of continuous differential equations… Eq.(27) over the time interval [0, T]…" in col. 16, lines 37-67); training the neural network across the tensors (see "apply training data to the deep tensor neural network to facilitate training the deep tensor neural network" in col. 4, lines 5-6) of the multiple processors with the input data and the corresponding output data to produce a (see "model using the disclosed tensor framework is able to be fit quickly while maintaining desirably high accuracy, and greater improvement in the model using the disclosed tensor framework as the parameters associated with the network are updated" in col. 21, lines 40-45); and, receiving subsequent input data and generating corresponding subsequent output data utilizing the (see "features of the input data can be extracted from the tensor-formatted input data based at least in part on the tensor-formatted parameters… network and the network management component, in conjunction with the network component, can perform forward propagation of the input data (e.g., as formatted as tensor-formatted input data) to facilitate extracting the features of the input data, for example in accordance with the applicable equations (e.g., equations relating to forward propagation)… network management component and/or the network can extract the features of the input data from the tensor-formatted input data in parallel. At 906, the tensor-formatted input data and the tensor-formatted parameters can be evolved based at least in part on a defined tensor-tensor layer evolution rule. The network (e.g., as managed by the network management component) can evolve the tensor-formatted input data and the tensor-formatted parameters (e.g., in the network) based at least in part on (e.g. in accordance with) the defined tensor-tensor layer evolution rule. At 908, output data can be generated based at least in part on the extracted features and evolution of the tensor-formatted input data" in col. 27, lines 18-47).
Horesh does not disclose, but Perdikaris discloses a surrogate . (See “[0028]… applying machine learning techniques to approximate operators between spaces of functions… In the context of learning the response of systems governed by differential equations, these learned models can function as fast surrogates of traditional numerical solvers").
Horesh and Perdikaris are analogous art because they are related to simulating natural or physical systems.
Therefore, it would have been obvious to one of ordinary skill in this art before the effective filing date of the claimed invention to use Perdikaris with Horesh, because Perdikaris points out that "[0133] [a] main application of operator learning methods is for PDEs, where they are used as surrogates for traditional numerical solvers… the solution of a PDE under many different initial conditions can be obtained quickly. This is incredibly useful in design and optimal control problems where many inputs must be tested to produce a desired outcome from a physical system", and as a result, Perdikaris reports that "[0133]… successful application of operator learning methods to predict the output of physical systems from control inputs can have a significant impact in the design of optimal inputs and controls".
As to claim 2, Horesh discloses wherein partitioning tensors of a neural network across multiple processors comprises partitioning the tensors across multiple graphics processing units (GPUs) (see "processor… comprise… graphics processing units (GPUs)" in col. 25, lines 41-43)
As to claim 3, Horesh discloses wherein distributing the input data across multiple parallel (see "network management component and/or the network can extract the features of the input data from the tensor-formatted input data in parallel" in col. 27, lines 23-36) cloud (see "component can emulate an electronic component via a virtual machine, e.g., within a cloud computing system" in col. 33, lines 5-7) processing resources comprises distributing the input data across multiple virtual machines or physical machines (see "devices provide both input and output capabilities such as remote computer(s)" in col. 29, lines 51-53).
As to claim 9, Horesh discloses a system comprising: a hardware processing unit (see "a suitable operating environment 1000 for implementing… include a computer" in col. 28, lines 31-33); and a storage resource storing computer-readable instructions which, when executed by the hardware processing unit (see "system bus 1018 couples system components including, but not limited to, the system memory 1016 to the processing unit" in col. 28, lines 35-37), cause the hardware processing unit to: obtain input data relating to a physical system (see "devices provide both input and output capabilities such as remote computer(s)" in col. 29, lines 51-53); partition tensors of a neural network across multiple parallel processors (see "generating, training, and managing neural networks (e.g., deep tensor neural networks)… performed by, for example, the processor component, the network management component, the network component, and/or the network" in col. 26, lines 55-60; "processor… comprise… graphics processing units (GPUs)" in col. 25, lines 41-43); distribute the input data across multiple parallel (see "network management component and/or the network can extract the features of the input data from the tensor-formatted input data in parallel" in col. 27, lines 23-36) processing resources (see "network and the network management component, in conjunction with the network component, can perform forward propagation of the input data (e.g., as formatted as tensor-formatted input data) to facilitate extracting the features of the input data, for example in accordance with the applicable equations (e.g., equations relating to forward propagation)… network management component and/or the network can extract the features of the input data from the tensor-formatted input data in parallel" in col. 27, lines 23-36) for numerical simulations (see "system… utilized… deep tensor neural network… simulation" in col. 4, lines 33-36) involving partial differential equations to produce corresponding output data (see "employing the network management component 220 and network 204) can create a stable tensor network architecture for the network… consider Eq. (26) to be the explicit Euler discretization of the following system of continuous differential equations… Eq.(27) over the time interval [0, T]…" in col. 16, lines 37-67); train the neural network across the tensors (see "apply training data to the deep tensor neural network to facilitate training the deep tensor neural network" in col. 4, lines 5-6) of the multiple parallel processors (see '"processor" can refer to substantially any computing processing unit or device comprising… parallel platforms; and parallel platforms with distributed shared memory… processor can also be implemented as a combination of computing processing units' in col. 33, lines 29-48) with the input data and the corresponding output data to produce a (see "model using the disclosed tensor framework is able to be fit quickly while maintaining desirably high accuracy, and greater improvement in the model using the disclosed tensor framework as the parameters associated with the network are updated" in col. 21, lines 40-45); and, receive subsequent input data and generate corresponding subsequent output data utilizing the (see "features of the input data can be extracted from the tensor-formatted input data based at least in part on the tensor-formatted parameters… network and the network management component, in conjunction with the network component, can perform forward propagation of the input data (e.g., as formatted as tensor-formatted input data) to facilitate extracting the features of the input data, for example in accordance with the applicable equations (e.g., equations relating to forward propagation)… network management component and/or the network can extract the features of the input data from the tensor-formatted input data in parallel. At 906, the tensor-formatted input data and the tensor-formatted parameters can be evolved based at least in part on a defined tensor-tensor layer evolution rule. The network (e.g., as managed by the network management component) can evolve the tensor-formatted input data and the tensor-formatted parameters (e.g., in the network) based at least in part on (e.g. in accordance with) the defined tensor-tensor layer evolution rule. At 908, output data can be generated based at least in part on the extracted features and evolution of the tensor-formatted input data" in col. 27, lines 18-47)…
Horesh does not disclose, but Perdikaris discloses a surrogate (see “[0028]… applying machine learning techniques to approximate operators between spaces of functions… In the context of learning the response of systems governed by differential equations, these learned models can function as fast surrogates of traditional numerical solvers")… without performing additional numerical simulations on the subsequent input data (see "[0042]… u could represent the initial condition to a PDE and s the corresponding solution. In this case, G would correspond to the true solution operator and F would be an approximate surrogate model… with an appropriate choice of architecture for F the approximate model can result in significant computational speedups and the ability to efficiently compute sensitivities with respect to the inputs").
Therefore, it would have been obvious to one of ordinary skill in this art before the effective filing date of the claimed invention to use Perdikaris with Horesh, (see supra).
As to claim 10, Horesh discloses wherein the hardware processing unit is further configured to cause the (see "respective components can execute from various computer readable media having various data structures stored thereon. The components can communicate via local and/or remote processes such as in accordance with a signal having one or more data packets (e.g., data from one component interacting with another component in a local system, distributed system, and/or across a network such as the Internet with other systems via the signal)… a component can emulate an electronic component via a virtual machine, e.g., within a cloud computing system" in col. 32, line 52 to col. 33, line 7) to be stored in a library (see "system bus 1018 couples system components including, but not limited to, the system memory 1016 to the processing unit" in col. 28, lines 35-37).
Horesh does not disclose, but Perdikaris discloses a surrogate . (See “[0028]… applying machine learning techniques to approximate operators between spaces of functions… In the context of learning the response of systems governed by differential equations, these learned models can function as fast surrogates of traditional numerical solvers").
Therefore, it would have been obvious to one of ordinary skill in this art before the effective filing date of the claimed invention to use Perdikaris with Horesh, (see supra).
As to claim 11, Horesh discloses wherein the library comprises a registry stored on a cloud (see "respective components can execute from various computer readable media having various data structures stored thereon. The components can communicate via local and/or remote processes such as in accordance with a signal having one or more data packets (e.g., data from one component interacting with another component in a local system, distributed system, and/or across a network such as the Internet with other systems via the signal)… a component can emulate an electronic component via a virtual machine, e.g., within a cloud computing system" in col. 32, line 52 to col. 33, line 7) object storage component configured for storing unstructured data (see "system bus 1018 couples system components including, but not limited to, the system memory 1016 to the processing unit" in col. 28, lines 35-37).
As to claim 12, Horesh discloses wherein the hardware processing unit is further configured to receive additional input data relating to the physical system that has different partial differential equation (PDE) coefficients that reflect material parameters of the physical system (see "features of the input data can be extracted from the tensor-formatted input data based at least in part on the tensor-formatted parameters… network and the network management component, in conjunction with the network component, can perform forward propagation of the input data (e.g., as formatted as tensor-formatted input data) to facilitate extracting the features of the input data, for example in accordance with the applicable equations (e.g., equations relating to forward propagation)" in col. 27, lines 18-29).
As to claim 13, Horesh discloses wherein the hardware processing unit is configured to retrieve the (see "respective components can execute from various computer readable media having various data structures stored thereon. The components can communicate via local and/or remote processes such as in accordance with a signal having one or more data packets (e.g., data from one component interacting with another component in a local system, distributed system, and/or across a network such as the Internet with other systems via the signal)… a component can emulate an electronic component via a virtual machine, e.g., within a cloud computing system" in col. 32, line 52 to col. 33, line 7) from the library (see "system bus 1018 couples system components including, but not limited to, the system memory 1016 to the processing unit" in col. 28, lines 35-37) and to generate additional output data from the (see "features of the input data can be extracted from the tensor-formatted input data based at least in part on the tensor-formatted parameters… network and the network management component, in conjunction with the network component, can perform forward propagation of the input data (e.g., as formatted as tensor-formatted input data) to facilitate extracting the features of the input data, for example in accordance with the applicable equations (e.g., equations relating to forward propagation)… network management component and/or the network can extract the features of the input data from the tensor-formatted input data in parallel. At 906, the tensor-formatted input data and the tensor-formatted parameters can be evolved based at least in part on a defined tensor-tensor layer evolution rule. The network (e.g., as managed by the network management component) can evolve the tensor-formatted input data and the tensor-formatted parameters (e.g., in the network) based at least in part on (e.g. in accordance with) the defined tensor-tensor layer evolution rule. At 908, output data can be generated based at least in part on the extracted features and evolution of the tensor-formatted input data" in col. 27, lines 18-47).
Horesh does not disclose, but Perdikaris discloses a surrogate . (See “[0028]… applying machine learning techniques to approximate operators between spaces of functions… In the context of learning the response of systems governed by differential equations, these learned models can function as fast surrogates of traditional numerical solvers").
Therefore, it would have been obvious to one of ordinary skill in this art before the effective filing date of the claimed invention to use Perdikaris with Horesh, (see supra).
As to claim 14, Horesh discloses wherein the hardware processing unit comprises a central processing unit (see "computer 1012 can also include a processing unit" in col. 28, lines 33-35).
As to claim 15, Horesh discloses wherein the multiple parallel processing resources include (see "network and the network management component, in conjunction with the network component, can perform forward propagation of the input data (e.g., as formatted as tensor-formatted input data) to facilitate extracting the features of the input data, for example in accordance with the applicable equations (e.g., equations relating to forward propagation)… network management component and/or the network can extract the features of the input data from the tensor-formatted input data in parallel" in col. 27, lines 23-36).
As to claim 16, Horesh discloses wherein the multiple parallel (see "network management component and/or the network can extract the features of the input data from the tensor-formatted input data in parallel" in col. 27, lines 23-36) processing resources include the multiple processors (see '"processor" can refer to substantially any computing processing unit or device comprising… parallel platforms; and parallel platforms with distributed shared memory… processor can also be implemented as a combination of computing processing units' in col. 33, lines 29-48).
As to claim 17, Horesh discloses wherein the multiple processors comprise multiple processors on a single physical computing device (see "a suitable operating environment 1000 for implementing… include a computer" in col. 28, lines 31-33)
As to claim 18, Horesh discloses wherein the multiple processors comprise multiple processors on a single virtual machine (see "component can emulate an electronic component via a virtual machine, e.g., within a cloud computing system" in col. 33, lines 5-7) .
As to claim 19, Horesh discloses wherein the multiple parallel (see "network management component and/or the network can extract the features of the input data from the tensor-formatted input data in parallel" in col. 27, lines 23-36) processing resources include the multiple processors, or wherein the multiple parallel processing resources are distinct from the multiple processors (see "processor… employ one or more processors, microprocessors, or controllers…" in col. 25, lines 21-33).
As to claim 20, Horesh discloses wherein the system includes the multiple parallel (see "network management component and/or the network can extract the features of the input data from the tensor-formatted input data in parallel" in col. 27, lines 23-36) processing resources and the multiple processors, or wherein the system communicates with the multiple parallel processing resources and the multiple processors (see "processor… employ one or more processors, microprocessors, or controllers that can process data, such as information relating to tensors, a network, nodes, edges, network structure, input data, output data, parameters, formulas or equations, calculations, network training, objective functions, simulated output data, updates to parameters, tensor-tensor layer evolution rules, defined network management criteria, algorithms (e.g., network management algorithms, network training algorithms… data traffic flows (e.g., between components or devices, and/or across a network(s)), protocols, policies, interfaces, tools, and/or other information" in col. 25, lines 21-33).
Claims 4-8 are rejected under 35 U.S.C. 103(a) as being unpatentable over Horesh taken in view of Perdikaris and further in view of Srinivas Sridharan, (Sridharan hereinafter), U.S. Patent 11704565.
As to claim 4, Horesh discloses a device comprising: a communication component configured to communicate with other computing resources (see "network management component 700 can comprise a communicator component" in col. 22, lines 6-8)… obtain input data relating to a physical system (see "devices provide both input and output capabilities such as remote computer(s)" in col. 29, lines 51-53); simulate the physical system (see "simulator component 718 can simulate outcomes ( e.g., expected or predicted outcomes) of the network" in col. 24, lines 33-35) by employing partial differential equations on the other computing resources to produce corresponding output data (see "employing the network management component 220 and network 204) can create a stable tensor network architecture for the network… consider Eq. (26) to be the explicit Euler discretization of the following system of continuous differential equations… Eq.(27) over the time interval [0, T]…" in col. 16, lines 37-67); partition tensors of a neural network across multiple parallel processors of the other computing resources (see "generating, training, and managing neural networks (e.g., deep tensor neural networks)… performed by, for example, the processor component, the network management component, the network component, and/or the network" in col. 26, lines 55-60; "processor… comprise… graphics processing units (GPUs)" in col. 25, lines 41-43); distribute the input data and the corresponding output data across the multiple parallel (see "network management component and/or the network can extract the features of the input data from the tensor-formatted input data in parallel" in col. 27, lines 23-36) processors (see "deep tensor neural network can generate output training data as an output from the deep tensor neural network based at least in part on the tensor-formatted training data applied as an input to the deep tensor neural network" in col. 4, lines 13-17; "processor… comprise… graphics processing units (GPUs)" in col. 25, lines 41-43) to train the neural network across the tensors (see "apply training data to the deep tensor neural network to facilitate training the deep tensor neural network" in col. 4, lines 5-6) of the multiple parallel processors (see '"processor" can refer to substantially any computing processing unit or device comprising… parallel platforms; and parallel platforms with distributed shared memory… processor can also be implemented as a combination of computing processing units' in col. 33, lines 29-48) with the input data and the corresponding output data to produce a trained (see "apply training data to the deep tensor neural network to facilitate training the deep tensor neural network" in col. 4, lines 5-6) (see "model using the disclosed tensor framework is able to be fit quickly while maintaining desirably high accuracy, and greater improvement in the model using the disclosed tensor framework as the parameters associated with the network are updated" in col. 21, lines 40-45); and, receive subsequent input data and generate corresponding subsequent output data utilizing the trained surrogate model (see "features of the input data can be extracted from the tensor-formatted input data based at least in part on the tensor-formatted parameters… network and the network management component, in conjunction with the network component, can perform forward propagation of the input data (e.g., as formatted as tensor-formatted input data) to facilitate extracting the features of the input data, for example in accordance with the applicable equations (e.g., equations relating to forward propagation)… network management component and/or the network can extract the features of the input data from the tensor-formatted input data in parallel. At 906, the tensor-formatted input data and the tensor-formatted parameters can be evolved based at least in part on a defined tensor-tensor layer evolution rule. The network (e.g., as managed by the network management component) can evolve the tensor-formatted input data and the tensor-formatted parameters (e.g., in the network) based at least in part on (e.g. in accordance with) the defined tensor-tensor layer evolution rule. At 908, output data can be generated based at least in part on the extracted features and evolution of the tensor-formatted input data" in col. 27, lines 18-47).
Horesh does not disclose, but Perdikaris discloses a surrogate . (See “[0028]… applying machine learning techniques to approximate operators between spaces of functions… In the context of learning the response of systems governed by differential equations, these learned models can function as fast surrogates of traditional numerical solvers").
Therefore, it would have been obvious to one of ordinary skill in this art before the effective filing date of the claimed invention to use Perdikaris with Horesh, (see supra).
While Horesh discloses "the network management component and/or the network can extract the features of the input data from the tensor-formatted input data in parallel" in col. 27, lines 23-36 and '"processor" can refer to substantially any computing processing unit or device comprising… parallel platforms; and parallel platforms with distributed shared memory… processor can also be implemented as a combination of computing processing units' in col. 33, lines 29-48, Horesh and Perdikaris do not disclose a hybrid parallelism manager configured to.
Sridharan discloses a hybrid parallelism manager configured to (see "Distributed machine learning can be implemented using a variety of parallelism patterns, such as data parallelism, model parallelism, or a hybrid of data and model parallelism… data parallelism uses the same model for each compute node, with each node processing different portions of the data. Model parallelism uses the same data for each compute node, with the model split among compute nodes" in col. 36, lines 35-42).
Horesh, Perdikaris, and Sridharan are analogous art because they are related to simulating natural or physical systems.
Therefore, it would have been obvious to one of ordinary skill in this art before the effective filing date of the claimed invention to use Sridharan with Horesh and Perdikaris, because Sridharan points out that "accuracy of a machine learning algorithm can be affected significantly by the quality of the data set used to train the algorithm. The training process can be computationally intensive and may require a significant amount of time on a conventional general-purpose processor. Accordingly, parallel processing hardware is used to train many types of machine learning algorithms. This is particularly useful for optimizing the training of neural networks, as the computations performed in adjusting the coefficients in neural networks lend themselves naturally to parallel implementations" (see col. 29, lines 13-23), and as a result, Sridharan reports that "[t]he computing architecture provided… can be configured to perform the types of parallel processing that is particularly suited for training and deploying neural networks for machine learning" (see col. 30, lines 8-11).
As to claim 5, Horesh discloses… distributing the input data and the corresponding output data across the multiple parallel (see "network management component and/or the network can extract the features of the input data from the tensor-formatted input data in parallel" in col. 27, lines 23-36) processors (see "devices provide both input and output capabilities such as remote computer(s)" in col. 29, lines 51-53).
Horesh and Perdikaris do not disclose, but Sridharan discloses wherein the hybrid parallelism manager (see "Distributed machine learning can be implemented using a variety of parallelism patterns, such as data parallelism, model parallelism, or a hybrid of data and model parallelism… data parallelism uses the same model for each compute node, with each node processing different portions of the data. Model parallelism uses the same data for each compute node, with the model split among compute nodes" in col. 36, lines 35-42) includes an application program interface (API) (see "Multi-purpose execution logic (e.g., execution units) within the graphics core(s) 415A-414B of the graphics core array 414 includes support for various 3D API shader languages and can execute multiple simultaneous execution threads associated with multiple shaders" in col. 9, lines 19-23).
Therefore, it would have been obvious to one of ordinary skill in this art before the effective filing date of the claimed invention to use Sridharan with Horesh and Perdikaris, (see supra).
As to claim 6, Horesh discloses… store the trained (see "apply training data to the deep tensor neural network to facilitate training the deep tensor neural network" in col. 4, lines 5-6) (see "communicator component 702 can receive information, via a desired interface, from a user and/or the database component… operations manager component 704 also can facilitate controlling data flow between the components of the network management component 700 and controlling data flow between the network management component 700 and another component(s) or device(s) (e.g., the network, a node, the network component, the database component, a display screen or other interface" in col. 22, lines 32-65) that includes trained (see "apply training data to the deep tensor neural network to facilitate training the deep tensor neural network" in col. 4, lines 5-6) (see "data store 726 can store data structures (e.g., user data, metadata), code structure(s) (e.g., modules, objects, hashes, classes, procedures) or instructions, information relating to… formulas or equations, calculations… simulated output data" in col. 25, lines 48-54).
Horesh does not disclose, but Perdikaris discloses a surrogate . (See “[0028]… applying machine learning techniques to approximate operators between spaces of functions… In the context of learning the response of systems governed by differential equations, these learned models can function as fast surrogates of traditional numerical solvers").
Therefore, it would have been obvious to one of ordinary skill in this art before the effective filing date of the claimed invention to use Sridharan with Horesh and Perdikaris, (see supra).
Horesh and Perdikaris do not disclose, but Sridharan discloses wherein the hybrid parallelism manager is configured (see "Distributed machine learning can be implemented using a variety of parallelism patterns, such as data parallelism, model parallelism, or a hybrid of data and model parallelism… data parallelism uses the same model for each compute node, with each node processing different portions of the data. Model parallelism uses the same data for each compute node, with the model split among compute nodes" in col. 36, lines 35-42)
Therefore, it would have been obvious to one of ordinary skill in this art before the effective filing date of the claimed invention to use Perdikaris with Horesh, (see supra).
As to claim 7, Horesh discloses… generate a graphical user interface through which further input data relating to an individual physical system can be paired with an individual trained (see "apply training data to the deep tensor neural network to facilitate training the deep tensor neural network" in col. 4, lines 5-6) (see "communicator component 702 can receive information, via a desired interface, from a user and/or the database component… operations manager component 704 also can facilitate controlling data flow between the components of the network management component 700 and controlling data flow between the network management component 700 and another component(s) or device(s) (e.g., the network, a node, the network component, the database component, a display screen or other interface" in col. 22, lines 32-65).
Horesh does not disclose, but Perdikaris discloses a surrogate . (See “[0028]… applying machine learning techniques to approximate operators between spaces of functions… In the context of learning the response of systems governed by differential equations, these learned models can function as fast surrogates of traditional numerical solvers").
Therefore, it would have been obvious to one of ordinary skill in this art before the effective filing date of the claimed invention to use Perdikaris with Horesh, (see supra).
Horesh and Perdikaris do not disclose, but Sridharan discloses wherein the hybrid parallelism manager is configured (see "Distributed machine learning can be implemented using a variety of parallelism patterns, such as data parallelism, model parallelism, or a hybrid of data and model parallelism… data parallelism uses the same model for each compute node, with each node processing different portions of the data. Model parallelism uses the same data for each compute node, with the model split among compute nodes" in col. 36, lines 35-42)
Therefore, it would have been obvious to one of ordinary skill in this art before the effective filing date of the claimed invention to use Sridharan with Horesh and Perdikaris, (see supra).
As to claim 8, Horesh discloses… cause the further input data to be run on the individual trained (see "apply training data to the deep tensor neural network to facilitate training the deep tensor neural network" in col. 4, lines 5-6) (see "features of the input data can be extracted from the tensor-formatted input data based at least in part on the tensor-formatted parameters… network and the network management component, in conjunction with the network component, can perform forward propagation of the input data (e.g., as formatted as tensor-formatted input data) to facilitate extracting the features of the input data, for example in accordance with the applicable equations (e.g., equations relating to forward propagation)… network management component and/or the network can extract the features of the input data from the tensor-formatted input data in parallel. At 906, the tensor-formatted input data and the tensor-formatted parameters can be evolved based at least in part on a defined tensor-tensor layer evolution rule. The network (e.g., as managed by the network management component) can evolve the tensor-formatted input data and the tensor-formatted parameters (e.g., in the network) based at least in part on (e.g. in accordance with) the defined tensor-tensor layer evolution rule. At 908, output data can be generated based at least in part on the extracted features and evolution of the tensor-formatted input data" in col. 27, lines 18-47).
Horesh does not disclose, but Perdikaris discloses a surrogate . (See “[0028]… applying machine learning techniques to approximate operators between spaces of functions… In the context of learning the response of systems governed by differential equations, these learned models can function as fast surrogates of traditional numerical solvers").
Therefore, it would have been obvious to one of ordinary skill in this art before the effective filing date of the claimed invention to use Perdikaris with Horesh, (see supra).
Horesh and Perdikaris do not disclose, but Sridharan discloses wherein the hybrid parallelism manager is configured (see "Distributed machine learning can be implemented using a variety of parallelism patterns, such as data parallelism, model parallelism, or a hybrid of data and model parallelism… data parallelism uses the same model for each compute node, with each node processing different portions of the data. Model parallelism uses the same data for each compute node, with the model split among compute nodes" in col. 36, lines 35-42)
Therefore, it would have been obvious to one of ordinary skill in this art before the effective filing date of the claimed invention to use Sridharan with Horesh and Perdikaris, (see supra).
Conclusion
Examiner would like to point out that any reference to specific figures, columns and lines should not be considered limiting in any way, the entire reference is considered to provide disclosure relating to the claimed invention.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JUAN CARLOS OCHOA whose telephone number is (571)272-2625. The examiner can normally be reached Mondays, Tuesdays, Thursdays, and Fridays 9:30AM - 8:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Renee Chavez can be reached at 571-270-1104. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JUAN C OCHOA/Primary Examiner, Art Unit 2186
1 Electric Power Group, LLC v. Alstom S.A., 119 USPQ2d 1739 Fed. Cir. 2016
Prosecution Insights