Prosecution Insights
Last updated: November 04, 2026
Application No. 17/743,112

PHYSICS-INFORMED ATTENTION-BASED NEURAL NETWORK

Non-Final OA §103
Filed
May 12, 2022
Priority
May 13, 2021 — provisional 63/188,087
Examiner
KIM, HARRISON CHAN YOUNG
Art Unit
2145
Tech Center
2100 — Computer Architecture & Software
Assignee
Origen
OA Round
3 (Non-Final)
61%
Grant Probability
Moderate
3-4
OA Rounds
0m
Est. Remaining
94%
With Interview

Examiner Intelligence

Grants 61% of resolved cases
61%
Career Allowance Rate
11 granted / 18 resolved
+6.1% vs TC avg
Strong +33% interview lift
Without
With
+33.3%
Interview Lift
resolved cases with interview
Typical timeline
4y 0m
Avg Prosecution
13 currently pending
Career history
47
Total Applications
across all art units

Statute-Specific Performance

§101
38.6%
-1.4% vs TC avg
§103
47.7%
+7.7% vs TC avg
§102
4.6%
-35.4% vs TC avg
§112
8.5%
-31.5% vs TC avg
Black line = Tech Center average estimate • Based on career data from 18 resolved cases

Office Action

§103
DETAILED ACTION Notice of Pre-AIA or AIA Status The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . This action is made non-final. Claims 1-13 and 15-30 are pending. Claims 1, 20 and 28 are independent claims. Response to Arguments Applicant’s arguments dated 9/8/2026, regarding the 35 U.S.C. 103 rejections of the previous office action have been fully considered. Due to the claim amendments, the scope of the claims has changed and new grounds of rejection are applied – see the updated rejection below. Note: the previous rejection refers to McClenny et al. (“Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanism”, published 7 Sep. 2020 and included in IDS), herein McClenny 1. The updated rejection refers to an updated version of this paper (“Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanism”, published 15 Sep. 2020), herein referred to as “McClenny 2”. A. Due to the claim amendments, the scope of the claims has changed, and new grounds of rejection are applied – see the updated rejection below. 1. Applicant's arguments with respect to the “GRU-hidden-state-derived attention weights” recited in the amended claims 1, 20 and 28 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. Due to the claim amendments, the scope of the claims has changed, and new grounds of rejection are applied – see the updated rejection below. 2. Applicant argues that focusing on training on difficult regions does not disclose automatically identifying shock locations. Examiner argues that McClenny 2 determines shocks, or difficult regions. Under BRI, assigning higher weights to difficult areas in a PDE solution (McClenny 2, pg. 2, describes: On the other hand, it has been observed that the baseline PINN has convergence and accuracy problems when solving “stiff” PDEs [11] with solutions that contain sharp and intricate space and time transitions [4, 12]. This is the case, for example, of the Allen-Cahn and Cahn-Hilliard equations of phase-field models [13]… To address this issue, various modifications of the baseline PINN algorithm have been proposed… The consensus has been that adaptation mechanisms are essential to make PINNs more stable and able to approximate well difficult regions of the solution… This paper introduces Self-Adaptive PINNs, a fundamentally new method to train PINNs adaptively, which uses trainable weights as a soft multiplicative mask reminiscent of the attention mechanism used in computer vision [14, 15]. The adaptation weights are trained concurrently with the network weights. As a result, initial, boundary or collocation points in difficult regions of the solution are automatically weighted more in the loss function, forcing the approximation to improve on those points – examiner underlined portions for emphasis) can be interpreted as detecting shocks in a PDE solution. The creation of fig. 5 in McClenny 2, pg. 9, based on the learned attention weights, specifically how the shock locations are highlighted with brighter points, can be interpreted under BRI as identifying a location of a shock. Applicant argues that McClenny does not disclose “producing a shock-location determination” from weights. Examiner argues that the amended claim 1 does not recite the “production” of a shock location determination (i.e., an explicit output step), only an automatic detection of shocks and attention weights “used” to identify a location of a shock. 3-5. Applicant's arguments with respect to the application of Bihorac to claim(s) 1-13 and 15-30 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. Due to the claim amendments, the scope of the claims has changed, and new grounds of rejection are applied – see the updated rejection below. B-C. Applicant' s arguments with respect to claim(s) 20-30 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. Due to the claim amendments, the scope of the claims has changed, and new grounds of rejection are applied – see the updated rejection below. D. Applicant' s arguments with respect to claim(s) 1-13 and 15-30 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. Due to the claim amendments, the scope of the claims has changed, and new grounds of rejection are applied – see the updated rejection below. E. Applicant argues that none of the references used in the previous office action, including McClenny 1, teaches an attention distribution that tracks a physical discontinuity across output coordinates. Examiner argues that both McClenny 2 and McClenny 1 disclose a system capable of tracking physical discontinuities, especially in view of the more recent version, McClenny 2, which includes pg. 9, fig. 5, in which the physical discontinuity can be “tracked” by viewing the learned weights. Due to the claim amendments, the scope of the claims has changed, and new grounds of rejection are applied – see the updated rejection below. Claim Rejections - 35 USC § 103 The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. Claims 1-4, 6-13, 15-25 and 27-30 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (“A GRU-based Encoder-Decoder Approach with Attention for Online Handwritten Mathematical Expression Recognition”, 2017), herein Zhang, in view of McClenny et al. (“Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanism”, published 15 Sep. 2020), herein McClenny 2, and Fraces et al. ("Physics Informed Deep Learning for Transport in Porous Media. Buckley Leverett Problem.", 2020), herein Fraces. Regarding claim 1, Zhang discloses: A computer system implementing a… attention-based neural network… comprising: at least one processor; and a memory storing computer instructions, the instructions implementing a… ANN (pg. 1, Inherently unlike traditional approaches, our model optimizes symbol segmentation automatically through its attention mechanism… Moreover, the encoder and the decoder are jointly trained – training a model implies that a computer is used, i.e., a processor and memory storing computer readable instructions), wherein, when the at least one processor executes the instructions, the …ANN is trained to learn a solution or model (Abstract, we present a novel end-to-end approach based on the encoder-decoder framework with the attention mechanism)… wherein the ANN includes a neural network (…NN) implementing a deep neural network (pg. 4, The encoder consists of 4 layers of bi-directional GRUs… The decoder is a single layer with 256 forward GRU units)… the …NN including an attention mechanism (Abstract, with the attention mechanism)… and wherein the …ANN includes a gated recurrent unit (GRU) architecture that determines which parts of input parameters… are relevant (pg. 3, Intuitively, for each predicted symbol from the decoder, not the entire input sequence is useful. Only a subset of adjacent trajectory points should mainly contribute to the computation of context vector ct at each time step t. Therefore, the decoder can adopt an attention mechanism to link to the related part of input sequence and then assign a higher weight to the corresponding annotation vector ai), wherein the attention mechanism computes attention weights based on hidden states of the GRU architecture (pg. 3, Therefore, the decoder can adopt an attention mechanism to link to the related part of input sequence and then assign a higher weight to the corresponding annotation vector – and – pg. 3, eqs. 15 and 16, which calculate attention weights in the form of context vector modification, based on an annotation vector (the output of the encoder), as described on pg. 3, Meanwhile, the bi-directional GRU encoder produces an annotation sequence A with a length L, also see related definition in eq. 9), wherein the attention mechanism… determines which parts of the input parameters… are relevant, based on the hidden states of the GRU architecture (pg. 3, Therefore, the decoder can adopt an attention mechanism to link to the related part of input sequence and then assign a higher weight to the corresponding annotation vector – and – pg. 3, eqs. 15 and 16, which calculate attention weights in the form of context vector modification, based on an annotation vector (the output of the encoder), as described on pg. 3, Meanwhile, the bi-directional GRU encoder produces an annotation sequence A with a length L, also see related definition in eq. 9); and wherein the GRU architecture processes a complete input sequence (pg. 1, the input two-dimensional ink trajectory information of handwritten expression is encoded to the high level representations via the stack of bi-directional gated recurrent unit based recurrent neural network (GRU-RNN). Then the decoder is implemented by a unidirectional GRU RNN with a coverage-based attention model [19]–[21]. The attention mechanism built into the decoder scans the entire input sequence and chooses the most relevant region to describe a segmented symbol or implicit spatial operator) over a spatial grid (pg. 1, the input handwritten mathematical expression consists of a sequence of trajectory points with a variable-length N: {[x1, y1,s1], [x2,y2,s2], ... , [xN,yN,sN]} where xi and yi are the xy-coordinates of the pen movements and si indicates which stroke the ith point belongs to), respective hidden states of the GRU architecture correspond to respective spatial coordinates of the grid (pg. 2, C, Meanwhile, the bi-directional GRU encoder produces an annotation sequence A with a length L. If there is no pooling in the bi-directional GRU encoder, L = N – the encoder may produce an annotation vector for each element of the input sequence (i.e., each annotation corresponds to respective spatial coordinates)), and the attention weights represent respective degrees of influence of encoded sequence components (pg. 1, The attention mechanism built into the decoder scans the entire input sequence and chooses the most relevant region to describe a segmented symbol or implicit spatial operator)… Zhang fails to explicitly disclose: physics-informed… (PIANN)… PIANN… PI… for at least one nonlinear partial differential equation (PDE)… PI… physics-informed… PI… and a transition zone detector… PI… configured to automatically detect shocks in the solution of the at least one nonlinear PDE… of the at least one nonlinear PDE… PI… automatically detects the shocks in the solution of the at least one nonlinear PDE and… of the at least one nonlinear PDE… on respective PDE solution values at the spatial coordinates and are used to identify a location of a shock. However, in the same field of endeavor, McClenny 2 discloses: physics-informed… (PIANN)… PIANN… PI… for at least one nonlinear partial differential equation (PDE)… PI… physics-informed… PI… and a transition zone detector… PI… configured to automatically detect shocks in the solution of the at least one nonlinear PDE… of the at least one nonlinear PDE… PI… automatically detects the shocks in the solution of the at least one nonlinear PDE (pg. 2, This paper introduces Self-Adaptive PINNs, a fundamentally new method to train PINNs adaptively, which uses trainable weights as a soft multiplicative mask reminiscent of the attention mechanism used in computer vision [14, 15]. The adaptation weights are trained concurrently with the network weights. As a result, initial, boundary or collocation points in difficult regions of the solution are automatically weighted more in the loss function, forcing the approximation to improve on those points – learned weights for the Burger equation solution are displayed on pg. 9, fig. 5 – where the transition zone, or shock, is emphasized by higher weights) and… of the at least one nonlinear PDE… on respective PDE solution values at the spatial coordinates and are used to identify a location of a shock (learned weights for the Burger equation solution are displayed on pg. 9, fig. 5 – the attention weights automatically “discover” the shock at x = 0.). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the attention-based GRU encoder-decoder architecture disclosed by Zhang for use in physical non-linear PDE solving as disclosed by McClenny 2 because Zhang and McClenny 2 both apply attention-based models to process spatial, sequential data, and McClenny 2 discloses that adding an “adaptation mechanism” (such as an attention-like mask) improves stability (McClenny 2, pg. 2, The consensus has been that adaptation mechanisms are essential to make PINNs more stable and able to approximate well difficult regions of the solution. This paper introduces self-adaptive PINNs, a simple solution to the adaptation problem for solving partial difference equations (PDEs), which uses trainable weights as a soft multiplicative mask reminiscent of the attention mechanism used in computer vision. The adaptation weights are trained concurrently with the network weights. As a result, initial, boundary or collocation points in difficult regions of the solution are automatically weighted more in the loss function, forcing the approximation to improve on those points). Zhang in view of McClenny 2 fails to explicitly disclose: respecting one or more physical constraints… However, in the same field of endeavor, Fraces discloses: respecting one or more physical constraints (pg. 7, eq. 10, uses a term that penalizes model output that varies from the governing differential equation – and – pg. 7, The residual of the PDE defines the physics constraint to the problem)… Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize physical constraints within a PDE-solving PINN as disclosed by Fraces in the system disclosed by Zhang in view of McClenny 2 to accurately model physical problems (pg. 19, This method is a new way to produce high fidelity surrogates that honor governing laws. It can be perceived as a data assimilation under physics constraints for virtually any type of modeling. The computation times are order of magnitudes smaller for the neural network approach than they are for the finite volume approach and we suspect that the gain grows as the size and number of discrete elements gets larger). Regarding claim 2, Zhang further discloses: The system of claim 1, wherein the PIANN includes an encoder and a decoder (Abstract, In this study, we present a novel end-to-end approach based on the encoder-decoder framework with the attention mechanism for online handwritten mathematical expression recognition). Regarding claim 3, Zhang further discloses: The system of claim 2, wherein the encoder is used to map an encoder input to an encoder output in an embedding space, and wherein the decoder is used to map a decoder input in the embedding space to a decoder output (pg. 3, As shown in Fig. 1, the decoder generates a corresponding LaTeX sequence of the input handwritten mathematical expression. The output sequence Y is encoded as a sequence of one-shot vectors… Meanwhile, the bi-directional GRU encoder produces an annotation sequence A with a length L – one input to the encoder is the input handwriting sequence feature vector, as discussed on pg. 2, in section A – the encoder outputs an annotation vector which (after applying attention weights becomes the context vector, see eq. 17 on pg. 3) the decoder takes as one of its inputs – pg. 3, The GRU decoder also takes the… context vector ct as input). Regarding claim 4, Zhang further discloses: The system of claim 3, wherein the encoder input size and decoder input size are equal (pg. 2, Suppose the input handwritten mathematical expression consists of a sequence of trajectory points with a variable length N – the encoder takes a sequence vector as one of its inputs, and produces an annotation vector that can be the same size as the sequence: pg. 2, Meanwhile, the bi-directional GRU encoder produces an annotation sequence A with a length L. If there is no pooling in the bi-directional GRU encoder, L = N – the encoder output is provided as an input to the decoder in the form of a context vector – pg. 3, Given the context vector ct, we utilize unidirectional GRU to produce the LaTeX sequences symbol by symbol – the context vector is generated based on the annotation vector as described in pg. 3, eq.17. These two inputs have the same size of N in an embodiment). Regarding claim 6, Zhang further discloses: The system of claim 1, wherein the PIANN includes a plurality of RNN units (pg. 2, fig. 1, multiple GRUs). Regarding claim 7, Zhang further discloses: The system of claim 6, wherein the plurality of RNN units include at least one of: the GRU, or at least one long short-term memory (LSTM) (pg. 2, fig. 1, multiple GRUs). Regarding claim 8, Zhang further discloses: The system of claim 6, wherein the plurality of RNN units include a plurality of GRUs (pg. 2, fig. 1, multiple GRUs). Regarding claim 9, Zhang further discloses: The system of claim 6, wherein the plurality of RNN units include a first plurality of GRUs and a second plurality of GRUs, and wherein the first plurality of GRUs are used as a part of an encoder of the PIANN and the second plurality of GRUs are used as a part of a decoder of the PIANN (pg. 2, fig. 1, multiple GRUs, including an encoder (2) and decoder (3), both made of GRUs). Regarding claim 10, Zhang further discloses: The system of claim 9, wherein the transition zone detector includes the attention mechanism implemented as an attention layer introduced between the first plurality of GRUs and the second plurality of GRUs (pg. 2, fig. 1, multiple GRUs, including an encoder (2) and decoder (3), the attention mechanism referred to in (3) and pg. 3, Only a subset of adjacent trajectory points should mainly contribute to the computation of context vector ct at each time step t. Therefore, the decoder can adopt an attention mechanism to link to the related part of input sequence and then assign a higher weight to the corresponding annotation vector ai). Regarding claim 11, Zhang further discloses: The system of claim 10, wherein the attention layer is used to calculate a context vector based on encoder hidden states corresponding to the first plurality of GRUs (pg. 2, fig. 1, multiple GRUs, including an encoder (2) and decoder (3), the attention mechanism referred to in (3) – and – pg. 3, Only a subset of adjacent trajectory points should mainly contribute to the computation of context vector ct at each time step t. Therefore, the decoder can adopt an attention mechanism to link to the related part of input sequence and then assign a higher weight to the corresponding annotation vector ai – see eqns. 15-17, which utilize the annotation vector (encoder hidden states) to calculate attention weights). Regarding claim 12, Zhang further discloses: The system of claim 11, wherein the context vector is calculated based on attention weights that are determined based on the encoder hidden states corresponding to the first plurality of GRUs (pg. 2, fig. 1, multiple GRUs, including an encoder (2) and decoder (3), the attention mechanism referred to in (3) and pg. 3, Only a subset of adjacent trajectory points should mainly contribute to the computation of context vector ct at each time step t. Therefore, the decoder can adopt an attention mechanism to link to the related part of input sequence and then assign a higher weight to the corresponding annotation vector ai – see eqns. 15-17, which utilize the annotation vector (encoder hidden states) to calculate attention weights). Regarding claim 13, Zhang further discloses: The system of claim 12, wherein the context vector is used as input into at least one of the second plurality of GRUs (pg. 3, The GRU decoder also takes the previous target symbol yt−1 and the context vector ct as input). Regarding claim 15, Zhang in view of McClenny 2 fails to explicitly disclose: The system of claim 1, wherein the non-linear PDE is a hyperbolic PDE. However, in the same field of endeavor, Fraces discloses: wherein the non-linear PDE is a hyperbolic PDE (pg. 16, in section 5.4 where we are interested in the problem of gravity segregation where the added physics acts as another closure condition for the hyperbolic problem). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to solve a hyperbolic PDE as disclosed by Fraces using the system disclosed by Zhang in view of McClenny 2 to accurately model specific physics-based problems (pg. 19, This method is a new way to produce high fidelity surrogates that honor governing laws). Regarding claim 16, Zhang in view of McClenny 2 fails to explicitly disclose: The system of claim 1, wherein the PIANN includes an automatic differentiator for producing a differentiation output, and wherein the differentiation output is used for training the PIANN in order to update one or more parameters or weights of the PIANN. However, in the same field of endeavor, Fraces discloses: wherein the PIANN includes an automatic differentiator for producing a differentiation output, and wherein the differentiation output is used for training the PIANN in order to update one or more parameters or weights of the PIANN (pg. 1, Derivatives of the estimated physical quantities are computed using deep learning libraries that feature automatic differentiation algorithms. This strategy conditions the neural network to honor physical principles – Fraces refers to automatic differentiation as “AD”). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use an automatic differentiator in the training process as disclosed by Fraces in the PIANN system disclosed by Zhang in view of McClenny 2 in order to train models faster (pg. 4, Open source software libraries for AD such as Tensorflow ([16]) allow the creation and compilation of computational graphs that are then used to differentiate quantities of interest at observation points. Once computed, the graph is used through the entire optimization process (whether for training a neural network or for computing the residual of a PDE). This static implementation results in significant savings in terms of computational time). Regarding claim 17, Zhang in view of Fraces fails to explicitly disclose: The system of claim 16, wherein the one or more parameters or weights of the PIANN include one or more transition zone detector weights or parameters. However, in the same field of endeavor, McClenny 2 discloses: wherein the one or more parameters or weights of the PIANN include one or more transition zone detector weights or parameters (pg. 9, fig. 5 – light colored weights focused around x=0 can be interpreted under BRI as “transition zone detector weights”). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize transition zone detector weights as disclosed by McClenny 2 in the system disclosed by Zhang in view of Fraces to improve solver performance (pg. 2,The consensus has been that adaptation mechanisms are essential to make PINNs more stable and able to approximate well difficult regions of the solution… adaptation weights are trained concurrently with the network weights. As a result, initial, boundary or collocation points in difficult regions of the solution are automatically weighted more in the loss function, forcing the approximation to improve on those points). Regarding claim 18, Zhang in view of McClenny 2 further discloses: The system of claim 17, wherein the transition zone detector is an attention mechanism and the transition zone detector weights or parameters are attention weights of the attention mechanism (Zhang, pg. 3, Only a subset of adjacent trajectory points should mainly contribute to the computation of context vector ct at each time step t. Therefore, the decoder can adopt an attention mechanism to link to the related part of input sequence and then assign a higher weight to the corresponding annotation vector ai). Regarding claim 19, Zhang further discloses: The PIANN system of claim 1, wherein the PIANN is structured as a seq-to-seq RNN (pg. 2, Suppose the input handwritten mathematical expression consists of a sequence of trajectory points – and – pg. 2, fig. 1, 4, LaTeX sequence – i.e., the network (made of GRUs) takes in a sequence and outputs a sequence). Regarding claim 20, Zhang discloses: A computer system implementing a ANN, the computer system comprising: at least one processor; and a memory storing computer instructions (pg. 1, Inherently unlike traditional approaches, our model optimizes symbol segmentation automatically through its attention mechanism… Moreover, the encoder and the decoder are jointly trained – training a model implies that a computer is used, i.e., a processor and memory storing computer readable instructions), wherein, when the at least one processor executes the computer instructions… wherein the ANN includes a… neural network … with an attention mechanism (Abstract, we present a novel end-to-end approach based on the encoder-decoder framework with the attention mechanism) introduced into an embedding space of the NN between an encoder and a decoder of the NN, and wherein the ANN includes a gated recurrent unit (GRU) architecture that determines which parts of input parameters… are relevant (pg. 2, fig. 1, multiple GRUs, including an encoder (2) and decoder (3), the attention mechanism referred to in (3) and pg. 3, Only a subset of adjacent trajectory points should mainly contribute to the computation of context vector ct at each time step t. Therefore, the decoder can adopt an attention mechanism to link to the related part of input sequence and then assign a higher weight to the corresponding annotation vector ai) wherein the attention mechanism computes attention weights based on hidden states of the GRU architecture (pg. 3, Therefore, the decoder can adopt an attention mechanism to link to the related part of input sequence and then assign a higher weight to the corresponding annotation vector – and – pg. 3, eqs. 15 and 16, which calculate attention weights in the form of context vector modification, based on an annotation vector (the output of the encoder), as described on pg. 3, Meanwhile, the bi-directional GRU encoder produces an annotation sequence A with a length L, also see related definition in eq. 9), and wherein the attention mechanism … determines which parts of the input parameters… are relevant, based on the hidden states of the GRU architecture (pg. 3, Therefore, the decoder can adopt an attention mechanism to link to the related part of input sequence and then assign a higher weight to the corresponding annotation vector – and – pg. 3, eqs. 15 and 16, which calculate attention weights in the form of context vector modification, based on an annotation vector (the output of the encoder), as described on pg. 3: Meanwhile, the bi-directional GRU encoder produces an annotation sequence A with a length L. Also see related definition in eq. 9); and wherein the GRU architecture processes a complete input sequence (pg. 1, the input two-dimensional ink trajectory information of handwritten expression is encoded to the high level representations via the stack of bi-directional gated recurrent unit based recurrent neural network (GRU-RNN). Then the decoder is implemented by a unidirectional GRU RNN with a coverage-based attention model [19]–[21]. The attention mechanism built into the decoder scans the entire input sequence and chooses the most relevant region to describe a segmented symbol or implicit spatial operator) over a spatial grid (pg. 1, the input handwritten mathematical expression consists of a sequence of trajectory points with a variable-length N: {[x1, y1,s1], [x2,y2,s2], ... , [xN,yN,sN]} where xi and yi are the xy-coordinates of the pen movements and si indicates which stroke the ith point belongs to), hidden states of the GRU architecture respectively correspond to respective spatial coordinates of the grid (pg. 2, C, Meanwhile, the bi-directional GRU encoder produces an annotation sequence A with a length L. If there is no pooling in the bi-directional GRU encoder, L = N – the encoder may produce an annotation vector for each element of the input sequence (i.e., each annotation corresponds to respective spatial coordinates)), and the attention weights represent respective degrees of influence of encoded sequence components on respective solution values… at the spatial coordinates (pg. 1, The attention mechanism built into the decoder scans the entire input sequence and chooses the most relevant region to describe a segmented symbol or implicit spatial operator)… Zhang fails to disclose: PIANN… PIANN… physics-informed… (PINN)… PI… PI… the attention mechanism configured to automatically detect shocks in the solution of at least one… PDE… PI… of the at least one… PDE… automatically detects the shocks in the solution of the at least one… PDE and… of the at least one… PDE… of the at least one… PDE… and are used to identify a location of a shock. However, in the same field of endeavor, McClenny 2 discloses: PIANN… PIANN… physics-informed… (PINN)… PI… PI… the attention mechanism configured to automatically detect shocks in the solution of at least one… PDE… PI… of the at least one… PDE… automatically detects the shocks in the solution of the at least one… PDE (pg. 2, This paper introduces Self-Adaptive PINNs, a fundamentally new method to train PINNs adaptively, which uses trainable weights as a soft multiplicative mask reminiscent of the attention mechanism used in computer vision [14, 15]. The adaptation weights are trained concurrently with the network weights. As a result, initial, boundary or collocation points in difficult regions of the solution are automatically weighted more in the loss function, forcing the approximation to improve on those points – learned weights for the Burger equation solution are displayed on pg. 9, fig. 5 – where the transition zone, or shock, is emphasized by higher weights) and… of the at least one… PDE… of the at least one… PDE… and are used to identify a location of a shock (learned weights for the Burger equation solution are displayed on pg. 9, fig. 5 – the attention weights automatically discover the shock at x = 0.). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the attention-based GRU encoder-decoder architecture disclosed by Zhang for use in physical non-linear PDE solving as disclosed by McClenny 2 because Zhang and McClenny 2 both apply attention-based models to process spatial, sequential data, and McClenny 2 discloses that adding an “adaptation mechanism” (such as an attention-like mask) improves stability (McClenny 2, pg. 2, The consensus has been that adaptation mechanisms are essential to make PINNs more stable and able to approximate well difficult regions of the solution. This paper introduces self-adaptive PINNs, a simple solution to the adaptation problem for solving partial difference equations (PDEs), which uses trainable weights as a soft multiplicative mask reminiscent of the attention mechanism used in computer vision). Zhang in view of McClenny 2 fails to disclose: the PIANN is used to generate a surrogate model for use in generating a simulation output… hyperbolic… hyperbolic… hyperbolic... hyperbolic… hyperbolic… However, in the same field of endeavor, Fraces discloses: the PIANN is used to generate a surrogate model for use in generating a simulation output (pg. 19, This method is a new way to produce high fidelity surrogates that honor governing laws. It can be perceived as a data assimilation under physics constraints for virtually any type of modeling)… hyperbolic… hyperbolic… hyperbolic... hyperbolic… hyperbolic (pg. 16, in section 5.4 where we are interested in the problem of gravity segregation where the added physics acts as another closure condition for the hyperbolic problem)… Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to solve a hyperbolic PDE with a PINN-derived surrogate model as disclosed by Fraces using the system disclosed by Zhang in view of McClenny 2 to accurately and efficiently model specific physics-based problems (pg. 19, a new way to produce high fidelity surrogates that honor governing laws…The computation times are order of magnitudes smaller for the neural network approach than they are for the finite volume approach and we suspect that the gain grows as the size and number of discrete elements gets larger). Regarding claim 21, Zhang further discloses: The system of claim 20, wherein the attention mechanism is one of: a soft attention mechanism, or a hard attention mechanism (pg. 3, Therefore, the decoder can adopt an attention mechanism to link to the related part of input sequence and then assign a higher weight to the corresponding annotation vector ai – Zhang’s approach is a soft attention mechanism, i.e., weights are not just 1 or 0). Regarding claim 22, Zhang further discloses: The system of claim 20, wherein the PIANN includes an encoder-decoder recurrent neural network (RNN) configuration having a plurality of RNN units, and wherein the plurality of RNN units are used to generate an encoder output in the embedding space and/or to generate a decoder input in the embedding space (pg. 3, As shown in Fig. 1, the decoder generates a corresponding LaTeX sequence of the input handwritten mathematical expression. The output sequence Y is encoded as a sequence of one-shot vectors… Meanwhile, the bi-directional GRU encoder produces an annotation sequence A with a length L – also see fig. 1 for a plurality of GRUs). Regarding claim 23, Zhang in view of McClenny 2 fails to disclose: The system of claim 20, wherein the PIANN includes an automatic differentiator for producing a differentiation output, and wherein the differentiation output is used by a physics-informed learning unit for physics-informed learning or training of the PINN. However, in the same field of endeavor, Fraces discloses: wherein the PIANN includes an automatic differentiator for producing a differentiation output, and wherein the differentiation output is used by a physics-informed learning unit for physics-informed learning or training of the PINN (pg. 1, Derivatives of the estimated physical quantities are computed using deep learning libraries that feature automatic differentiation algorithms. This strategy conditions the neural network to honor physical principles – Fraces refers to automatic differentiation as “AD”). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use an automatic differentiator in the training process as disclosed by Fraces in the PIANN system disclosed by Zhang in view of McClenny 2 in order to train models faster (pg. 4, Open source software libraries for AD such as Tensorflow ([16]) allow the creation and compilation of computational graphs that are then used to differentiate quantities of interest at observation points. Once computed, the graph is used through the entire optimization process (whether for training a neural network or for computing the residual of a PDE). This static implementation results in significant savings in terms of computational time). Regarding claim 24, Zhang in view of McClenny 2 fails to disclose: The system of claim 23, wherein the PIANN includes a physics-informed learning unit that uses a physical loss function. However, in the same field of endeavor, Fraces explicitly discloses: wherein the PIANN includes a physics-informed learning unit that uses a physical loss function (pg. 7, equation 10 is a loss function formulated based on the Burger’s equation, also see pg. 8 for residual loss based on the Buckley-Leverett equation – and – pg. 7, The residual of the PDE defines the physics constraint to the problem). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use a physics-informed learning unit with a physical loss function as disclosed by Fraces in the system disclosed by Zhang in view of McClenny 2 to accurately model specific physical problems (pg. 19, to produce high fidelity surrogates that honor governing laws). Regarding claim 25, Zhang in view of McClenny 2 fails to disclose: The system of claim 24, wherein the physical loss function is formulated based on a Buckley-Leverett (BL) equation. However, in the same field of endeavor, Fraces discloses: wherein the physical loss function is formulated based on a Buckley-Leverett (BL) equation (pg. 7, equation 10 is a loss function formulated based on the Burger’s equation, and a similar regression loss is used to model the Buckley-Leverett equation on pg. 8, and also pg. 11, eq. 16). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the Buckley-Leverett equation in the loss function as disclosed by Fraces in the system disclosed by Zhang in view of McClenny 2 to accurately model specific physical problems (pg. 19, to produce high fidelity surrogates that honor governing laws). Regarding claim 27, Zhang in view of McClenny 2 fails to explicitly disclose: The system of claim 24, wherein the physics-informed learning unit enforces one or more initial conditions and one or more boundary conditions representing or selected in accordance with one or more physical constraints. However, in the same field of endeavor, Fraces discloses: wherein the physics-informed learning unit enforces one or more initial conditions and one or more boundary conditions representing or selected in accordance with one or more physical constraints (pg. 10, The inference problem consists in finding the forward solution of a PDE considering a limited set of observations located at the boundaries and initial conditions – Fraces discusses this arrangement on pg. 10 (and after) as well: assessing the parameters of a Buckley-Leverett equation from data can be achieved provided that enough training data is available. We notice that the error on the PDE tends to increase when a large portion of the dataset is not used for training. This problem becomes critical in cases where only boundary data and initial are used for training (Inference problem). Section 4.3 addresses this issue along with the solutions proposed to alleviate it). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to enforce at least one boundary and one initial condition as disclosed by Fraces in the system disclosed by Zhang in view of McClenny 2 to ensure unique solutions exist and can be discovered by the model (pg. 10, Note that this is the minimum required number of auxiliary conditions to establish a unique solution to a first order PDE with two variables). Regarding claim 28, Zhang discloses: A computer system implementing a deep neural network (DNN) comprising: at least one processor; and a memory storing computer instructions, wherein, when the at least one processor executes the computer instructions, the DNN architecture is used to generate a DNN output (pg. 1, Inherently unlike traditional approaches, our model optimizes symbol segmentation automatically through its attention mechanism… Moreover, the encoder and the decoder are jointly trained – training a model implies that a computer is used, i.e., a processor and memory storing computer readable instructions); wherein the DNN includes an encoder and a decoder coupled together in an embedding space (pg. 4, The encoder consists of 4 layers of bi-directional GRUs… The decoder is a single layer with 256 forward GRU units – multiple hidden layers, i.e., a deep network) through an attention layer including an attention mechanism (pg. 3, Therefore, the decoder can adopt an attention mechanism to link to the related part of input sequence and then assign a higher weight to the corresponding annotation vector – and – pg. 3, eqs. 15 and 16, which calculate attention weights in the form of context vector modification, based on an annotation vector (the output of the encoder), as described on pg. 3, Meanwhile, the bi-directional GRU encoder produces an annotation sequence A with a length L – also see related definition in eq. 9); wherein the encoder comprises a first gated recurrent unit (GRU) architecture and the decoder comprises a second gated recurrent unit (GRU) architecture (pg. 2, fig. 1, multiple GRUs, including an encoder (2) and decoder (3) – both of which are implemented by multiple GRUs), and wherein the attention mechanism computes attention weights based on hidden states of the first GRU architecture (pg. 3, Therefore, the decoder can adopt an attention mechanism to link to the related part of input sequence and then assign a higher weight to the corresponding annotation vector – and – pg. 3, eqs. 15 and 16, which calculate attention weights in the form of context vector modification, based on an annotation vector (the output of the encoder), as described on pg. 3, Meanwhile, the bi-directional GRU encoder produces an annotation sequence A with a length L, also see related definition in eq. 9) ; and wherein the attention layer is configured to: use a full sequence of input data to determine which parts of input parameters… are relevant (pg. 1, the input two-dimensional ink trajectory information of handwritten expression is encoded to the high level representations via the stack of bi-directional gated recurrent unit based recurrent neural network (GRU-RNN). Then the decoder is implemented by a unidirectional GRU RNN with a coverage-based attention model [19]–[21]. The attention mechanism built into the decoder scans the entire input sequence and chooses the most relevant region to describe a segmented symbol or implicit spatial operator), based on full sequences over a spatial/temporal grid (pg. 1, the input handwritten mathematical expression consists of a sequence of trajectory points with a variable-length N: {[x1, y1,s1], [x2,y2,s2], ... , [xN,yN,sN]} where xi and yi are the xy-coordinates of the pen movements and si indicates which stroke the ith point belongs to)… and wherein the first GRU architecture comprises an encoder sequence of GRU blocks, each GRU block of the encoder sequence corresponding to a respective spatial coordinate of the spatial/temporal grid (pg. 2, fig. 1, input sequence is fed one by one into the encoder blocks, also, the input sequence elements are associated with spatial coordinates – spatial/temporal grid is interpreted under BRI to mean a spatial and/or temporal grid, i.e., a spatial grid); wherein the second GRU architecture comprises a decoder sequence of GRU blocks that generates respective solution values… for the spatial coordinates, each decoder GRU block generating a respective solution value using a solution value for a preceding spatial coordinate, a context vector calculated as a weighted sum of hidden states of the encoder sequence, and a preceding hidden state of the decoder sequence (pg. 3, The GRU decoder also takes the previous target symbol yt−1 and the context vector ct as input, and employs a single unidirectional GRU layer to calculate the hidden state st – and – definition of context vector, eq. 17, which is an attention weighted sum of the encoder sequence. The previous hidden state of the decoder (st-1) is seen in eqs. 11-14, defining the calculation of the current decoder hidden state – also see eq. 10); and wherein the attention weights represent respective degrees of influence of encoded sequence components on the respective solution value (pg. 1, Then the decoder is implemented by a unidirectional GRU RNN with a coverage-based attention model [19]–[21]. The attention mechanism built into the decoder scans the entire input sequence and chooses the most relevant region to describe a segmented symbol or implicit spatial operator). Zhang fails to disclose: of a… PDE… and automatically identify shock locations in the solution of at least one… PDE… of the… PDE… and are used to identify a location of a shock. However, in the same field of endeavor, McClenny 2 discloses: of a… PDE… and automatically identify shock locations in the solution of at least one… PDE… of the… PDE… and are used to identify a location of a shock (pg. 2, This paper introduces Self-Adaptive PINNs, a fundamentally new method to train PINNs adaptively, which uses trainable weights as a soft multiplicative mask reminiscent of the attention mechanism used in computer vision [14, 15]. The adaptation weights are trained concurrently with the network weights. As a result, initial, boundary or collocation points in difficult regions of the solution are automatically weighted more in the loss function, forcing the approximation to improve on those points – learned weights for the Burger equation solution are displayed on pg. 9, fig. 5 – where the transition zone, or shock, is emphasized by higher weights). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the attention-based GRU encoder-decoder architecture disclosed by Zhang for use in PDE solving as disclosed by McClenny 2 because Zhang and McClenny 2 both apply attention-based models to process spatial, sequential data, and McClenny 2 discloses that adding an “adaptation mechanism” (such as an attention-like mask) improves stability (McClenny 2, pg. 2, The consensus has been that adaptation mechanisms are essential to make PINNs more stable and able to approximate well difficult regions of the solution. This paper introduces self-adaptive PINNs, a simple solution to the adaptation problem for solving partial difference equations (PDEs), which uses trainable weights as a soft multiplicative mask reminiscent of the attention mechanism used in computer vision). Zhang in view of McClenny 2 fails to explicitly disclose: that respects one or more predetermined physical constraints… hyperbolic… hyperbolic… hyperbolic… However, in the same field of endeavor, Fraces discloses: that respects one or more predetermined physical constraints (pg. 7, eq. 10, uses a term that penalizes model output that varies from the governing differential equation – and – pg. 7, The residual of the PDE defines the physics constraint to the problem)… hyperbolic… hyperbolic… hyperbolic (pg. 16, in section 5.4 where we are interested in the problem of gravity segregation where the added physics acts as another closure condition for the hyperbolic problem)… Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to solve a hyperbolic PDE with a model that respects physical constraints as disclosed by Fraces using the system disclosed by Zhang in view of McClenny 2 to accurately and efficiently model specific physics-based problems (Section 5.4, a new way to produce high fidelity surrogates that honor governing laws… The computation times are order of magnitudes smaller for the neural network approach than they are for the finite volume approach and we suspect that the gain grows as the size and number of discrete elements gets larger). Regarding claim 29, Zhang further discloses: The system of claim 28, wherein the attention layer is coupled to a plurality of recurrent neural network (RNN) units of the encoder and coupled to a plurality of RNN units of the decoder (pg. 2, fig. 1, multiple GRUs, including an encoder (2) and decoder (3), and attention layer, described on pg. 3, Only a subset of adjacent trajectory points should mainly contribute to the computation of context vector ct at each time step t. Therefore, the decoder can adopt an attention mechanism to link to the related part of input sequence and then assign a higher weight to the corresponding annotation vector ai). Regarding claim 30, Zhang in view of McClenny 2 fails to explicitly disclose: The system of claim 29, wherein one or more weights or parameters of the encoder, the decoder, and/or the attention layer are trained using a physics-informed learning unit that uses a physical loss function and that respects the one or more predetermined physical constraints. However, in the same field of endeavor, Fraces discloses: wherein one or more weights or parameters of a model are trained using a physics-informed learning unit that uses a physical loss function and that respects the one or more predetermined physical constraints (pg. 7, equation 10 is a loss function formulated based on the Burger’s equation – pg. 7, The residual of the PDE defines the physics constraint to the problem – also see pg. 8 for a similar metric, error based on PDE residual but for the Buckley-Leverett equation instead). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the Buckley-Leverett equation in the loss function as disclosed by Fraces in the attended encoder/decoder system disclosed by Zhang in view of McClenny 2 to accurately model specific physical problems (pg. 19, to produce high fidelity surrogates that honor governing laws). Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of McClenny 2 and Fraces as applied to claim 3 above, and further in view of O’ Shea (US 20180322388 A1). Regarding claim 5, Zhang in view of McClenny 2 and Fraces fails to disclose: The system of claim 3, wherein a linear transition layer is introduced in the embedding space between the encoder and the decoder. However, in the same field of endeavor, O’ Shea discloses: wherein a linear transition layer is introduced in the embedding space between the encoder and the decoder (¶54, In some implementations, a linear regression layer may be implemented on the output of the encoder 202). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to add a linear layer between the encoder and decoder as disclosed by O’Shea in the PIANN system disclosed by Zhang in view of McClenny 2 and Fraces to better handle problem complexity (¶26, By implementing machine-learning networks that may be trained to learn suitable encoding and decoding techniques for different types of communication media, techniques disclosed herein offer various advantages, such as improved power, resiliency, and complexity advantages). Claim 26 is rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of McClenny 2 and Fraces as applied to claim 24 above, and further in view of Shahkarami et al. (US 20200242497 A1), herein Shahkarami. Regarding claim 26, Zhang in view of McClenny 2 and Fraces fails to disclose: The system of claim 20, wherein the surrogate model is an oil and gas reservoir model. However, in the same field of endeavor, Shahkarami discloses: wherein the surrogate model is an oil and gas reservoir model (¶16, The surrogate model is a simplified mathematical model of the reservoir that provides the same output or close to the same output obtained from the reservoir simulations – and – ¶19, a reservoir of a hydrocarbon (liquid and/or gas or a combination)). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to generate a surrogate model for an oil and gas reservoir model as disclosed by Shahkarami with the system disclosed by Zhang in view of McClenny 2 and Fraces to reduce computation time (¶16, In general, the surrogate model uses much less computational time to run than the dynamic flow simulations using the detailed reservoir model. For example, while it may take several hours to run the reservoir simulator model, the surrogate model can be run in less than an hour – and – ¶6, Reservoir simulation is utilized to model a well completion plan and design before implementing it in a field. Generating a well completion plan may require many reservoir simulations using a reservoir simulator in order to optimize the number of flow control devices required and their locations and settings, and number and location of packers. Unfortunately, each reservoir simulation has a high computational cost and it may take several hours to days to perform one simulation). Conclusion Any inquiry concerning this communication or earlier communications from the examiner should be directed to HARRISON CHAN YOUNG KIM whose telephone number is (571)272-0713. The examiner can normally be reached Monday - Friday 10:00 am - 6:00 pm. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Cesar Paula can be reached on (571) 272-4128. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /HARRISON C KIM/Examiner, Art Unit 2145 /CESAR B PAULA/Supervisory Patent Examiner, Art Unit 2145
Read full office action

Prosecution Timeline

Show 2 earlier events
Nov 06, 2025
Response Filed
Nov 10, 2025
Interview Requested
Dec 01, 2025
Examiner Interview Summary
Mar 05, 2026
Final Rejection mailed — §103
Sep 08, 2026
Request for Continued Examination
Sep 08, 2026
Response after Non-Final Action
Sep 11, 2026
Response after Non-Final Action
Oct 05, 2026
Non-Final Rejection mailed — §103 (current)

Precedent Cases

Applications granted by this same examiner with similar technology

Patent 12786726
KNOWLEDGE-BASED AUTHENTICATION LEVERAGING MOBILE DEVICES
4y 8m to grant Granted Nov 03, 2026
Patent 12783877
ARTIFICIAL NEURAL NETWORK FOR SPARSE DATA PROCESSING
3y 9m to grant Granted Nov 03, 2026
Patent 12711372
TRAINING GIANT NEURAL NETWORKS USING PIPELINE PARALLELISM
4y 7m to grant Granted Aug 18, 2026
Patent 12674702
SOUND EVENT EARLY DETECTION
3y 10m to grant Granted Jul 07, 2026
Patent 12608607
METHOD FOR PREDICTING REMAINING USEFUL LIFE OF RAILWAY TRAIN BEARING BASED ON CAN-LSTM
3y 5m to grant Granted Apr 21, 2026
Study what changed to get past this examiner. Based on 5 most recent grants.

Strategy Recommendation AI-generated — please review before filing

Get a prosecution strategy drawn from examiner precedents, rejection analysis, and claim mapping.
Typically takes 5-10 seconds — AI-generated, attorney review required before filing

Prosecution Projections

3-4
Expected OA Rounds
61%
Grant Probability
94%
With Interview (+33.3%)
4y 0m (~0m remaining)
Median Time to Grant
High
PTA Risk
Based on 18 resolved cases by this examiner. Grant probability derived from career allowance rate.

Sign in with your work email

Enter your email to receive a magic link. No password needed.

Personal email addresses (Gmail, Yahoo, etc.) are not accepted.

Free tier: 3 strategy analyses per month