DETAILED ACTION
This non-final office action is responsive to application 18/404,113 as submitted 04 Jan. 2024.
Claim status is currently pending and under examination for claims 1-20 of which independent claims are 1, 9 and 16.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
As required by MPEP 609(c), the applicant’s submissions of the Information Disclosure Statement dated 05/09/25 – 05/06/26 is acknowledged by the examiner and the cited references have been considered in the examination of the claims now pending. As required by MPEP 609 C(2), a copy of the PTOL-1449 initialed and dated by the examiner is attached to the instant office action.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b), as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, regards as the invention. Particularly, antecedent basis is insufficient in two cases where independent claims 1, 9 and 16 recite the limitation "the trained machine learning model" in limitation train and further dependent claims 4, 12 and 18 recite “the machine learning model” in single limitation. The reason for deficiency is that the claims also recite a neural network which is a machine learning model as well under the broadest reasonable interpretation. Therefore, it is uncertain whether “the” machine learning model may be referring to one of the other neural networks or to a different type of model. As such, clarity is lacking and antecedent basis is insufficient for the claims. For purposes of further examination, the machine learning model is broadly interpreted to comprise any of machine learning models or neural networks. Claims depending from the affected language fail to further remedy or clarify the matter. Accordingly, claims 1-20 are rejected as indefinite under 35 U.S.C. 112(b) lacking antecedent basis.
Claim Rejection – 35 USC § 101
7. 35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
8. Claims 16-20 are rejected under 35 U.S.C. 101 for being directed to non-statutory subject matter. Claim 16 is drawn to a “computer-readable storage medium” which may include transitory forms of signal transmission as a signal per-se. It fails to assert that the medium is non-transitory or tangible to preclude such matter under the broadest reasonable interpretation. When read in light of specification [0111] the medium refers to media with examples. However, examples or embodiments of such elements do not strictly limit or otherwise disavow the inclusion of transitory signals. Therefore, the claims may encompass forms of signal transmission that are transitory in nature which is statutorily ineligible subject matter. Accordingly, claim 16 is rejected as being drawn to non-statutory subject matter. Claims 17-20 recite this medium without further specifying the medium as non-transitory. Accordingly, claims 16-20 are rejected under 35 U.S.C. 101 as being drawn to non-statutory subject matter. The issue may be remediated by reciting the medium as “non-transitory” to overcome the issue.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 3-4, 8-9, 11-12, 15-16 and 18 are rejected under 35 U.S.C. 103 as unpatentable over:
Meidani et al., “SNIP: Bridging Mathematical Symbolic and Numeric Realms with Unified Pre-Training” (arXiv: 2310.02227v2, Carnegie Mellon), in view of
Holt et al., “Deep Generative Symbolic Regression” hereinafter Holt (arXiv: 2401.00282v1, Univ. Cambridge), and further in view of
Tran et Nguyen, “Modeling Power Systems Dynamics with Symbolic Physics-Informed Neural Networks” hereinafter Tran (arXiv: 2311.06580v1).
With respect to claim 1, Meidani teaches:
A system for training a model with a learned loss function {Meidani Figs 1 and 4 illustrate SNIP for Symbolic Regression shows training of encoder-decoder model with loss e.g. Eq.1, the system implemented with GPU and memory [P.18 App. B.2]}, the system comprising:
a processor {Meidani [P.18 App. B.2] “we utilize 4 GPUs”}; and
a memory device that stores program code structured to cause the processor {Meidani see [P.18 App. B.2] “48GB of memory” for code Algorithm 1 [P.24]} to:
generate a first trained neural network based on application of a first loss function thereto, the first trained neural network comprising a plurality of layers {Meidani Fig 4(a) shows training encoder-decoder with loss function and describes generative transformer models [P.7-8 Sect. 5.1] introduced [P.3-6 Sect.3] detailing loss function and layers. The implementation specifies [P.17 App. B.1] “feedforward neural network” shown as FFN in Fig 1, loss function comprises Eq.1 [P.5]};
extract a set of values of one of the plurality of layers of the first trained neural network {Meidani Fig 1 “Numeric Encoder” described [P.4 Sect. 3.1] integrates an “embedder” FFN (feed-forward network) with multi-layer transformer and provides l-th layer representation, such that extraction is by encoder’s embedding. The values include Vl = EnclV(Vl-1) with ZV being the value’s latent representation. Further detail at [P.18 App. B.1] projects dimension, Fig 4};
However Meidani does not appear to disclose the following limitations which is met by Holt:
train a machine learning model using the set of values and a set of labels, the trained machine learning model outputting a symbolic equation based on the training {Holt Fig 2 Eq.1 at [P.4 Sect. 3.1] “train the conditional generator… both the encoder and decoder are trained” for “predicted outputs from the equations generated” generates/outputs equations by DGSR deep generative symbolic regression so-titled, introduced [P.3 Sect.3] where values may be “encoded with an embedding into an additional latent vector… ∈ Rw ” set of real numbers and labels are interpreted to comprise output y of pair (Xi, yi) and/or “set of w unique ground truth” [P.6 ¶4] noting w as the superscript of Rw, datasets include e.g. “labelled Feynman” [P.23 Last¶]. See also Algs.1-2 [P.17-18]}; and
Holt is directed to generative deep network training thus being analogous. A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to train according to the teachings of Holt in combination for a motivation “our goal is to find the single best fitting equation for the observed dataset” [P.5 ¶2], [P.1 ¶1] and which “attempts to fulfill the following properties… can generalize to unseen variables” [P.2 ¶2] contributions.
However, Holt does not appear to disclose the following limitation and which is met by Tran:
apply a second loss function to the first trained neural network to generate a second trained neural network, the second loss function based on the symbolic equation {Tran [P.2] Fig 2 shows NN(s) multiple neural networks for symbolic PINN physics-informed neural networks described [P.2 Sect. II.B] loss is Eq.7 “each loss function of (3) and (4) is multiplied by some factors and added together to form the final loss function” final loss is second loss based on the initial losses Eqs.3-4. See also [P.3 Sect.C] update for loss functions}.
Tran is directed to symbolic neural network training thus being analogous. A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to apply loss per Tran in combination to arrive at the invention as claimed as applying a known technique to a known method ready from improvement to yield predictable results and/or for a motivation to “integrate the loss function over the relevant domain for each portion of the loss function… reducing the number of trainable parameters and the training time… re-evaluating the factors that contribute to the loss function and adjusting their weights to enhance the performance of the network” [P.1 ¶4].
With respect to claim 3, the combination of Meidani, Holt and Tran teaches the system of claim 1, wherein the program code is structured to cause the processor to
apply the second loss function to the first trained neural network by replacing the first loss function with the second loss function {Tran [P.2] Eq.7 final loss replaces each loss Eqs.2-3 integrated and applied to neural networks Fig 2 with updated parameter θ}.
With respect to claim 4, the combination of Meidani, Holt and Tran teaches the system of claim 1, wherein
the machine learning model comprises a symbolic regression model {Holt [P.2 ¶3] “we propose the Deep Generative Symbolic Regression (DGSR) framework” Fig 2 encoder-decoder models with conditional generator. Also Meidani Fig 4 Symbolic Regression encoder-decoder models}.
With respect to claim 8, the combination of Meidani, Holt and Tran teaches the system of claim 1, wherein the program code is further structured to cause the processor to extract the set of values by:
passing each of a plurality of samples through the first trained neural network in an inference mode {Meidani Fig 4(b) Inference shows sampling and encoder-decoder with f(x) where x are samples described e.g. [P.16 App. A.1] “sampling a function f, followed by sampling N numeric input points xi” and/or [P.7 ¶3] “1K-sample test set” and the encoder-decoder is trained neural network Fig 1};
extracting an embedding corresponding to each of the plurality of samples to generate a plurality of embeddings {Meidani [P.4 ¶3] “embedder maps each input point to a unique embedding space” said input points being sampled e.g. [P.5 ¶6] “we sample N input points” for the [P.9 ¶3] “encodings from sampled inputs” Meidani discloses following suggestion of Kamienny_2022 Fig 2}; and
storing the plurality of embeddings as the set of values {Meidani discloses [P.17 App. B.1 ¶1] “retains an embedding… Vl = EnclV(Vl-1)” in an [P.4 ¶3] “embedding space” as well as [P.18 ¶1] main “memory”}.
With respect to claim 9, the rejection of claim 1 is incorporated. The difference in scope being a method to perform limitations of system claim 1. Meidani discloses [P.4 ¶6] “methodology as discussed in Sec. 3.1” and/or [P.2 ¶2] “SNIP, a pioneering pre-training method.” The remainder of this claim is rejected for the same rationale as claim 1.
With respect to claim 11, the combination of Meidani, Holt and Tran teaches the method of claim 9, and further teaches the limitation of claim 3. Therefore, the rejection of claim 3 is applied to claim 11.
With respect to claim 12, the combination of Meidani, Holt and Tran teaches the method of claim 9, and further teaches the limitation of claim 4. Therefore, the rejection of claim 3 is applied to claim 12.
With respect to claim 15, the combination of Meidani, Holt and Tran teaches the method of claim 9, and further teaches the limitation of claim 8. Therefore, the rejection of claim 8 is applied to claim 15.
With respect to claim 16, the rejection of claim 1 is incorporated. The difference in scope being a computer-readable storage medium recording computer program code executed by processor to perform limitations of claim 1. Meidani discloses [P.18 App. B.2] “we utilize 4 GPUs equipped with 48GB of memory” for implementing code Alg.1 [P.24]. The remainder of this claim is rejected for the same rationale as claim 1.
With respect to claim 18, the combination of Meidani, Holt and Tran teaches the computer-readable storage medium of claim 16, and further teaches the limitation of claim 4. Therefore, the rejection of claim 4 is applied to claim 18.
Claims 2, 10 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Meidani, Holt and Tran in view of
Li et al., “A Neural-Guided Dynamic Symbolic Network for Exploring Mathematical Expressions from Data” hereinafter Li (arXiv: 2309.13705v1).
With respect to claim 2, the combination of Meidani, Holt and Tran teaches the system of claim 1. Li teaches
wherein the second trained neural network is trained based on a set of training data in a target environment, the target environment comprising one of: a purchasing environment, a services environment, a computing environment, or an economics environment {Li [P.6 Sect. 3.2 ¶2] RL setting “observes the environment (current architecture) and, based on the observation, takes an action (next available architecture parameter) and transitions into a new state (new architecture)” reinforcement learning is described using trained RNN recurrent neural network shown Fig 2, the target environment is a computing environment which is optimized by reward of RL with policy gradient and trained RNN}; and
wherein the second trained neural network is configured to generate a prediction in the target environment {Li [P.6 Sect. 3.2 ¶2] “RNN predicts can be viewed as a list of actions a1:T ∈ A to design an architecture… predicted target” e.g. [P.7 ¶2] “predicted value for the i-th observation” Fig 2. See also Algs.1-3 [P.15-16]}.
Li is directed to symbolic regression with trained neural networks thus being analogous. A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to specify environment for prediction per Li in combination to arrive at the invention as claimed for a motivation [P.1 ¶1] “Numerous phenomena in the natural world, such as physical laws, can be precisely described using mathematical expressions. Symbolic regression (SR) is an effective machine learning technique that involves discovering mathematical expressions that describe a dataset with accuracy” and because reinforcement setting enables one to “encourage exploration” [P.7 ¶1].
With respect to claim 10, the combination of Meidani, Holt and Tran teaches the method of claim 9, and further combination with Li teaches the limitation of claim 2. Therefore, the rejection of claim 2 with equal motivation is applied to claim 10.
With respect to claim 17, the combination of Meidani, Holt and Tran teaches the computer-readable storage medium of claim 16, and further combination with Li teaches the limitation of claim 2. Therefore, the rejection of claim 2 with equal motivation is applied to claim 17.
Claims 5, 13 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Meidani, Holt and Tran in view of
Scholl et al., “PARFAM – Symbolic Regression based on Continuous Global Optimization” hereinafter Scholl (arXiv: 2310.05537v2).
With respect to claim 5, the combination of Meidani, Holt and Tran teaches the system of claim 4. Scholl teaches wherein the program code is further structured to cause the processor to:
limit a search space of the symbolic regression model to differentiable equations {Scholl see [P.4 Sect. 2.1.2 ¶1] “Restricting the search space to functions of the parametric family given by Equation 1 yields the advantage that we can translate the discrete SR problem into a continuous one” SR being symbolic regression, continuous being continuously differentiable e.g. [P.5 ¶2-3] “yields an end-to-end differentiable pipeline” given dataset [P.6 ¶4] “14 differential equations” listed Table 3}; and
generate the symbolic equation as a differentiable equation based on the limited search space {Scholl [P.5 Last¶] “we train the NN on synthetically generated data… yij = fθj(xi) is the output of the corresponding function” function f(∙) being a differentiable function output by generative neural network NN shown Fig 1. See also Alg.1 [P.16]}.
Scholl is directed to symbolic regression with trained neural network thus being analogous. A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to restrict search space and generate with differentiable pipeline for differential equations per Scholl in combination to arrive at the invention as claimed for a motivation [P.5 ¶2-3] “avoids the evaluation on the data grid in every training step” thereby “reducing the computation time and success rate for any global optimizer” and/or [P.2 ¶3] “grants users precise control over the search space… enables the simple application of pre-trained NNs to reduce the dimensionality of the search space.”
With respect to claim 13, the combination of Meidani, Holt and Tran teaches the method of claim 12, and further combination with Scholl teaches the limitation of claim 5. Therefore, the rejection of claim 5 with equal motivation is applied to claim 13.
With respect to claim 19, the combination of Meidani, Holt and Tran teaches the computer-readable storage medium of claim 18, and further combination with Scholl teaches the limitation of claim 5. Therefore, the rejection of claim 5 with equal motivation is applied to claim 19.
Claims 6-7, 14 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Meidani, Holt and Tran in view of
Liu et al., “SNR: Symbolic network-based rectifiable learning framework for symbolic regression” hereinafter Liu.
With respect to claim 6, the combination of Meidani, Holt and Tran teaches the system of claim 1. Liu teaches
wherein the plurality of layers comprises a plurality of hidden layers {Liu Figs 3-4 illustrate “number of hidden layers” similar Fig 1}, and
wherein the set of values comprises a set of weights of one of the hidden layers {Liu Figs 3-4 illustrate weights-w as connections between the hidden layers with associated values, similar at Fig 1}.
Liu is directed to symbolic regression with trained neural networks thus being analogous. A person having ordinary skill in the art would have considered it obvious prior to the effective filing date to specify hidden layers with weights per Liu in combination to arrive at the invention as claimed as obvious to try in choosing from a finite number of identified, predictable solutions for deep networks to yield predictable results and/or for a motivation [P.1023 Sect.3.2 ¶4] “fine-tune the model… SNR enjoys the benefits of allowing a pre-trained model and fine-tuning” e.g. tuning number of layers as a common hyperparameter, and/or masking of encoding so as to [P.1024 ¶3] “mitigate the problem of overfitting”.
With respect to claim 7, the combination of Meidani, Holt, Tran and Liu teaches the system of claim 6, wherein
the one of the hidden layers comprises a final hidden layer prior to an output layer {Liu shows Figs 3-4 sequence of hidden layers prior to output, this may include e.g. “maxpool layer” [P.1032 Sect. 5.5.3 ¶2] and/or “Linear” functions prior to softmax output Figs 2,1 Eq.5}. Motivation for combination is applied similarly as in claim 6.
With respect to claim 14, the combination of Meidani, Holt and Tran teaches the method of claim 9, and further combination with Liu teaches the limitation of claim 6. Therefore, the rejection of claim 6 with equal motivation is applied to claim 14.
With respect to claim 20, the combination of Meidani, Holt and Tran teaches the computer-readable storage medium of claim 16, and further combination with Liu teaches the limitation of claim 6. Therefore, the rejection of claim 6 with equal motivation is applied to claim 20.
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Kamienny et al., “End-to-End symbolic regression with transformers” arXiv: 2204.10532v1 see Fig 2 illustrating embedder architecture which is referenced by Meidani.
Majumdar et al., “Symbolic Regression for PDEs using Pruned Differentiable Programs” arXiv: 2303.07009v1 Tata-India, see Alg.2 Line9 “finetuned-loss”
Horesh et al., US Patent No 11,657,194B2 claim 1 symbolic regression
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Chase P Hinckley whose telephone number is (571)272-7935. The examiner can normally be reached M-F 9:00 - 5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Miranda M. Huang can be reached at 571-270-7092. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHASE P. HINCKLEY/Examiner, Art Unit 2124
Prosecution Insights