DETAILED ACTION
The present application, filed on 6/23/2023 is being examined under the AIA first inventor to file provisions.
The following is a non-final First Office Action on the Merits. Claims 1-20 are pending and have been considered below.
Information Disclosure Statement (IDS)
The information disclosure statement (IDS) submitted on 6/23/2023 is in compliance with the provisions of 37 CFR 1.97. Accordingly, such IDS is being considered by Examiner.
Claim Rejections - 35 USC § 101
35 USC 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 USC 101 because the claimed invention is not directed to patent eligible subject matter. The claimed matter is directed to a judicial exception, i.e. an abstract idea, not integrated into a practical application, and without significantly more.
Per Step 1 of the multi-step eligibility analysis, claims 1-7 are directed to a computer implemented method, claims 8-14 are directed to computer implemented method, and claims 15-20 are directed to a system.
Thus, on its face, each independent claim and the associated dependent claims are directed to a statutory category of invention.
[INDEPENDENT CLAIMS]
Per Step 2A.1. Independent claim 1, is rejected under 35 USC 101 because the independent claim is directed to an abstract idea, a judicial exception, without reciting additional elements that integrate the judicial exception into a practical application.
The limitations of the independent claim 1 recite an abstract idea, shown in bold below:
[A] A method comprising:
[B] receiving user input identifying a deep equilibrium model and identifying a training dataset; and
[C] training the deep equilibrium model on the training dataset,
[D] wherein the training includes performing a normalization method according to: W = W o min (t, f) = W o min (t, g/N(W)),
[E] where f is the deep equilibrium model, W is a weight matrix, g is a learnable scaling factor, o is a row-wise multiplication, t is a threshold for clipping the scaling factor g, and N is a computation of a norm for the weight matrix W.
Independent claim 1 recites: receiving user input ([B]); training a deep equilibrium model ([C], [D]), which, based on the claim language and in view of the application disclosure, represents a process aimed at: training a deep equilibrium model, by employing a training dataset.
This is a combination that, under its broadest reasonable interpretation, covers performance of limitations expressing mathematical concepts like mathematical relationships, mathematical formulas or equations, mathematical calculations. These fall under the Mathematical Concepts. i.e., mathematical relationships, mathematical formulas or equations, or mathematical calculations grouping of abstract ideas (see MPEP 2106.04(a)(2) I).
Accordingly, it is concluded that independent claim 1 recites an abstract idea that corresponds to a judicial exception.
[INDEPENDENT CLAIMS – Additional Elements]
Per Step 2A.2. The identified abstract idea is not integrated into a practical application because the additional elements in the independent claims only amount to instructions to apply the judicial exception to a computer, or are a general link to a technological environment (see MPEP 2106.05(f); MPEP 2106.05(h)).
For example, the qualifiers “where f is the deep equilibrium model, W is a weight matrix, g is a learnable scaling factor, o is a row-wise multiplication, t is a threshold for clipping the scaling factor g, and N is a computation of a norm for the weight matrix W.” as applied to the training equation, are nothing more than (a) descriptive limitations of claim elements, such as describing the nature, structure and/or content of other claim elements, or (b) general links to the computing environment, which amount to instructions to “apply it,” or equivalent (MPEP 2106.05(f)).
These qualifiers of the independent claims do not preclude from carrying out the identified abstract idea training a deep equilibrium model, by employing a training dataset, and do not serve to integrate the identified abstract idea into a practical application.
Therefore, the additional claim elements of independent claim 1 do not integrate the identified abstract idea into a practical application and the claims remain a judicial exception.
Per Step 2B. Independent claim 1 does not include additional elements that are sufficient to amount to significantly more than the judicial exception because, when the independent claim is reevaluated as a whole, as an ordered combination under the considerations of Step 2B, the outcome is the same like under Step 2A.2.
Overall, it is concluded that independent claim 1 is deemed ineligible.
Per Step 2A.1. Independent claim 8, is rejected under 35 USC 101 because the independent claim is directed to an abstract idea, a judicial exception, without reciting additional elements that integrate the judicial exception into a practical application.
The limitations of the independent claim 8 recite an abstract idea, shown in bold below:
[A] A method:
[B] receiving user input identifying a deep equilibrium model and
[C] identifying a training dataset; and
[D] training the deep equilibrium model on the training dataset,
wherein the training includes
[E] performing forward and backward solvers to conduct forward and backward passes through the deep equilibrium model, and
[F] wherein the forward and backward solvers are identified in the user input.
Independent claim 8 recites: identifying a training dataset ([C]); training a deep equilibrium model by utilizing forward and backward passes ([D], [E]), which, based on the claim language and in view of the application disclosure, represents a process aimed at: training a deep equilibrium model, by employing a training dataset.
This is a combination that, under its broadest reasonable interpretation, covers performance of limitations expressing mathematical concepts like mathematical relationships, mathematical formulas or equations, mathematical calculations. These fall under the Mathematical Concepts. i.e., mathematical relationships, mathematical formulas or equations, or mathematical calculations grouping of abstract ideas (see MPEP 2106.04(a)(2) I).
Accordingly, it is concluded that independent claim 8 recites an abstract idea that corresponds to a judicial exception.
[INDEPENDENT CLAIMS – Additional Elements]
Per Step 2A.2. The identified abstract idea is not integrated into a practical application because the additional elements in the independent claims only amount to instructions to apply the judicial exception to a computer, or are a general link to a technological environment (see MPEP 2106.05(f); MPEP 2106.05(h)).
For example, the qualifiers “wherein the forward and backward solvers are identified in the user input” as applied to the backward solvers, are nothing more than (a) descriptive limitations of claim elements, such as describing the nature, structure and/or content of other claim elements, or (b) general links to the computing environment, which amount to instructions to “apply it,” or equivalent (MPEP 2106.05(f)).
These qualifiers of the independent claims do not preclude from carrying out the identified abstract idea training a deep equilibrium model, by employing a training dataset, and do not serve to integrate the identified abstract idea into a practical application.
The additional steps in the independent claims, shown not bolded above, recite: receiving user input identifying a deep equilibrium model ([B]). When considered individually, they amount to nothing more than receiving data, processing data, storing results or transmitting data that serves merely to implement the abstract idea using computing components for performing computer functions (corresponding to the words “apply it” or an equivalent), or merely uses a computer as a tool to perform the identified abstract idea. Thus, it is concluded that these claim elements do not integrate the identified abstract idea (training a deep equilibrium model, by employing a training dataset) into a practical application (see MPEP 2106.05(f)(2)).
Therefore, the additional claim elements of independent claim 8 do not integrate the identified abstract idea into a practical application and the claims remain a judicial exception.
Per Step 2B. Independent claim 8 does not include additional elements that are sufficient to amount to significantly more than the judicial exception because, when the independent claim is reevaluated as a whole, as an ordered combination under the considerations of Step 2B, the outcome is the same like under Step 2A.2.
Overall, it is concluded that independent claim 8 is deemed ineligible.
Per Step 2A.1. Independent claim 15, is rejected under 35 USC 101 because the independent claim is directed to an abstract idea, a judicial exception, without reciting additional elements that integrate the judicial exception into a practical application.
The limitations of the independent claim 15 recite an abstract idea, shown in bold below:
[A] A system comprising: one or more processors; and non-transitory memory including processor-executable instructions:
[B] receiving user input identifying a deep equilibrium model and
[C] identifying a training dataset; and
[D] training the deep equilibrium model on the training dataset,
wherein the training includes:
[E] performing forward and backward solvers to conduct forward and backward passes through the deep equilibrium model,
[F] wherein the forward and backward solvers are identified in the user input; and
[G] performing one or more of the following: automatic normalization of weight tensors; Jacobian regularization; and fixed point correction.
Independent claim 15 recites: identifying a training data set ([C]); training a deep equilibrium model by performing forward and backward solvers([D], [E]); performing selected training operations ([G]), which, based on the claim language and in view of the application disclosure, represents a process aimed at: training a deep equilibrium model, by employing a training dataset.
This is a combination that, under its broadest reasonable interpretation, covers performance of limitations expressing mathematical concepts like mathematical relationships, mathematical formulas or equations, mathematical calculations. These fall under the Mathematical Concepts. i.e., mathematical relationships, mathematical formulas or equations, or mathematical calculations grouping of abstract ideas (see MPEP 2106.04(a)(2) I).
Accordingly, it is concluded that independent claim 15 recites an abstract idea that corresponds to a judicial exception.
[INDEPENDENT CLAIMS – Additional Elements]
Per Step 2A.2. The identified abstract idea is not integrated into a practical application because the additional elements in the independent claims only amount to instructions to apply the judicial exception to a computer, or are a general link to a technological environment (see MPEP 2106.05(f); MPEP 2106.05(h)).
For example, the added elements “processors,” “memory,” recite computing elements at a high level of generality, generally linking the use of a judicial exception to a particular technological environment (see MPEP 2106.05(h)), or merely using a computer as a tool to perform an abstract idea (MPEP 2106.05(f)). Further, the qualifiers “wherein the forward and backward solvers are identified in the user input” as applied to the forward and backward solvers, are nothing more than (a) descriptive limitations of claim elements, such as describing the nature, structure and/or content of other claim elements, or (b) general links to the computing environment, which amount to instructions to “apply it,” or equivalent (MPEP 2106.05(f)).
These qualifiers of the independent claims do not preclude from carrying out the identified abstract idea training a deep equilibrium model, by employing a training dataset, and do not serve to integrate the identified abstract idea into a practical application.
The additional steps in the independent claims, shown not bolded above, recite: receiving user input identifying a deep equilibrium model ([B]). When considered individually, they amount to nothing more than receiving data, processing data, storing results or transmitting data that serves merely to implement the abstract idea using computing components for performing computer functions (corresponding to the words “apply it” or an equivalent), or merely uses a computer as a tool to perform the identified abstract idea. Thus, it is concluded that these claim elements do not integrate the identified abstract idea (training a deep equilibrium model, by employing a training dataset) into a practical application (see MPEP 2106.05(f)(2)).
Therefore, the additional claim elements of independent claim 15 do not integrate the identified abstract idea into a practical application and the claims remain a judicial exception.
Per Step 2B. Independent claim 15 does not include additional elements that are sufficient to amount to significantly more than the judicial exception because, when the independent claim is reevaluated as a whole, as an ordered combination under the considerations of Step 2B, the outcome is the same like under Step 2A.2.
Overall, it is concluded that independent claim 15 is deemed ineligible.
[DEPENDENT CLAIMS]
Dependent claim 2 (which is representative of claims 11, 16) recites:
identifies an injection module.
When considered individually, these added claim elements further elaborate on the abstract idea identified in the independent claims, because the dependent claim continues to recite the identified abstract idea: training a deep equilibrium model, by employing a training dataset. The elements in this dependent claim are comparable to “sorting information” i.e. comparing data, which has been recognized by a controlling court as "well-understood, routine and conventional computing functions" when claimed generically as they are in these dependent claims. Thus, it is concluded that these claim elements do not integrate the identified abstract idea (training a deep equilibrium model, by employing a training dataset) into a practical application (see MPEP 2106.05(d) II)).
The dependent claim elements have the same relationship to the underlying abstract idea (training a deep equilibrium model, by employing a training dataset) as outlined in the independent claims analysis above. Thus, the dependent claim elements are not directed to any specific improvements of the independent claims and do not practically or significantly alter how the identified abstract idea would be performed. When considered as a whole, as an ordered combination, the dependent claim further elaborates on the previously identified abstract idea (training a deep equilibrium model, by employing a training dataset).
Therefore, dependent claim 2 (which is representative of claims 11, 16) is deemed ineligible.
Dependent claim 3 (which is representative of claims 12, 17) recites:
identifies a decoder module.
When considered individually, these added claim elements further elaborate on the abstract idea identified in the independent claims, because the dependent claim continues to recite the identified abstract idea: training a deep equilibrium model, by employing a training dataset. The elements in this dependent claim are comparable to “sorting information” i.e. comparing data, which has been recognized by a controlling court as "well-understood, routine and conventional computing functions" when claimed generically as they are in these dependent claims. Thus, it is concluded that these claim elements do not integrate the identified abstract idea (training a deep equilibrium model, by employing a training dataset) into a practical application (see MPEP 2106.05(d) II)).
The dependent claim elements have the same relationship to the underlying abstract idea (training a deep equilibrium model, by employing a training dataset) as outlined in the independent claims analysis above. Thus, the dependent claim elements are not directed to any specific improvements of the independent claims and do not practically or significantly alter how the identified abstract idea would be performed. When considered as a whole, as an ordered combination, the dependent claim further elaborates on the previously identified abstract idea (training a deep equilibrium model, by employing a training dataset).
Therefore, dependent claim 3 (which is representative of claims 12, 17) is deemed ineligible.
Dependent claim 4 recites:
performing forward and backward solvers to conduct forward and backward passes through the deep equilibrium model.
When considered individually, these added claim elements further elaborate on the abstract idea identified in the independent claims, because the dependent claim continues to recite the identified abstract idea: training a deep equilibrium model, by employing a training dataset. The elements in this dependent claim are comparable to performance of limitations expressing mathematical concepts like mathematical relationships, mathematical formulas or equations, mathematical calculations. These fall under the Mathematical Concepts. i.e., mathematical relationships, mathematical formulas or equations, or mathematical calculations grouping of abstract ideas (see MPEP 2106.04(a)(2) I).
The dependent claim elements have the same relationship to the underlying abstract idea (training a deep equilibrium model, by employing a training dataset) as outlined in the independent claims analysis above. Thus, the dependent claim elements are not directed to any specific improvements of the independent claims and do not practically or significantly alter how the identified abstract idea would be performed. When considered as a whole, as an ordered combination, the dependent claim further elaborates on the previously identified abstract idea (training a deep equilibrium model, by employing a training dataset).
Therefore, dependent claim 4 is deemed ineligible.
Dependent claim 6 (which is representative of claims 7, 10) recites:
performing one or more of the following: automatic normalization of weight tensors; Jacobian regularization; and fixed point correction.
When considered individually, these added claim elements further elaborate on the abstract idea identified in the independent claims, because the dependent claim continues to recite the identified abstract idea: training a deep equilibrium model, by employing a training dataset. The elements in this dependent claim are comparable to performance of limitations expressing mathematical concepts like mathematical relationships, mathematical formulas or equations, mathematical calculations. These fall under the Mathematical Concepts. i.e., mathematical relationships, mathematical formulas or equations, or mathematical calculations grouping of abstract ideas (see MPEP 2106.04(a)(2) I).
The dependent claim elements have the same relationship to the underlying abstract idea (training a deep equilibrium model, by employing a training dataset) as outlined in the independent claims analysis above. Thus, the dependent claim elements are not directed to any specific improvements of the independent claims and do not practically or significantly alter how the identified abstract idea would be performed. When considered as a whole, as an ordered combination, the dependent claim further elaborates on the previously identified abstract idea (training a deep equilibrium model, by employing a training dataset).
Therefore, dependent claim 6 (which is representative of claims 7, 10) is deemed ineligible.
Dependent claim 13 (which is representative of claims 14, 19, 20) recites:
performing a normalization method according to:
W= W o min (t, f) = W o min (t, g/N(W)),
When considered individually, these added claim elements further elaborate on the abstract idea identified in the independent claims, because the dependent claim continues to recite the identified abstract idea: training a deep equilibrium model, by employing a training dataset. The elements in this dependent claim are comparable to performance of limitations expressing mathematical concepts like mathematical relationships, mathematical formulas or equations, mathematical calculations. These fall under the Mathematical Concepts. i.e., mathematical relationships, mathematical formulas or equations, or mathematical calculations grouping of abstract ideas (see MPEP 2106.04(a)(2) I).
The dependent claim elements have the same relationship to the underlying abstract idea (training a deep equilibrium model, by employing a training dataset) as outlined in the independent claims analysis above. Thus, the dependent claim elements are not directed to any specific improvements of the independent claims and do not practically or significantly alter how the identified abstract idea would be performed. When considered as a whole, as an ordered combination, the dependent claim further elaborates on the previously identified abstract idea (training a deep equilibrium model, by employing a training dataset).
Therefore, dependent claim 13 (which is representative of claims 14, 19, 20) is deemed ineligible.
Dependent claims 5, 9, 18 recite:
wherein one or more of the forward and backward solvers are modified by parameters included in the user input.
wherein one or more of the forward and backward solvers are modified by parameters included in the user input.
wherein one or more of the forward and backward solvers are modified by parameters included in the user input.
These further elements in the dependent claims do not perform any claimed method steps. They describe the nature, structure and/or content of other claim elements – forward and backward solvers – and as such, cannot change the nature of the identified abstract idea (training a deep equilibrium model, by employing a training dataset), from a judicial exception into eligible subject matter, because they do not represent significantly more (see MPEP 2106.07). The nature, form or structure of the other claim elements themselves do not practically or significantly alter how the identified abstract idea would be performed and do not provide more than a general link to a technological environment.
Therefore, dependent claims 5, 9, 18 are deemed ineligible.
When the dependent claims are considered as a whole, as an ordered combination, the claim elements noted above appear to merely apply the abstract concept to a technical environment in a very general sense. The most significant elements, which form the abstract concept, are set forth in the independent claims. The fact that the computing devices and the dependent claims are facilitating the abstract concept is not enough to confer statutory subject matter eligibility, since their individual and combined significance do not transform the identified abstract concept at the core of the claimed invention into eligible subject matter. Therefore, it is concluded that the dependent claims of the instant application, considered individually, or as a as a whole, as an ordered combination, do not amount to significantly more (see MPEP 2106.07(a)II).
In sum, Claims 1-20 are rejected under 35 USC 101 as being directed to non-statutory subject matter.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the difference between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103(a) are summarized as follows:
i. Determining the scope and contents of the prior art.
ii. Ascertaining the differences between the prior art and the claims at issue.
iii. Resolving the level of ordinary skill in the pertinent art.
iv. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 8-12, 15-18 are rejected under 35 U.S.C. 103 as being unpatentable over Bai et al (US 2021/0042606).
Regarding Claim 8: Bai first embodiment discloses: A method comprising:
wherein the training includes
performing forward and backward solvers to conduct forward and backward passes through the deep equilibrium model, and {see at least [0004] forward/backward passes; fig4b, rc480, [0163] equilibrium solver}
wherein the forward and backward solvers are identified in the user input. {see at least fig3, rc310, [0160] equilibrium solver; [0161] input injection (reads on user input)}
Bai first embodiment does not disclose, however, Bai second embodiment discloses:
receiving user input identifying a deep equilibrium model and identifying a training dataset; and {see at least [0168] DEQ approach evaluated in large datasets (reads on deep equilibrium model and datasets}
training the deep equilibrium model on the training dataset, {see at least [0169] training the DEQ from the scratch}
It would have been obvious to one of ordinary skill in the art, at the time of filing, to modify Bai first embodiment to include the elements of Bai second embodiment. One would have been motivated to do so, in order to provide the model with the necessary training data. In the instant case, Bai first embodiment evidently discloses performing forward and backward solvers. Bai second embodiment is merely relied upon to illustrate the functionality of training a DEQ model in the same or similar context. Since both performing forward and backward solvers, as well as training a DEQ model are implemented through well-known computer technologies in the same or similar context, combining their features as outlined above using such well-known computer technologies (i.e., conventional software/hardware configurations), would be reasonable, according to one of ordinary skill in the art. Moreover, since the elements disclosed by Bai first embodiment, as well as Bai second embodiment would function in the same manner in combination as they do in their separate embodiments, it would be reasonable to conclude that their resulting combination would be predictable. Accordingly, the claimed subject matter is obvious over Bai. **Examiner notes that the reference is being used here as a one-reference combination in this 103 rejection because the reference teaches two clearly different embodiments within the same cited reference.**
Regarding Claims 9, 18: Bai discloses the limitations of Claims 8, 15. Bai further discloses:
wherein one or more of the forward and backward solvers are modified by parameters included in the user input. {see at least [0161] input injection (reads on user input)}
Regarding Claims 10: Bai discloses the limitations of Claims 8. Bai further discloses: wherein the training includes
performing one or more of the following: automatic normalization of weight tensors; Jacobian regularization; and fixed point correction. {see at least [0022] Jacobian at equilibrium; Jacobian matrix}
Regarding Claims 11, 16: Bai discloses the limitations of Claims 8, 15. Bai further discloses:
identifies an injection module. {see at least [0161] injection module}
Regarding Claims 12, 17: Bai discloses the limitations of Claims 8, 15. Bai further discloses:
identifies a decoder module. {see at least [0138] supports generalization (based on BRI (MPEP 2111), reads on decoding}
Regarding Claim 15: Bai first embodiment discloses: A system comprising: one or more processors; and non-transitory memory including processor-executable instructions that, when executed by the one or more processors, causes the system to perform operations including: {see at least fig1, rc100, [0109]-[0100] processor, memory}
performing forward and backward solvers to conduct forward and backward passes through the deep equilibrium model, {see at least [0004] forward/backward passes; fig4b, rc480, [0163] equilibrium solver}
wherein the forward and backward solvers are identified in the user input; and {see at least fig3, rc310, [0160] equilibrium solver; [0161] input injection (reads on user input)}
Bai first embodiment does not disclose, however, Bai second embodiment discloses:
receiving user input identifying a deep equilibrium model and identifying a training dataset; and {see at least [0168] DEQ approach evaluated in large datasets (reads on deep equilibrium model and datasets}
training the deep equilibrium model on the training dataset, wherein the training includes: {see at least [0169] training the DEQ from the scratch}
performing one or more of the following: automatic normalization of weight tensors; Jacobian regularization; and fixed point correction. {see at least [0022] Jacobian at equilibrium; Jacobian matrix}
It would have been obvious to one of ordinary skill in the art, at the time of filing, to modify Bai first embodiment to include the elements of Bai second embodiment. One would have been motivated to do so, in order to provide the model with the necessary training data. In the instant case, Bai first embodiment evidently discloses performing forward and backward solvers. Bai second embodiment is merely relied upon to illustrate the functionality of training a DEQ model in the same or similar context. Since both performing forward and backward solvers, as well as training a DEQ model are implemented through well-known computer technologies in the same or similar context, combining their features as outlined above using such well-known computer technologies (i.e., conventional software/hardware configurations), would be reasonable, according to one of ordinary skill in the art. Moreover, since the elements disclosed by Bai first embodiment, as well as Bai second embodiment would function in the same manner in combination as they do in their separate embodiments, it would be reasonable to conclude that their resulting combination would be predictable. Accordingly, the claimed subject matter is obvious over Bai. **Examiner notes that the reference is being used here as a one-reference combination in this 103 rejection because the reference teaches two clearly different embodiments within the same cited reference.**
Claim Objections
Claims 1-7 are objected to for being allowable, if they overcome the 35 USC 101 rejection. The allowable feature is the equation of the independent claim:
W= W o min (t, f) = W o min (t, g/N(W))
Claims 13-14, 18-19 are objected to for being allowable, if
rewritten in independent form including all of the limitations of the base claim and any intervening claims.
they overcome the 35 USC 101 rejection.
The prior art made of record and not relied upon which, however, is considered pertinent to applicant's disclosure:
US 20220027130 A1 KASHMIRI; Sayyed Mahdi et al. TIME DOMAIN RATIOMETRIC READOUT INTERFACES FOR ANALOG MIXED-SIGNAL IN MEMORY COMPUTE CROSSBAR NETWORKS A circuit configured to compute matrix multiply-and-add calculations that includes a digital-to-time converter configured to receive a digital input and output a signal proportional to the digital input and modulated in time-domain associated with a reference time, a memory including a crossbar network, wherein the memory is configured to receive the time modulated signal from the digital-to-time converter and output a weighted signal scaled in response to network weights of the crossbar network and the time modulated input signal, and an output interface in communication with the crossbar network and configured to receive its weighted output signal and output a digital value proportional to at least the reference time using a time-to-digital converter.
US 20210326663 A1 WINSTON; Ezra et al. SYSTEM AND METHOD OF A MONOTONE OPERATOR NEURAL NETWORK A system for training a neural work that includes an input interface for accessing input data for the neural network and a processor in communication with the input interface. The processor is programmed to receive input at the neural network and output a trained neural networking utilizing a forward prorogation and a backward propagation, wherein the forward propagation includes utilizing a root-finding procedure to identify a fixed point associated with one or more parameters of the neural network, wherein the backward propagation includes identifying a derivative of a loss associated with the parameters of the network.
US 20210383234 A1 BAI; Shaojie et al. SYSTEM AND METHOD FOR MULTISCALE DEEP EQUILIBRIUM MODELS A computer-implemented method for a classification and training a neural network includes receiving input at the neural network, wherein the input includes a plurality of resolution inputs of varying resolutions, outputting a plurality of feature tensors for each corresponding resolution of the plurality of resolution inputs, fusing the plurality of feature tensors utilizing upsampling or down sampling for the vary resolutions, utilizing an equilibrium solver to identify one or more prediction vectors from the plurality of feature tensors, and outputting a loss in response to the one or more prediction vectors.
US 20230101812 A1 FENG; Zhili et al. MONOTONE MEAN-FIELD INFERENCE IN DEEP MARKOV RANDOM FIELDS Methods and systems for inferring data to supplement an input utilizing a neural network, and training such a system, are disclosed. In embodiments, an input is received from a sensor at the neural network. Iterations of approximate probabilities can be determined based on hidden-to-hidden Markov random field (MRF) potentials, observed-to-hidden MRF potentials, and unary MRF potentials. A constant can be identified using a root-finding algorithm. The iterations can continue until convergence. The final iteration of the approximate probability can be used to supplement the input to produce an output.
US 20220027723 A1 KOLTER; Jeremy et al. HARDWARE COMPUTE FABRICS FOR DEEP EQUILIBRIUM MODELS A dynamic equilibrium (DEQ) model circuit includes a first multiplier configured to receive an input, scale the input by a first weight, and output the scaled input, second multiplier configured to receive a root, scale the root by a second weight, and output the scaled root, a summation block configured to combine the scaled input, a bias input, and the scaled root and output a non-linear input, and a first non-linear function configured to receive the non-linear input and output the root, wherein the first weight and second weight are based on a trained DEQ model of a neural network.
US 20220277859 A1 Alesiani; Francesco METHOD AND SYSTEM TO DIFFERENTIATE THROUGH BILEVEL OPTIMIZATION PROBLEMS USING MACHINE LEARNING The present invention provides a method for bilevel optimization using machine learning. The method comprises: obtaining input data associated with the bilevel optimization; determining a solution for the bilevel problem; updating, based on the solution for the bilevel problem, a neural network using one or more intermediate parameters associated with the neural network and the bilevel optimization, wherein the one or more intermediate parameters are based on first output from the neural network and second output from a loss function associated with the neural network, wherein the first output is generated based on inputting the input data into the neural network; and outputting one or more finalized parameters for the bilevel optimization based on a change of the one or more intermediate parameters reaching a pre-determined threshold.
US 20220028444 A1 PAPAGEORGIOU; Efthymios et al. READ ONLY MEMORY ARCHITECTURE FOR ANALOG MATRIX OPERATIONS A read-only memory (ROM) computing unit utilized in matrix operations of a neural network comprising a unit element including one or more connections, wherein a weight associated with the computing unit is responsive to either a connection or lack of connection internal to the unit cell or between the unit element and a wordline and a bitline utilized to form an array of rows and columns in the ROM computing unit, and one or more passive or active electrical elements located in the unit element, wherein the passive or active electrical elements are configured to adjust the weight associated with the compute unit, wherein the ROM computing unit is configured to receive an input and output a value associated with the matrix operation, wherein the value is responsive to the input and weight.
US 11558620 B2 Besenbruch; Chri et al. Image encoding and decoding, video encoding and decoding: methods, systems and training methods Lossy or lossless compression and transmission, comprising the steps of: (i) receiving an input image; (ii) encoding it using an encoder trained neural network, to produce a y latent representation; (iii) encoding the y latent representation using a hyperencoder trained neural network, to produce a z hyperlatent representation; (iv) quantizing the z hyperlatent representation using a predetermined entropy parameter to produce a quantized z hyperlatent representation; (v) entropy encoding the quantized z hyperlatent representation into a first bitstream, using predetermined entropy parameters; (vi) processing the quantized z hyperlatent representation using a hyperdecoder trained neural network to obtain a location entropy parameter μ.sub.y, an entropy scale parameter σ.sub.y, and a context matrix A.sub.y of the y latent representation; (vii) processing the y latent representation, the location entropy parameter μ.sub.y and the context matrix A.sub.y, to obtain quantized latent residuals; (viii) entropy encoding the quantized latent residuals into a second bitstream, using the entropy scale parameter σ.sub.y; and (ix) transmitting the bitstreams.
US 11763545 B2 Shelhamer; Evan et al. Generating confidence-adaptive pixel-level predictions utilizing a multi-exit pixel-level prediction neural network The present disclosure relates to systems, methods, and non-transitory computer readable media for efficiently, quickly, and flexibly generating and providing pixel-wise classification predictions utilizing early exit heads of a multi-exit pixel-level prediction neural network. For example, the disclosed systems utilize a multi-exit pixel-level prediction neural network to generate classification predictions for a digital image on the pixel level. The multi-exit pixel-level prediction neural network includes a specialized architecture with early exit heads having unique encoder-decoder architectures for generating pixel-wise classification predictions at different early exit stages. In some embodiments, the disclosed systems implement a spatial confidence-adaptive scheme to mask certain predicted pixels to prevent further processing of the masked pixels and thereby reduce computation.
US 11755890 B2 Kumar; Suhas et al. Local training of neural networks A method for performing learning is described. A free inference is performed on a learning network for input signals. The input signals correspond to target output signals. The learning network includes inputs that receive the input signals, neurons, weights interconnecting the neurons, and outputs. The learning network is described by an energy for the free inference. The energy includes an interaction term corresponding to interactions consisting of neuron pair interactions. The free inference results in output signals. A first portion of the plurality of weights corresponding to data flow for the free inference. A biased inference is performed on the learning network by providing the input signals to the inputs and bias signals to the outputs. The bias signals are based on the target output signals and the output signals. The bias signals are feedback to the learning network through a second portion of the weights corresponding to a transpose of the first portion of the weights. At locations in the learning network, learning network equilibrium states are determined for the biased inference. The weights are updated based on the learning network equilibrium states.
Inquiries
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Radu Andrei whose telephone number is 313.446.4948. The examiner can normally be reached on Monday – Friday 8:30am – 5pm EST. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Patrick McAtee can be reached at 571.272.7575. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http:/www.uspto.gov/interviewpractice.
As disclosed in MPEP 502.03, communications via Internet e-mail are at the discretion of the applicant. Without a written authorization by applicant in place, the USPTO will not respond via Internet e-mail to any Internet correspondence which contains information subject to the confidentiality requirement as set forth in 35 U.S.C. 122. A paper copy of such correspondence will be placed in the appropriate patent application. The following is a sample authorization form which may be used by applicant:
“Recognizing that Internet communications are not secure, I hereby authorize the USPTO to communicate with me concerning any subject matter of this application by electronic mail. I understand that a copy of these communications will be made of record in the application file.”
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Status information for published applications may be obtained from Patent Center information webpage. Status information for unpublished applications is available to registered users through Patent Center information webpage only.
To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov.
Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (in USA or CANADA) or 571-272-1000.
Any response to this action should be mailed to:
Commissioner of Patents and Trademarks
P.O. Box 1450
Alexandria, VA 22313-1450
or faxed to 571-273-8300
/Radu Andrei/
Primary Examiner, AU 3698