DETAILED ACTION
Response to Amendment
1. This office action is in response to applicant’s communication filed on 03/10/2026 in response to PTO Office Action mailed 01/07/2026. The Applicant’s remarks and amendments to the claims and/or the specification were considered with the results as follows.
2. In response to the last Office Action, claims 1 and 6 are amended. Claims 8-11 are added. Claims 2-4 are canceled. As a result, claims 1 and 5-11 are pending in this office action.
Response to Arguments
3. Applicant’s arguments with respect 35 USC 101 rejections have been considered but are not persuasive and the details are as follow:
Applicant’s argument stated as “The amended claim 1 is directed to a specific technical improvement in learning neural networks that utilize structurally-constrained weight matrices.”.
In response to Applicant’s argument, the Examiner disagrees, because claim 1 recites an abstract idea of updating rotation angle of a rotation matrix of a neural network by using a differential equation with parameters. Claim 1 is an abstract idea because its limitations describe a process that, under the broadest reasonable interpretation, can be performed mentally using pen and paper. Specifically, claim 1 involves training parameters in a machine learning model, formulating a differential equation based on those parameters and generating an updated rotation angle of a rotation matrix. Applicant uses computer technology [e.g., a machine learning model] to obtain data, analyze, and manipulate the data to produce results. However, Courts have determined that collecting and analyzing information are steps that can be performed mentally, which fall within the category of Mental Processes of abstract ideas. Therefore, claim 1 is directed to an abstract idea, which lacks sufficient additional elements to transform into a patent-eligible application. It is not considered an improvement in computer-related technology, as it merely implements the abstract idea on a generic computer. Therefore, claim 1 is not eligible.
4. Applicant’s arguments with respect to claims 1 and 5-11 have been considered but are moot in view of the new ground of rejection(s).
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
5. Claims 1 and 5-11 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter.
Claims 1 and 5-11 are rejected under 35 U.S.C 101 because the claimed invention is directed to a judicial exception (i.e., an abstract idea) without significantly more. Claim 1 is directed to an abstract idea of learning a neural network including a linear transformational layer achieved by a weight matrix with a complex number as an element, as explained in detail below. The claim does not include elements that are sufficient to amount to significantly more than the judicial exception because the elements can be concepts performed in the human mind which do not add meaningful limits to practicing the abstract idea.
Claim 1 recites an apparatus comprising at least in part:
formulate a differential equation of a loss function with respect to each of conjugate variables corresponding to input variables of the linear transformation layer based on Wirtinger calculus and a differential equation of the loss of function with respect to each of parameters of the neural network defined in the weight matrix, wherein the parameters include a rotation angle contained in a Givens matrix constituting the weight matrix (e.g., observing and formulating a differential equation with respect to input variables in a neural network can be performed in the human mind);
learn the parameters of the neural network by backpropagation using the formulated differential equations, thereby updating the rotation angle (e.g., observing and learning parameters of the neural network using the formulated differential equations can be performed in the human mind);
Claim 1, as it is recited, falls within one of the groupings of abstract ideas [e.g., mental process] enumerated in the 2019 PEG. The recited concept can be performed in the human mind, including observation, evaluation, judgement, and opinion, using pen and paper. That is, other than reciting a learning apparatus comprising a processor and a memory storing program instructions that cause the processor to learns a neural network, nothing in the claim precludes the step from practically being performed in the mind. The formulating and the learning features in the claim are recited at a high level of generality and add no more to the claimed invention than a computer to perform an abstract idea. The additional feature merely uses a computer/device as a tool to learn data after a series of data gathering step is insignificant extra-solution activity, thus, the judicial exception is not integrated into a practical application. The additional feature does not appear to be improvements to the functioning of a computer or to any other technology or technical field. The additional feature does not amount to significantly more than the above-identified judicial exception (the abstract idea). Looking at the limitation as an ordered combination adds nothing that is not already present when looking at the elements taken individually. There is no indication that the combination of elements improves the functioning of a computer or improves any other technology. Their collective functions merely provide conventional computer implementation. Therefore, claim 1 is not patent eligible.
Claim 5 recites similar features as claim 1, is also falls within one of the groupings of abstract ideas [e.g., mental process] enumerated in the 2019 PEG. The recited concept can be performed in human mind including observation, evaluation, judgement, opinion. Claim 5 recites a computation graph to learn parameters. The additional feature merely uses a computer/device as a tool to transform data after a series of data gathering step is insignificant extra-solution activity, thus, the judicial exception is not integrated into a practical application. The additional feature does not appear to be improvements to the functioning of a computer or to any other technology or technical field. The additional feature does not amount to significantly more than the above-identified judicial exception (the abstract idea). Looking at the limitation as an ordered combination adds nothing that is not already present when looking at the elements taken individually. There is no indication that the combination of elements improves the functioning of a computer or improves any other technology. Their collective functions merely provide conventional computer implementation. Therefore, claim 5 is not patent eligible.
Claim 6 recites a method comprising at least in part:
formulate a differential equation of a loss function with respect to each of conjugate variables corresponding to input variables of the linear transformation layer based on Wirtinger calculus and a differential equation of the loss of function with respect to each of parameters of the neural network defined in the weight matrix, wherein the parameters include a rotation angle contained in a Givens rotation matrix constituting the weight matrix (e.g., observing and formulating a differential equation with respect to input variables in a neural network can be performed in the human mind);
learn the parameters of the neural network by backpropagation using the formulated differential equations, thereby updating the rotation angle (e.g., observing and learning parameters of the neural network using the formulated differential equations can be performed in the human mind);
Claim 6, as it is recited, falls within one of the groupings of abstract ideas [e.g., mental process] enumerated in the 2019 PEG. The recited concept can be performed in the human mind, including observation, evaluation, judgement, and opinion, using pen and paper. That is, other than reciting a computer to learns a neural network, nothing in the claim precludes the step from practically being performed in the mind. The formulating and the learning features in the claim are recited at a high level of generality and add no more to the claimed invention than a computer to perform an abstract idea. The additional feature merely uses a computer/device as a tool to learn data after a series of data gathering step is insignificant extra-solution activity, thus, the judicial exception is not integrated into a practical application. The additional feature does not appear to be improvements to the functioning of a computer or to any other technology or technical field. The additional feature does not amount to significantly more than the above-identified judicial exception (the abstract idea). Looking at the limitation as an ordered combination adds nothing that is not already present when looking at the elements taken individually. There is no indication that the combination of elements improves the functioning of a computer or improves any other technology. Their collective functions merely provide conventional computer implementation. Therefore, claim 6 is not patent eligible.
Claim 7 recites a non-transitory computer readable medium comprising at least in part:
formulate a differential equation of a loss function with respect to each of conjugate variables corresponding to input variables of the linear transformation layer and a differential equation of the loss of function with respect to each of parameters of the neural network defined in the weight matrix, wherein the parameters include a rotation angle contained in a Givens rotation matrix constituting the weight matrix (e.g., observing and formulating a differential equation with respect to input variables in a neural network can be performed in the human mind);
learn the parameters of the neural network by backpropagation using the formulated differential equations thereby updating the rotation angle (e.g., observing and learning parameters of the neural network using the formulated differential equations can be performed in the human mind);
Claim 7, as it is recited, falls within one of the groupings of abstract ideas [e.g., mental process] enumerated in the 2019 PEG. The recited concept can be performed in the human mind, including observation, evaluation, judgement, and opinion, using pen and paper. That is, other than reciting a computer to learns a neural network, nothing in the claim precludes the step from practically being performed in the mind. The formulating and the learning features in the claim are recited at a high level of generality and add no more to the claimed invention than a computer to perform an abstract idea. The additional feature merely uses a computer/device as a tool to learn data after a series of data gathering step is insignificant extra-solution activity, thus, the judicial exception is not integrated into a practical application. The additional feature does not appear to be improvements to the functioning of a computer or to any other technology or technical field. The additional feature does not amount to significantly more than the above-identified judicial exception (the abstract idea). Looking at the limitation as an ordered combination adds nothing that is not already present when looking at the elements taken individually. There is no indication that the combination of elements improves the functioning of a computer or improves any other technology. Their collective functions merely provide conventional computer implementation. Therefore, claim 7 is not patent eligible.
Claims 8-11 recites similar features as claim 1, is also falls within one of the groupings of abstract ideas [e.g., mental process] enumerated in the 2019 PEG. The recited concept can be performed in human mind including observation, evaluation, judgement, opinion. Claims 8-11 further define the weight matrix, the parameter includes a rotation angle, a computation graph to formulate differential equations with a loss function. The additional feature merely uses a computer/device as a tool to transform data after a series of data gathering step is insignificant extra-solution activity, thus, the judicial exception is not integrated into a practical application. The additional feature does not appear to be improvements to the functioning of a computer or to any other technology or technical field. The additional feature does not amount to significantly more than the above-identified judicial exception (the abstract idea). Looking at the limitation as an ordered combination adds nothing that is not already present when looking at the elements taken individually. There is no indication that the combination of elements improves the functioning of a computer or improves any other technology. Their collective functions merely provide conventional computer implementation. Therefore, claims 8-11 are not patent eligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 5, 6, 7, 10 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. (US 2019/0258335 A1), hereinafter Wang and in view of Ruseckas “Differential equations as models of deep neural networks, 2019”.
Referring to claims 1, 6 and 7, Wang discloses a learning apparatus that learns a neural network (See para. [0004], one or more machine-learned models that learns artificial neural networks) including a linear transformation layer achieved by a weight matrix with a complex number as an element (See para. [0033], para. [0109], para. [0127], a linear weight matrix for input transformation and w.sub.2 is evaluated through a multi-layer perceptron, the weight matrix for input transformation comprises number calculations, the number of kernel evaluations are computed and stored can be reduced to NX|S|X0), the learning apparatus comprising: a processor; and a memory storing program instructions that cause the processor (See para. [0051], the computer system including one or more processors and a memory) to:
formulate a differential equation (See para. [0122], the continuous convolution layers comprise differentiable operators according equation 6) […] including entropy loss with respect to each of conjugate variables corresponding to input variables of the linear transformation layer (See para. [0036], para. [0087] and para. [0149], the continuous convolutional neural network comprises one or more continuous layers including training an operator such as a parametric continuous kernel function, in a specific example, the output of continuous convolutional neural network includes output features, object detections, object classifications, or motion estimations maybe used to train the network end-end using cross entropy loss) […]; and
learn the parameters of the neural network by backpropagation using the formulated differential equations (See para. [0122], the building blocks including parametric continuous convolution layers can be differentiable operators, such that the networks can be learned through back propagation, according to equation).
Wang does not explicitly disclose formulate a differential equation of a loss function with to each of conjugate variables based on calculus and a differential equation of the loss with respect to each parameter of the neural network defined in the weight matrix, wherein the parameters include a rotation angle contained in Givens rotation matrix constituting the weight matrix.
Ruseckas discloses formulate a differential equation of a loss function with to each of conjugate variables (See page 2, Section 2, generating nonlinear ordinary differential equations of a loss function with parameters q(t)) based on calculus (See page 3, the parameters q(t) need to use calculus of variations) and a differential equation of the loss with respect to each parameter of the neural network defined in the weight matrix (See page 6, section 3, the differential equation consisting a large number L of layers, the w is matrix of weights and B is a bias vector), wherein the parameters include a rotation angle contained in rotation matrix constituting the weight matrix (See page 11, section 4.2 the symmetric part of the weights matrix w can be diagonalized by a properly chosen orthogonal matrix, that is by a rotation in a space of activation vector x, the weight matrix causes a rotation in the N-dimensional space of activation vectors around the point); and updating the rotation angle (See page 12, updating the rotation angle based on vector x(t) and b(t)).
Therefore, it would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention was made to modify the differential equation of Wang to associate with a loss function, taught by Ruseckas. Skilled artisan would have been motivated to improve visual object recognition, object detection and many other domains (See Ruseckas, page 1, Introduction). In addition, both references (Wang and Ruseckas) teach features that are directed to analogous art and they are directed to the same field of endeavor, such as using machine learning to optimize a large number of parameters. This close relation between both references highly suggests an expectation of success.
As to claim 5, Wang discloses wherein the program instructions cause the processor to: create a computational graph (See para. [0100] and Figure 5, a fixed graph structure) by using the formulated differential equations, and learn the parameters of the neural network by calculating values of the differential equations by forward propagation calculation and backpropagation calculation using the computational graph (See para. [0122], the building blocks including parametric continuous convolution layers can be differentiable operators, such that the networks can be learned through back propagation, according to equation).
As to claim 11, Wang in view of Ruseckas discloses calculate the differential equation of the loss function with respect to each of the conjugate variables by reusing a matrix element value utilized during a forward calculation of the neural network and converting the matrix element into a conjugal value (See Ruseckas, page 1, Forward propagation of discrete vector x 2 Rn through residual network can be written as x(l+1) = x(l) + F(x(l); q(l)); where q(l) are the parameters that need to be determined by the training of the neural network, a connection of residual network with recurrent neural network, which is known as an approximation of a dynamical system, has been made).
Therefore, it would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention was made to modify the differential equation of Wang by reusing a matrix element value, taught by Ruseckas. Skilled artisan would have been motivated to lead to faster training of a model (See Ruseckas, page 6). In addition, both references (Wang and Ruseckas) teach features that are directed to analogous art and they are directed to the same field of endeavor, such as using machine learning to optimize a large number of parameters. This close relation between both references highly suggests an expectation of success.
Claims 8 and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Wang (US 2019/0258335 A1) and in view of and in view of Ruseckas “Differential equations as models of deep neural networks, 2019” and further in view of Gomez (US 2022/0164655 A1).
As to claim 8, Wang does not explicitly disclose the weight matrix is a unitary matrix represented by a product of diagonal matrix and a plurality of rotation matrices.
Gomez discloses the weight matrix is a unitary matrix represented by a product of diagonal matrix and a plurality of rotation matrices (See para. [0008], weight matrices for linear layers in an attention module and in a feed-forward training module, each pair of query and key matrices in an attention module needs to take a dot product, note in para. [0037] and para. [0038], the weight matrix is sliced with slice mask which is a rotation of the slick mask).
Therefore, it would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention was made to modify the Wang system to include a weight matrix represented by a product of diagonal matrix, taught by Gomez. Skilled artisan would have been motivated to improve the training process of large models using fewer computing resources (See Gomez, para. [0009]). In addition, all references (Gomez, Wang and Ruseckas) teach features that are directed to analogous art and they are directed to the same field of endeavor, such as using machine learning to optimize a large number of parameters. This close relation between all references highly suggests an expectation of success.
As to claim 9, Wang does not explicitly disclose update the parameters by directly updating the rotation angle using the formulated differential equation without performing matrix product operations for the weight matrix.
Ruseckas discloses update the parameters by directly updating the rotation angle using the formulated differential equation without performing matrix product operations for the weight matrix (See page 2, Section 2, generating nonlinear ordinary differential equations of a loss function with parameters q(t), note page 12, updating the rotation angle based on vector x(t) and b(t))
Therefore, it would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention was made to modify the differential equation of Wang to update the rotation angle directly, taught by Ruseckas. Skilled artisan would have been motivated to lead to faster training of a model (See Ruseckas, page 6). In addition, both references (Wang and Ruseckas) teach features that are directed to analogous art and they are directed to the same field of endeavor, such as using machine learning to optimize a large number of parameters. This close relation between both references highly suggests an expectation of success.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YUK TING CHOI whose telephone number is (571)270-1637. The examiner can normally be reached Monday-Friday 9am-6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, AMY NG can be reached at 5712701698. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YUK TING CHOI/Primary Examiner, Art Unit 2164