DETAILED ACTION
This action is responsive to the amendment filed on 03/06/2026 for application 17/733,874 with claims filed 03/06/2026. Claims 1, 3-6, 8-11, 13-16 and 18-20 are pending in the case. Claims 1, 6, 11 and 16 are independent claims. Claims 2, 7, 12 and 17 are cancelled and claims 1, 6, 11 and 16 are amended.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 3-6, 8-11, 13-16 and 18-20 are rejected under 35 U.S.C. 101 as they are directed to an abstract idea and do not integrate into a practical application.
Regarding claim 1:
The claim recites modeling a neural network as a set of polynomials which is an abstract idea (Mathematical Calculations (see MPEP 2106.04(a)(2)(I)(C))) as the polynomials are optimization problems. Alternatively, under the broadest reasonable interpretation, the limitation covers performance of the limitation in the mind. The limitations encompass a user creating/choosing polynomials that represent a neural network. See 2106.04.(a)(2).III.C.
The claim recites relaxing the global polynomial optimization problem including polynomial constraints with a plurality of semi-definite programs which is an abstract idea (Mathematical Relationships (see MPEP 2106.04(a)(2)(I)(A)))).
The claim recites solving the plurality of semi-definite programs based on a pre-defined structure comprising at least one of a sparse structure, a hierarchical structure, a block Hankel-plus-Toeplitz structure, a low-rank structure, or an underlying geometry structure which is an abstract idea (Mathematical Calculations (see MPEP 2106.04(a)(2)(I)(C))).
Subject Matter Eligibility Analysis Step 2A Prong 2:
processor-implemented(merely recites a generic computer on which to perform the abstract idea, e.g. "apply it on a computer" (see MPEP 2106.05(f)))
receiving as input, a global polynomial optimization problem (which amount to mere extra solution activity of obtaining and/or gathering data over a network, see MPEP §2106.05(g))
that approximates a training problem of the neural network based on the set of polynomials(merely specifies a particular technological environment in which the abstract idea is to take place, i.e. a field of use (see MPEP 2106.05(h)))
the semi-definite programs having dimensions corresponding to a total number of weights in the neural network, with weights deeper in the neural network having a higher associated polynomial degree, a last layer in the neural network being a deepest layer and weights in the last layer having a highest associated polynomial degree(merely specifies a particular technological environment in which the abstract idea is to take place, i.e. a field of use (see MPEP 2106.05(h)))
outputting a solution indicating a location of a global optimum of the optimization problem(which amount to mere extra solution activity of obtaining and/or gathering data over a network, see MPEP §2106.05(g))
with the neural network(merely recites a generic computer on which to perform the abstract idea, e.g. "apply it on a computer" (see MPEP 2106.05(f)))
training the neural network with weights deeper in the neural network having a higher associated polynomial degree based on the solution(merely recites a generic computer on which to perform the abstract idea, e.g. "apply it on a computer" (see MPEP 2106.05(f)))
Subject Matter Eligibility Analysis Step 2B:
Additional elements (a) (f) and (g) do not integrate the abstract idea into a practical application nor do the additional limitation provide significantly more than the abstract idea because the limitation amount to no more than mere instructions to apply the exception using a generic computer component. Please see MPEP §2106.05(f).
Additional elements (c) and (d) do not integrate the abstract idea into a practical application nor do the additional limitation provide significantly more than the abstract idea because the limitation merely specifies a field of use in which the abstract idea is to take place, i.e. a field of use (see MPEP 2106.05(h)).
Additional element (b) and (e) recites receiving and sending inputs/outputs which is a well-understood, routine, and conventional activity of “transmitting or receiving data over a network" (see MPEP 2106.05(d)(II)(i) using the Internet to gather data, buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network))
The additional element(s) (a) (b) (c) (d) (e) (f) and (g) in the claim do/does not include any additional elements, when considered separately and in combination, that amount to an integration of the judicial exception into a practical application, nor significantly more than the judicial exception for the reasons set forth in step 2A prong 2 analysis above. The claim is not patent eligible.
Regarding claim 3:
Subject Matter Eligibility Analysis Step 2A Prong 1:
The claim recites solving further includes dimensionality reduction which is an abstract idea(Mathematical Calculations (see MPEP 2106.04(a)(2)(I)(C))).
The claim recites and/or using a warm start which is a mathematical concept ( as disclosed in para. [0067] of the specification).
Subject Matter Eligibility Analysis Step 2A Prong 2:
The claim does not contain elements that would warrant a Step 2A Prong 2 analysis.
Subject Matter Eligibility Analysis Step 2B:
The claim does not include any additional element, when considered separately and in combination, that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible.
Regarding claim 4:
Subject Matter Eligibility Analysis Step 2A Prong 1:
The claim recites global polynomial optimization problem is non-convex which is an abstract idea (Mathematical Calculations (see MPEP 2106.04(a)(2)(I)(C))).
Subject Matter Eligibility Analysis Step 2A Prong 2:
The claim does not contain elements that would warrant a Step 2A Prong 2 analysis.
Subject Matter Eligibility Analysis Step 2B:
The claim does not include any additional element, when considered separately and in combination, that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible.
Regarding claim 5:
Subject Matter Eligibility Analysis Step 2A Prong 1:
The claim recites representing the global polynomial optimization problem with a Chebyshev basis which is an abstract idea (Mathematical Relationships (see MPEP 2106.04(a)(2)(I)(A)))).
Subject Matter Eligibility Analysis Step 2A Prong 2:
The claim does not contain elements that would warrant a Step 2A Prong 2 analysis.
Subject Matter Eligibility Analysis Step 2B:
The claim does not include any additional element, when considered separately and in combination, that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible.
Regarding claim 6:
The claim recites to model a neural network as a set of polynomials which is an abstract idea (Mathematical Calculations (see MPEP 2106.04(a)(2)(I)(C))) as the polynomials are optimization problems. Alternatively, under the broadest reasonable interpretation, covers performance of the limitation in the mind. The limitations encompass a user creating/choosing polynomials that represent a neural network. See 2106.04.(a)(2).III.C.
The claim recites to relax the global polynomial optimization problem including polynomial constraints with a plurality of semi-definite programs which is an abstract idea (Mathematical Relationships (see MPEP 2106.04(a)(2)(I)(A)))).
The claim recites to solve the plurality of semi-definite programs based on a pre-defined structure comprising at least one of a sparse structure, a hierarchical structure, a block Hankel-plus-Toeplitz structure, a low-rank structure, or an underlying geometry structure which is an abstract idea (Mathematical Calculations (see MPEP 2106.04(a)(2)(I)(C))).
Subject Matter Eligibility Analysis Step 2A Prong 2:
a memory; and at least one processor coupled to the memory, the at least one processor(merely recites a generic computer on which to perform the abstract idea, e.g. "apply it on a computer" (see MPEP 2106.05(f)))
to receive as input, a global polynomial optimization problem (which amount to mere extra solution activity of obtaining and/or gathering data over a network, see MPEP §2106.05(g))
that approximates a training problem of the neural network based on the set of polynomials (merely specifies a particular technological environment in which the abstract idea is to take place, i.e. a field of use (see MPEP 2106.05(h)))
the semi-definite programs having dimensions corresponding to a total number of weights in the neural network, with weights deeper in the neural network having a higher associated polynomial degree, a last layer in the neural network being a deepest layer and weights in the last layer having a highest associated polynomial degree(merely specifies a particular technological environment in which the abstract idea is to take place, i.e. a field of use (see MPEP 2106.05(h)))
and output a solution indicating a location of a global optimum of the optimization problem (which amount to mere extra solution activity of obtaining and/or gathering data over a network, see MPEP §2106.05(g))
with the neural network(merely recites a generic computer on which to perform the abstract idea, e.g. "apply it on a computer" (see MPEP 2106.05(f)))
training of the neural network with weights deeper in the neural network having a higher associated polynomial degree based on the solution (merely recites a generic computer on which to perform the abstract idea, e.g. "apply it on a computer" (see MPEP 2106.05(f)))
Subject Matter Eligibility Analysis Step 2B:
Additional elements (a) (f) and (g) do not integrate the abstract idea into a practical application nor do the additional limitation provide significantly more than the abstract idea because the limitation amount to no more than mere instructions to apply the exception using a generic computer component. Please see MPEP §2106.05(f).
Additional elements (c) and (d) do not integrate the abstract idea into a practical application nor do the additional limitation provide significantly more than the abstract idea because the limitation merely specifies a field of use in which the abstract idea is to take place, i.e. a field of use (see MPEP 2106.05(h)).
Additional element (b) and (e) recites receiving and sending inputs/outputs which is a well-understood, routine, and conventional activity of “transmitting or receiving data over a network" (see MPEP 2106.05(d)(II)(i) using the Internet to gather data, buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network))
The additional element(s) (a) (b) (c) (d) (e) (f) and (g) in the claim do/does not include any additional elements, when considered separately and in combination, that amount to an integration of the judicial exception into a practical application, nor significantly more than the judicial exception for the reasons set forth in step 2A prong 2 analysis above. The claim is not patent eligible.
Claims 8-10 are rejected under that same 101 claim analysis due to the substantially similarity of the limitations and additional elements of claims 8-5 found in claims 8-10 respectively.
Regarding claim 11:
The claim recites means for modeling a neural network as a set of polynomials; which is an abstract idea (Mathematical Calculations (see MPEP 2106.04(a)(2)(I)(C))) as the polynomials are optimization problems. Alternatively, under the broadest reasonable interpretation, covers performance of the limitation in the mind. The limitations encompass a user creating/choosing polynomials that represent a neural network. See 2106.04.(a)(2).III.C.
The claim recites means for relaxing the global polynomial optimization problem including polynomial constraints with a plurality of semi-definite programs which is an abstract idea (Mathematical Relationships (see MPEP 2106.04(a)(2)(I)(A)))).
The claim recites means for solving the plurality of semi-definite programs based on a pre-defined structure comprising at least one of a sparse structure, a hierarchical structure, a block Hankel-plus-Toeplitz structure, a low-rank structure, or an underlying geometry structure which is an abstract idea (Mathematical Calculations (see MPEP 2106.04(a)(2)(I)(C))).
Subject Matter Eligibility Analysis Step 2A Prong 2:
with the neural network(merely recites a generic computer on which to perform the abstract idea, e.g. "apply it on a computer" (see MPEP 2106.05(f)))
means for receiving as input, a global polynomial optimization problem (which amount to mere extra solution activity of obtaining and/or gathering data over a network, see MPEP §2106.05(g))
that approximates a training problem of the neural network based on the set of polynomials (merely specifies a particular technological environment in which the abstract idea is to take place, i.e. a field of use (see MPEP 2106.05(h)))
the semi-definite programs having dimensions corresponding to a total number of weights in the neural network, with weights deeper in the neural network having a higher associated polynomial degree, a last layer in the neural network being a deepest layer and weights in the last layer having a highest associated polynomial degree(merely specifies a particular technological environment in which the abstract idea is to take place, i.e. a field of use (see MPEP 2106.05(h)))
means for outputting a solution indicating a location of a global optimum of the optimization problem (which amount to mere extra solution activity of obtaining and/or gathering data over a network, see MPEP §2106.05(g))
training the neural network with weights deeper in the neural network having a higher associated polynomial degree based on the solution (merely recites a generic computer on which to perform the abstract idea, e.g. "apply it on a computer" (see MPEP 2106.05(f)))
Subject Matter Eligibility Analysis Step 2B:
Additional elements (a) and (f) do not integrate the abstract idea into a practical application nor do the additional limitation provide significantly more than the abstract idea because the limitation amount to no more than mere instructions to apply the exception using a generic computer component. Please see MPEP §2106.05(f).
Additional elements (c) and (d) do not integrate the abstract idea into a practical application nor do the additional limitation provide significantly more than the abstract idea because the limitation merely specifies a field of use in which the abstract idea is to take place, i.e. a field of use (see MPEP 2106.05(h)).
Additional element (b) and (e) recites receiving and sending inputs/outputs which is a well-understood, routine, and conventional activity of “transmitting or receiving data over a network" (see MPEP 2106.05(d)(II)(i) using the Internet to gather data, buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network))
The additional element(s) (a) (b) (c) (d) (e) and (f) in the claim do/does not include any additional elements, when considered separately and in combination, that amount to an integration of the judicial exception into a practical application, nor significantly more than the judicial exception for the reasons set forth in step 2A prong 2 analysis above. The claim is not patent eligible.
Claims 13-15 are rejected under that same 101 claim analysis due to the substantially similarity of the limitations and additional elements of claims 3-5 found in claims 13-15 respectively.
Regarding claim 16:
The claim recites program code to model a neural network as a set of polynomials which is an abstract idea (Mathematical Calculations (see MPEP 2106.04(a)(2)(I)(C))) as the polynomials are optimization problems. Alternatively, under the broadest reasonable interpretation, covers performance of the limitation in the mind. The limitations encompass a user creating/choosing polynomials that represent a neural network. See 2106.04.(a)(2).III.C.
The claim recites program code to relax the global polynomial optimization problem including polynomial constraints with a plurality of semi-definite programs which is an abstract idea (Mathematical Relationships (see MPEP 2106.04(a)(2)(I)(A)))).
The claim recites program code to solve the plurality of semi-definite programs based on a pre-defined structure comprising at least one of a sparse structure, a hierarchical structure, a block Hankel-plus-Toeplitz structure, a low-rank structure, or an underlying geometry structure which is an abstract idea (Mathematical Calculations (see MPEP 2106.04(a)(2)(I)(C))).
The claim recites program code to train … based on the solution which is an abstract idea (Mathematical Calculations (see MPEP 2106.04(a)(2)(I)(C))).
Subject Matter Eligibility Analysis Step 2A Prong 2:
non-transitory computer-readable medium having program code recorded thereon, the program code executed by a processor (merely recites a generic computer on which to perform the abstract idea, e.g. "apply it on a computer" (see MPEP 2106.05(f)))
program code to receive as input, a global polynomial optimization problem(which amount to mere extra solution activity of obtaining and/or gathering data over a network, see MPEP §2106.05(g))
that approximates a training problem of the neural network based on the set of polynomials (merely specifies a particular technological environment in which the abstract idea is to take place, i.e. a field of use (see MPEP 2106.05(h)))
the semi-definite programs having dimensions corresponding to a total number of weights in the neural network, with weights deeper in the neural network having a higher associated polynomial degree, a last layer in the neural network being a deepest layer and weights in the last layer having a highest associated polynomial degree(merely specifies a particular technological environment in which the abstract idea is to take place, i.e. a field of use (see MPEP 2106.05(h)))
and output a solution indicating a location of a global optimum of the optimization problem (which amount to mere extra solution activity of obtaining and/or gathering data over a network, see MPEP §2106.05(g))
with the neural network(merely recites a generic computer on which to perform the abstract idea, e.g. "apply it on a computer" (see MPEP 2106.05(f)))
train the neural network with weights deeper in the neural network having a higher associated polynomial degree based on the solution (merely recites a generic computer on which to perform the abstract idea, e.g. "apply it on a computer" (see MPEP 2106.05(f)))
Subject Matter Eligibility Analysis Step 2B:
Additional elements (a) (f) and (g) do not integrate the abstract idea into a practical application nor do the additional limitation provide significantly more than the abstract idea because the limitation amount to no more than mere instructions to apply the exception using a generic computer component. Please see MPEP §2106.05(f).
Additional elements (c) and (d) do not integrate the abstract idea into a practical application nor do the additional limitation provide significantly more than the abstract idea because the limitation merely specifies a field of use in which the abstract idea is to take place, i.e. a field of use (see MPEP 2106.05(h)).
Additional element (b) and (e) recites receiving and sending inputs/outputs which is a well-understood, routine, and conventional activity of “transmitting or receiving data over a network" (see MPEP 2106.05(d)(II)(i) using the Internet to gather data, buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network))
The additional element(s) (a) (b) (c) (d) (e) (f) and (g) in the claim do/does not include any additional elements, when considered separately and in combination, that amount to an integration of the judicial exception into a practical application, nor significantly more than the judicial exception for the reasons set forth in step 2A prong 2 analysis above. The claim is not patent eligible.
Claims 18-20 are rejected under that same 101 claim analysis due to the substantially similarity of the limitations and additional elements of claims 3-5 found in claims 18-20 respectively.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1, 3-4, 6, 8-9, 11, 13-14, 16 and 18-19 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Bartan et al.(Neural Spectrahedra and Semidefinite Lifts: Global Convex Optimization of Polynomial Activation Neural Networks in Fully Polynomial-Time).
Regarding claim 1:
Bartan discloses processor-implemented(Bartan, ABSTRACT, “Remarkably, we show that semidefinite lifting is always exact and therefore computational complexity for global optimization is polynomial in the input dimension and sample size for all input data” where using computation complexity as a measurement implies the method is completed on a computer/processor)
Bartan discloses modeling a neural network as a set of polynomials(Bartan, Page 7, Paragraph 5 and Equation 10, “Consider the network f(x) =
∑
j
=
1
m
σ
(
x
T
u
j
)
α
j
where the activation function σ is the degree two polynomial σ(u) = au2 + bu + c. First, we note that the neural network output can be written as [Equation 10]” where the conversion of the neural network with degree-2 polynomial activation into polynomial form is considered modeling a neural network as a set of polynomials.
Bartan discloses receiving as input, a global polynomial optimization problem that approximates a training problem of the neural network based on the set of polynomials(Bartan, Page 8, Definition 1, “we will show that the original non-convex neural network problem is solved exactly to global optimality when the optimization is performed over a convex set which we define as the Neural Spectrahedron” where the original non-convex neural network problem to be solved via global optimality using optimization over a convex set is considered receiving a global polynomial optimization problem that approximates a training problem of the neural network based on the set of polynomials.)
Bartan discloses relaxing the global polynomial optimization problem including polynomial constraints with a plurality of semi-definite programs(Bartan, Page 8, Definition 1, “ we will show that the original non-convex neural network problem is solved exactly to global optimality when the optimization is performed over a convex set which we define as the Neural Spectrahedron… Moreover, a Neural Spectrahedron can be described by a simple linear matrix inequality” where the original polynomial NN training problem is reformulated as an semi-definite program by representing the feasible parameter set (Neural Spectrahedron) via LMIs is considered relaxing a polynomial optimization problem including polynomial constraints with a plurality of semi-definite programs(See also: “It is equally important that our results characterize neural networks as constrained linear learning methods <φ(x), ψ> in the lifted feature space φ(x), where the constraints on the lifted parameters ψ are precisely described by a Neural Spectrahedron via linear matrix inequalities. These constraints can be easily tackled with convex semidefinite programming or closed-form projections onto these sets in iterative first-order algorithms”), the semi-definite programs having dimensions corresponding to a total number of weights in the neural network(Bartan, Page 7, Equation 10 and Bartan, Page 14, Theorem 3.1, where the SDP having dimensions corresponding to the total number of weights in the neural network as shown in Equation 10 and Theorem 3.1. In the lifted formulation, the optimization variables of PSD matrices Z, Z` of size (d+1) * (d+1) are used in the SDP that have the weights uj and αj encoded from the neural network parameters which is considered having semi-definite programs having dimensions corresponding to a total number of weights in the neural network(See also: Eq. 14 and 18 for PSD matrix construction)), with weights deeper in the neural network having a higher associated polynomial degree, a last layer in the neural network being a deepest layer and weights in the last layer having a highest associated polynomial degree(Barton, Page 3, Paragraph 1, where the training uses output of hidden layer σ(xTuj) after applying its hidden-layer weights uj and corresponds to one degree less than the weighted output contribution of the output layer σ(xTuj)αj and both the outputs use the weights of the hidden and output layer they both correspond to the having weights deeper in the neural network having a higher associated polynomial degreeI
Bartan discloses solving the plurality of semi-definite programs based on a pre-defined structure(Bartan, Page 9, Paragraph 2, “It is equally important that our results characterize neural networks as constrained linear learning methods <φ(x), ψ> in the lifted feature space φ(x), where the constraints on the lifted parameters ψ are precisely described by a Neural Spectrahedron via linear matrix inequalities. These constraints can be easily tackled with convex semidefinite programming or closed-form projections onto these sets in iterative first-order algorithms” where solving constraints described by the LMI of the Neural Spectrahedron using semidefinite programs is considered solving semi-definite programs based on a pre-defined structure) comprising at least one of a sparse structure, a hierarchical structure, a block Hankel- plus-Toeplitz structure, a low-rank structure, or an underlying geometry structure(Bartan, Page 9, Equation 14 and Paragraph 4, “We will show that a neural spectrahedron can be equivalently described as a linear matrix inequality
PNG
media_image1.png
24
458
media_image1.png
Greyscale
” where the neural spectrahedral is a geometrically defined convex set) and outputting a solution indicating a location of a global optimum of the optimization problem(Bartan, Page 14, Theorem 3.1, Equations 21 and 22, Paragraph 1, “In this section we show that the solution of the convex program (21) provides a lower bound for the solution of the non convex problem (22).” where the SDP is convex and exactly reformulates the original non-convex neural network training problem and the SDP yields Z*, Z`* which represents a globally optimal solution for the original non-convex polynomial optimization) and training the neural network with weights deeper in the neural network having the higher associated polynomial degree(Bartan, Page 11, equation 20, where equation 20 shows the outputs of the weights are used in training as shown in the minimizing the loss based on the prediction) based on the solution(Bartan, Page 18, Paragraph 1, " Decomposing the solution of the convex problem Z* and Z`* onto these cones, i.e., neural decomposition, enables the construction of neural network weights from Z* and Z`*" where after neural network decomposition and solving Z, Z`* the neural network and weights are reconstructed which is considered performing using the solution with the neural network to perform training)
Regarding claim 3:
The rejection of Bartan discloses the processor-implemented method of claim 1 is incorporated and further:
Bartan further discloses in which solving further includes dimensionality reduction, and/or using a warm start(Equation 10, Theorem 3.1, and , “The above identity shows that the nonlinear neural network output is linear over the lifted features
PNG
media_image2.png
35
379
media_image2.png
Greyscale
where the transformation of the original space into a lower-dimensional convex space with fixed size make to make solving in polynomial time feasible is considered dimensionality reduction(i.e. going from original space ℝm(d+1) to two PSD matrixes with fixed dimension (d+1)2). As the maximum neuron count (and thus maximum number of weight vectors) is directly bounded by the SDP matrix dimension from theorem 3.1(“It follows that the optimal number of neurons is upper bounded by m∗ ≤ 2(d + 1)”) the large space of m(d+1) parameters collapses to two PSD matrixes of size (d+1)2 independent of m)
Regarding claim 4:
The rejection of Bartan discloses the processor-implemented method of claim 1 is incorporated and further:
Bartan further discloses in which the global polynomial optimization problem is non-convex(“Remarkably, it can be shown that the original non-convex problem in (5) can be solved exactly by solving the convex SDP” where the non-convex problem is considered a non-convex global polynomial optimization)
Regarding claim 6:
Bartan discloses a memory; and at least one processor coupled to the memory, the at least one processor(Bartan, ABSTRACT, “Remarkably, we show that semidefinite lifting is always exact and therefore computational complexity for global optimization is polynomial in the input dimension and sample size for all input data” where using computation complexity as a measurement implies the method is completed on a computer/processor)
Bartan discloses modeling a neural network as a set of polynomials(Bartan, Page 7, Paragraph 5 and Equation 10, “Consider the network f(x) =
∑
j
=
1
m
σ
(
x
T
u
j
)
α
j
where the activation function σ is the degree two polynomial σ(u) = au2 + bu + c. First, we note that the neural network output can be written as [Equation 10]” where the conversion of the neural network with degree-2 polynomial activation into polynomial form is considered modeling a neural network as a set of polynomials.
Bartan discloses receiving as input, a global polynomial optimization problem that approximates a training problem of the neural network based on the set of polynomials(Bartan, Page 8, Definition 1, “we will show that the original non-convex neural network problem is solved exactly to global optimality when the optimization is performed over a convex set which we define as the Neural Spectrahedron” where the original non-convex neural network problem to be solved via global optimality using optimization over a convex set is considered receiving a global polynomial optimization problem that approximates a training problem of the neural network based on the set of polynomials.)
Bartan discloses to relax the global polynomial optimization problem including polynomial constraints with a plurality of semi-definite programs(Bartan, Page 8, Definition 1, “ we will show that the original non-convex neural network problem is solved exactly to global optimality when the optimization is performed over a convex set which we define as the Neural Spectrahedron… Moreover, a Neural Spectrahedron can be described by a simple linear matrix inequality” where the original polynomial NN training problem is reformulated as an semi-definite program by representing the feasible parameter set (Neural Spectrahedron) via LMIs is considered relaxing a polynomial optimization problem including polynomial constraints with a plurality of semi-definite programs(See also: “It is equally important that our results characterize neural networks as constrained linear learning methods <φ(x), ψ> in the lifted feature space φ(x), where the constraints on the lifted parameters ψ are precisely described by a Neural Spectrahedron via linear matrix inequalities. These constraints can be easily tackled with convex semidefinite programming or closed-form projections onto these sets in iterative first-order algorithms”), the semi-definite programs having dimensions corresponding to a total number of weights in the neural network(Bartan, Page 7, Equation 10 and Bartan, Page 14, Theorem 3.1, where the SDP having dimensions corresponding to the total number of weights in the neural network as shown in Equation 10 and Theorem 3.1. In the lifted formulation, the optimization variables of PSD matrices Z, Z` of size (d+1) * (d+1) are used in the SDP that have the weights uj and αj encoded from the neural network parameters which is considered having semi-definite programs having dimensions corresponding to a total number of weights in the neural network(See also: Eq. 14 and 18 for PSD matrix construction)), with weights deeper in the neural network having a higher associated polynomial degree, a last layer in the neural network being a deepest layer and weights in the last layer having a highest associated polynomial degree(Barton, Page 3, Paragraph 1, where the training uses output of hidden layer σ(xTuj) after applying its hidden-layer weights uj and corresponds to one degree less than the weighted output contribution of the output layer σ(xTuj)αj and both the outputs use the weights of the hidden and output layer they both correspond to the having weights deeper in the neural network having a higher associated polynomial degree.)
Bartan discloses to solve the plurality of semi-definite programs based on a pre-defined structure(Bartan, Page 9, Paragraph 2, “It is equally important that our results characterize neural networks as constrained linear learning methods <φ(x), ψ> in the lifted feature space φ(x), where the constraints on the lifted parameters ψ are precisely described by a Neural Spectrahedron via linear matrix inequalities. These constraints can be easily tackled with convex semidefinite programming or closed-form projections onto these sets in iterative first-order algorithms” where solving constraints described by the LMI of the Neural Spectrahedron using semidefinite programs is considered solving semi-definite programs based on a pre-defined structure) comprising at least one of a sparse structure, a hierarchical structure, a block Hankel- plus-Toeplitz structure, a low-rank structure, or an underlying geometry structure(Bartan, Page 9, Equation 14 and Paragraph 4, “We will show that a neural spectrahedron can be equivalently described as a linear matrix inequality
PNG
media_image1.png
24
458
media_image1.png
Greyscale
” where the neural spectrahedral is a geometrically defined convex set) and outputting a solution indicating a location of a global optimum of the optimization problem(Bartan, Page 14, Theorem 3.1, Equations 21 and 22, Paragraph 1, “In this section we show that the solution of the convex program (21) provides a lower bound for the solution of the non convex problem (22).” where the SDP is convex and exactly reformulates the original non-convex neural network training problem and the SDP yields Z*, Z`* which represents a globally optimal solution for the original non-convex polynomial optimization) and training the neural network with weights deeper in the neural network having the higher associated polynomial degree(Bartan, Page 11, equation 20, where equation 20 shows the outputs of the weights are used in training as shown in the minimizing the loss based on the prediction) based on the solution(Bartan, Page 18, Paragraph 1, " Decomposing the solution of the convex problem Z* and Z`* onto these cones, i.e., neural decomposition, enables the construction of neural network weights from Z* and Z`*" where after neural network decomposition and solving Z, Z`* the neural network and weights are reconstructed which is considered performing using the solution with the neural network to perform training)
Regarding claims 8-9:
The rejection of claim 6 is incorporated in claims 8-9. Claims 8-9 are rejected under the same rationale as set forth in the rejection of claim 3-4.
Regarding claim 11:
Bartan discloses means for a neural network as a set of polynomials(Bartan, Page 7, Paragraph 5 and Equation 10, “Consider the network f(x) =
∑
j
=
1
m
σ
(
x
T
u
j
)
α
j
where the activation function σ is the degree two polynomial σ(u) = au2 + bu + c. First, we note that the neural network output can be written as [Equation 10]” where the conversion of the neural network with degree-2 polynomial activation into polynomial form is considered modeling a neural network as a set of polynomials.
Bartan discloses means for receiving as input, a global polynomial optimization problem that approximates a training problem of the neural network based on the set of polynomials(Bartan, Page 8, Definition 1, “we will show that the original non-convex neural network problem is solved exactly to global optimality when the optimization is performed over a convex set which we define as the Neural Spectrahedron” where the original non-convex neural network problem to be solved via global optimality using optimization over a convex set is considered receiving a global polynomial optimization problem that approximates a training problem of the neural network based on the set of polynomials.)
Bartan discloses means for relaxing the global polynomial optimization problem including polynomial constraints with a plurality of semi-definite programs(Bartan, Page 8, Definition 1, “ we will show that the original non-convex neural network problem is solved exactly to global optimality when the optimization is performed over a convex set which we define as the Neural Spectrahedron… Moreover, a Neural Spectrahedron can be described by a simple linear matrix inequality” where the original polynomial NN training problem is reformulated as an semi-definite program by representing the feasible parameter set (Neural Spectrahedron) via LMIs is considered relaxing a polynomial optimization problem including polynomial constraints with a plurality of semi-definite programs(See also: “It is equally important that our results characterize neural networks as constrained linear learning methods <φ(x), ψ> in the lifted feature space φ(x), where the constraints on the lifted parameters ψ are precisely described by a Neural Spectrahedron via linear matrix inequalities. These constraints can be easily tackled with convex semidefinite programming or closed-form projections onto these sets in iterative first-order algorithms”), the semi-definite programs having dimensions corresponding to a total number of weights in the neural network(Bartan, Page 7, Equation 10 and Bartan, Page 14, Theorem 3.1, where the SDP having dimensions corresponding to the total number of weights in the neural network as shown in Equation 10 and Theorem 3.1. In the lifted formulation, the optimization variables of PSD matrices Z, Z` of size (d+1) * (d+1) are used in the SDP that have the weights uj and αj encoded from the neural network parameters which is considered having semi-definite programs having dimensions corresponding to a total number of weights in the neural network(See also: Eq. 14 and 18 for PSD matrix construction)), with weights deeper in the neural network having a higher associated polynomial degree, a last layer in the neural network being a deepest layer and weights in the last layer having a highest associated polynomial degree(Barton, Page 3, Paragraph 1, where the training uses output of hidden layer σ(xTuj) after applying its hidden-layer weights uj and corresponds to one degree less than the weighted output contribution of the output layer σ(xTuj)αj and both the outputs use the weights of the hidden and output layer they both correspond to the having weights deeper in the neural network having a higher associated polynomial degree)
Bartan discloses means for solving the plurality of semi-definite programs based on a pre-defined structure(Bartan, Page 9, Paragraph 2, “It is equally important that our results characterize neural networks as constrained linear learning methods <φ(x), ψ> in the lifted feature space φ(x), where the constraints on the lifted parameters ψ are precisely described by a Neural Spectrahedron via linear matrix inequalities. These constraints can be easily tackled with convex semidefinite programming or closed-form projections onto these sets in iterative first-order algorithms” where solving constraints described by the LMI of the Neural Spectrahedron using semidefinite programs is considered solving semi-definite programs based on a pre-defined structure) comprising at least one of a sparse structure, a hierarchical structure, a block Hankel- plus-Toeplitz structure, a low-rank structure, or an underlying geometry structure(Bartan, Page 9, Equation 14 and Paragraph 4, “We will show that a neural spectrahedron can be equivalently described as a linear matrix inequality
PNG
media_image1.png
24
458
media_image1.png
Greyscale
” where the neural spectrahedral is a geometrically defined convex set) and means for outputting a solution indicating a location of a global optimum of the optimization problem(Bartan, Page 14, Theorem 3.1, Equations 21 and 22, Paragraph 1, “In this section we show that the solution of the convex program (21) provides a lower bound for the solution of the non convex problem (22).” where the SDP is convex and exactly reformulates the original non-convex neural network training problem and the SDP yields Z*, Z`* which represents a globally optimal solution for the original non-convex polynomial optimization) and means for training the neural network with weights deeper in the neural network having the higher associated polynomial degree(Bartan, Page 11, equation 20, where equation 20 shows the outputs of the weights are used in training as shown in the minimizing the loss based on the prediction) based on the solution(Bartan, Page 18, Paragraph 1, " Decomposing the solution of the convex problem Z* and Z`* onto these cones, i.e., neural decomposition, enables the construction of neural network weights from Z* and Z`*" where after neural network decomposition and solving Z, Z`* the neural network and weights are reconstructed which is considered performing using the solution with the neural network to perform training)
Regarding claims 13-14:
The rejection of claim 11 is incorporated in claims 13-14. Claims 13-14 are rejected under the same rationale as set forth in the rejection of claim 3-4.
Regarding claim 16:
Bartan discloses A non-transitory computer-readable medium having program code recorded thereon, the program code executed by a processor(Bartan, ABSTRACT, “Remarkably, we show that semidefinite lifting is always exact and therefore computational complexity for global optimization is polynomial in the input dimension and sample size for all input data” where using computation complexity as a measurement implies the method is completed on a computer/processor)
Bartan discloses program code to model a neural network as a set of polynomials(Bartan, Page 7, Paragraph 5 and Equation 10, “Consider the network f(x) =
∑
j
=
1
m
σ
(
x
T
u
j
)
α
j
where the activation function σ is the degree two polynomial σ(u) = au2 + bu + c. First, we note that the neural network output can be written as [Equation 10]” where the conversion of the neural network with degree-2 polynomial activation into polynomial form is considered modeling a neural network as a set of polynomials.
Bartan discloses program code to receive as input, a global polynomial optimization problem that approximates a training problem of the neural network based on the set of polynomials(Bartan, Page 8, Definition 1, “we will show that the original non-convex neural network problem is solved exactly to global optimality when the optimization is performed over a convex set which we define as the Neural Spectrahedron” where the original non-convex neural network problem to be solved via global optimality using optimization over a convex set is considered receiving a global polynomial optimization problem that approximates a training problem of the neural network based on the set of polynomials.)
Bartan discloses program code to relaxing the global polynomial optimization problem including polynomial constraints with a plurality of semi-definite programs(Bartan, Page 8, Definition 1, “ we will show that the original non-convex neural network problem is solved exactly to global optimality when the optimization is performed over a convex set which we define as the Neural Spectrahedron… Moreover, a Neural Spectrahedron can be described by a simple linear matrix inequality” where the original polynomial NN training problem is reformulated as an semi-definite program by representing the feasible parameter set (Neural Spectrahedron) via LMIs is considered relaxing a polynomial optimization problem including polynomial constraints with a plurality of semi-definite programs(See also: “It is equally important that our results characterize neural networks as constrained linear learning methods <φ(x), ψ> in the lifted feature space φ(x), where the constraints on the lifted parameters ψ are precisely described by a Neural Spectrahedron via linear matrix inequalities. These constraints can be easily tackled with convex semidefinite programming or closed-form projections onto these sets in iterative first-order algorithms”), the semi-definite programs having dimensions corresponding to a total number of weights in the neural network(Bartan, Page 7, Equation 10 and Bartan, Page 14, Theorem 3.1, where the SDP having dimensions corresponding to the total number of weights in the neural network as shown in Equation 10 and Theorem 3.1. In the lifted formulation, the optimization variables of PSD matrices Z, Z` of size (d+1) * (d+1) are used in the SDP that have the weights uj and αj encoded from the neural network parameters which is considered having semi-definite programs having dimensions corresponding to a total number of weights in the neural network(See also: Eq. 14 and 18 for PSD matrix construction)), with weights deeper in the neural network having a higher associated polynomial degree, a last layer in the neural network being a deepest layer and weights in the last layer having a highest associated polynomial degree(Barton, Page 3, Paragraph 1, where the training uses output of hidden layer σ(xTuj) after applying its hidden-layer weights uj and corresponds to one degree less than the weighted output contribution of the output layer σ(xTuj)αj and both the outputs use the weights of the hidden and output layer they both correspond to the having weights deeper in the neural network having a higher associated polynomial degree)
Bartan discloses program code to solve the plurality of semi-definite programs based on a pre-defined structure(Bartan, Page 9, Paragraph 2, “It is equally important that our results characterize neural networks as constrained linear learning methods <φ(x), ψ> in the lifted feature space φ(x), where the constraints on the lifted parameters ψ are precisely described by a Neural Spectrahedron via linear matrix inequalities. These constraints can be easily tackled with convex semidefinite programming or closed-form projections onto these sets in iterative first-order algorithms” where solving constraints described by the LMI of the Neural Spectrahedron using semidefinite programs is considered solving semi-definite programs based on a pre-defined structure) comprising at least one of a sparse structure, a hierarchical structure, a block Hankel- plus-Toeplitz structure, a low-rank structure, or an underlying geometry structure(Bartan, Page 9, Equation 14 and Paragraph 4, “We will show that a neural spectrahedron can be equivalently described as a linear matrix inequality
PNG
media_image1.png
24
458
media_image1.png
Greyscale
” where the neural spectrahedral is a geometrically defined convex set) and means for outputting a solution indicating a location of a global optimum of the optimization problem(Bartan, Page 14, Theorem 3.1, Equations 21 and 22, Paragraph 1, “In this section we show that the solution of the convex program (21) provides a lower bound for the solution of the non convex problem (22).” where the SDP is convex and exactly reformulates the original non-convex neural network training problem and the SDP yields Z*, Z`* which represents a globally optimal solution for the original non-convex polynomial optimization) and means for training the neural network with weights deeper in the neural network having the higher associated polynomial degree(Bartan, Page 11, equation 20, where equation 20 shows the outputs of the weights are used in training as shown in the minimizing the loss based on the prediction) based on the solution(Bartan, Page 18, Paragraph 1, " Decomposing the solution of the convex problem Z* and Z`* onto these cones, i.e., neural decomposition, enables the construction of neural network weights from Z* and Z`*" where after neural network decomposition and solving Z, Z`* the neural network and weights are reconstructed which is considered performing using the solution with the neural network to perform training)
Regarding claims 18-19:
The rejection of claims 16 is incorporated in claims 18-19. Claims 18-19 are rejected under the same rationale as set forth in the rejection of claim 3-4.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 5, 10, 15 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Bartan et al.( Neural Spectrahedra and Semidefinite Lifts: Global Convex Optimization of Polynomial Activation Neural Networks in Fully Polynomial-Time) henceforth known as
Bartan and further in view of Henrion et al.(Solving nonconvex optimization problems) henceforth known as Henrion.
Regarding claim 5:
The rejection of Bartan discloses the processor-implemented method of claim 1 is incorporated and further:
Bartan does not explicitly disclose, however Henrion does disclose representing the global polynomial optimization problem with a Chebyshev basis(Henrion, Page 75, Col. 1, Paragraph 7, “…to construct a sequence of convex semidefinite programming(SDP) or LMI relaxations of P”) where the sequence of convex semidefinite programming(SDP) or LMI relaxations of P is considered a global polynomial optimization problem and Henrion, Page 82, Col. 2, Paragraph 2, “Alternative bases such as Chebyshev polynomials might also prove useful” where the disclosure of Chebyshev bases(plural of basis) as an alternative basis for the SDP and relaxation is considered using a Chebyshev basis with the global polynomial optimization problem)
References Bartan and Henrion are analogous art because they are from the using constraints in SDP convex problem optimizations.
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art, having the teachings of Bartan and Henrion before him or her, to modify the polynomial basis of Bartan to include the Chebyshev basis mentioned in Henrion as the alternative bases could be better conditioned for nonconvex SDP. The Suggestion/motivation for doing so would have been Henrion, Page 82, Col. 2, Paragraph 2, “It is well known that problems involving polynomial bases with monomials of increasing powers are naturally badly conditioned … Alternative bases such as Chebyshev polynomials might also prove useful.”
Regarding claims 10:
The rejection of claim 6 is incorporated in claim 10. Claim 10 is rejected under the same rationale as set forth in the rejection of claim 5.
Regarding claims 15:
The rejection of claim 11 is incorporated in claim 15. Claim 15 is rejected under the same rationale as set forth in the rejection of claim 5.
Regarding claims 20:
The rejection of claims 16 is incorporated in claim 20. Claim 20 is rejected under the same rationale as set forth in the rejection of claim 5.
Relevant Art
The following arts were found to be relevant as found art deal use semi definite problems while handing higher polynomials at deeper layers:
Kileel et al(“On the Expressive Power of Deep Polynomial Neural Networks”)
Zeyuan Allen-Zhu et. al (“Backward Feature Correction: How Deep Learning Performs Deep Learning”)
Response to Arguments
Applicant's arguments filed 03/06/2026 have been fully considered but they are not persuasive. A breakdown of the arguments can be found below:
101:
Applicant appears to argue that the amended limitation of training the neural network with weights deeper in the neural network having the higher associated polynomial degree ties training with layers of the network and therefore overcomes previous 101 objection.
Examiner respectfully disagrees as, after careful consideration, the limitation concerning weights deeper in the neural network having the higher associated polynomial degree does not overcome 101 as the weights as recited only limits the mathematical function. Amended language only recites specifics of the mathematical function recited in the claim and only limits the abstract idea further. The weights as recited do not add any specificity to the training process and training is recited generically as claims do not specify how the training is applied to layers or how the training is different from generic training concerning the weight and merely clarifies the neural network has weights with higher associated polynomial degrees based on weight depth but does not state how the neural network handles or uses those layers with higher associated polynomial degrees during training.
102/103:
Applicant appears to argue Bartan does not disclose solving an SDP "based on a pre-defined structure comprising at least one of a sparse structure, a hierarchical structure, a block Hankel- plus-Toeplitz structure, a low-rank structure, or an underlying geometry structure.
Examiner respectfully disagrees as Bartan SDP’s are solved based on semidefinite lifting of the network into a convex spectrahedral/LMI representation, which corresponds to underlying geometry structure.
Applicant appears to argue Bartan does not disclose "training the neural network with weights deeper in the neural network having the higher associated polynomial degree based on the solution."
Examiner respectfully disagrees as Bartan explains in page 3, paragraph 1 that the training uses output of hidden layer σ(xTuj) after applying its hidden-layer weights uj and corresponds to one degree less than the weighted output contribution of the output layer σ(xTuj)αj. As both the outputs use the weights of the hidden and output layer they both correspond to the having weights deeper in the neural network having a higher associated polynomial degree. The output layer corresponds to a deeper layer as it is after the sequential order of the hidden layer and the outputs of the weights are used in training as shown in the minimizing the loss based on the prediction in Page 11, equation 20.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHARLES JEFFREY JONES JR whose telephone number is (703)756-1414. The examiner can normally be reached Monday - Friday 8:00 - 5:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kakali Chaki can be reached at 571-272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/C.J.J./Examiner, Art Unit 2122
/KAKALI CHAKI/Supervisory Patent Examiner, Art Unit 2122
Prosecution Insights