Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The office action is responsive to the amendment filed on 02/09/2026 . The status of the claims is as follows: claims 1-3 and 5-20 are amended. Claims 1-20 are pending for examination.
Response to Arguments
Regarding the 35 U.S.C § 112 rejection:
Applicant’s arguments, see pg. 7, filed 02/09/2026 , with respect to objection to claims 1-20 have been fully considered and are persuasive. The rejection of claims 1-20 has been withdrawn.
Regarding the 35 U.S.C § 101 Rejection:
Applicant's further arguments see pg. 7-13, filed 02/09/2026, have been fully considered but they are not persuasive.
APPLICANT ARGUMENT:
Applicant argues, “the present rejections under 35 U.S.C. § 101 require reconsideration under the applicable standards set forth in the Desjardins Memo and that, when such standards are properly applied, the present rejections must be withdrawn”. In addition, under Step 2A: Prong One applicant argues, the claim features of independent claim 1 “define a training process in which neural network weight vectors are iteratively updated using a learning rate and reused across iterations until convergence is achieved”. Thus, the “training behavior reflects an iterative training process in which updated model parameters are reused across iterations, not a mental process or a mathematical concept”. In addition, applicant argues, claim 1 is directed to a specific improvement in a technical field as the claimed features of claim 1, improve how the machine learning model is trained. Moreover, under Step 2A, Prong Two applicant argue that independent claims 1, 12, and 15, integrate any alleged judicial exception into a practical application and are therefore patent- eligible. Lastly, applicant argues “the interaction of all claim limitations must be evaluated together, rather than viewing the limitations in isolation”. Accordantly, applicant submit Independent claims 1, 12 and 15 are therefore patent-eligible under Step 2A, Prong One and Prong Two. Dependent claims 2-11, 13, 14, and 16-20 are patentable at least by virtue of their dependencies on patentable independent claims 1, 12, and 15. Reconsideration and withdrawal of the rejections, and allowance of the claims is therefore respectfully requested.
EXAMINER RESPONSE: Examiner respectfully disagree; applicant argument is not persuasive. Amended independent claim 1 is rejected under 35 US.C. § 101 because the claims recites the limitation of:
selecting an initial weight vector for a convex optimization sub-problem associated with a neural network having a non-convex network architecture loss surface;
which might be practically performed in the human mind using observation, evaluation, judgment, and opinion. For example, the claimed selecting an initial weight vector encompasses judgment and evaluation, such as selecting a specific weight to solve an convex optimization sub-problem . And further recites the following limitations:
...approximating a solution to the convex optimization sub- problem that obtains a search direction, to learn a classifier from training data;
...updating the initial weight vector by subtracting the approximate solution to the convex optimization sub-problem times a first learning rate; and
...repeating the approximating and updating steps, for a plurality of iterations, with the updated weight vector from a given one of the iterations taken as the initial weight vector for a next one of the iterations, to obtain a final weight vector for the neural network, until convergence to a global minimum is achieved, to implement the classifier.
which under the broadest reasonable interpretation of approximation and updating steps requires specific mathematical calculations (basic calculus and mathematical equations) to perform the training of the neural network and therefore encompasses mathematical concepts. Therefore, the additional elements of: “with at least one processor “…to learn a classifier from training data” and “to implement the classifier” as disclosed above alone or in combination of the abstract idea are not sufficient to amount to significantly more than the judicial exception as they are mere generic computer functions being implemented with generic computer elements in a high level of generality to perform the disclosed abstract idea above.
Regarding applicant argument under Step 2A: Prong One: First, examiner disagree with applicant arguments that the “ iterative process is a computer-implemented function and not a mathematical calculation” as when the claims is evaluated as a whole in light of the specification it recite a mathematical calculation derived from basic calculus ( see [0051] of the instant application) and also recites various mathematical functions ( e.g., equation 8 of the instant application). Moreover, examiner will like to emphasize the “repetitive calculations” as those disclosed in claim 1, can also be interpreted as an extra solution activity of well understood routine conventional activity as identified by court, MPEP 2106.05(d)(II). Second, the solution as claimed in claim 1 and as disclosed in specification requires specific mathematical calculations (basic calculus and mathematical equations) to perform the training of the neural network and therefore encompasses mathematical concepts which is an abstract idea. Third, claim 1 does not recite "An improvement in the functioning of a computer, or an improvement to other technology or technical field" claim 1 does not specify the improvement nor is it disclosed in the claim. Rather, the claim recites an improvement of an abstract idea such as selecting a specific weight to solve an convex optimization sub-problem and performing approximation and updating steps which requires specific mathematical calculations. The applicant is reminded that any improvement should be mentioned in the claim language (See MPEP 2106.05 (a)). Fourth, as stated in the “ the USPTO Memo on "Advance notice of change to the MPEP in light of Ex Parte Desjardins" of December 5, 2025 (hereinafter referred to as the "USPTO Memo") the “updates are not intended to announce any new USPTO practice or procedure and are meant to be consistent with existing USPTO guidance”. Regarding applicant argument, the claims presented for examination are not analogues to the Ex Parte Desjardins, Appeal No. 2024-000567 (hereinafter Ex Parte Desjardins) as the claims do not recite "an improvement in the functioning of a computer, or an improvement to other technology or technical field", do not specify the improvement nor is disclosed in the claim. Rather the claim as presented for examination, do not specify the technical features or details for how the computer functionality is being improved, in the Ex Parte Desjardins, the claims were found to be patent eligible because when evaluating the claim as whole, the limitation of independent claim 1, constituted an improvement to how the machine learning model itself operated, and not, for example, the identified mathematical calculation (emphasis added). However, the claims of the instant applicant do not reflect such improvement to the machine learning model, instead independent claim 1 recites claim language in a generic manner which appear to be an improvements of an abstract idea and mathematical concept. The following paragraph from the USPTO Memo regarding the Ex Parte Desjardins, recites:
“The Appeals Review Panel (ARP) overall credited benefits including reduced storage, reduced system complexity and streamlining, and preservation of performance attributes associated with earlier tasks during subsequent computational tasks as technological improvements that were disclosed in the patent application specification. Specifically, the ARP upheld the Step 2A Prong One finding that the claims recited an abstract idea (i.e., mathematical concept). In Step 2A Prong Two, the ARP then determined that the specification identified improvements as to how the machine learning model itself operates, including training a machine learning model to learn new tasks while protecting knowledge about previous tasks to overcome the problem of “catastrophic forgetting” encountered in continual learning systems. Importantly, the ARP evaluated the claims as a whole in discerning at least the limitation “adjust the first values of the plurality of parameters to optimize performance of the machine learning model on the second machine learning task while protecting performance of the machine learning model on the first machine learning task” reflected the improvement disclosed in the specification. Accordingly, the claims as a whole integrated what would otherwise be a judicial exception instead into a practical application at Step 2A Prong Two, and therefore the claims were deemed to be outside any specific, enumerated judicial exception (Step 2A: NO)”. (emphasis added, see MPEP § 2106.04(d), subsection III).
Per contra, independent claim 1 when view as whole do not appear to recite features which provide concrete benefits, for example, the claim does not provide details in how for example the machine learning model is being optimize, how deployment times are being decreased or how lower computational cost for training in the target domain are being achieved. Further, applicant is reminder that while “the claim itself does not need to explicitly recite the improvement described in the specification (e.g., “thereby increasing the bandwidth of the channel”)”, if the specification sets forth an improvement in technology or a technical field, the claims must “includes the components or steps of the invention that provide the improvement described in the specification” (see MPEP § 2106.04(d)(1)).
Regarding the arguments for Step 2A: Prong Two; amended claim 1 as presented does not integrate into a practical application under the second prong of the two-prong analysis since the claimed invention do not improve the functioning of a computer or improves another technology or technical field. Rather the claim recites additional element as stated above, merely recite the words apply it" (or an equivalent) with the judicial exception, as discussed in MPEP § 2106.05(f) which the courts have identified such limitations do not integrate a judicial exception into a practical application (see MPEP 2106.04(d)(I)).
Lastly, under STEP 2B claim 1 when evaluated as whole the additional elements of claim 1 are best mere instructions to “apply” the abstract ideas, which cannot provide an inventive concept. See MPEP 2106.05(f).
Therefore for the above reason, claims 1-20 are not directed to patent-eligible
subject matter under 35 U.S.C § 101.
Regarding the 35 U.S.C § 103 Rejection:
Applicant's further arguments see pg. 13-17, filed 02/09/2026, have been fully considered but they are not persuasive.
APPLICANT ARGUMENT:
Applicant argues the following: “Nguyen does not disclose or suggest approximating a solution to a convex optimization sub-problem, using that approximation as a search direction and updating an initial weight vector by subtracting an approximate sub-problem solution multiplied by a learning rate, as recited in claim 1. Nguyen's discussion of learning rate pertains to algorithm termination and complexity analysis, not to the claimed update rule based on an approximate solution of a convex sub-problem”. Further, “Nguyen, in the cited portion, describes regularized empirical risk minimization problems with convex losses and it does not relate to global minimum that is achieved by approximating and updating steps with the updated weight vector. In fact, the cited portions or elsewhere in Nguyen does not mention that implementation of the classifier is performed based on the achieved global minimum”. In addition, applicant argues the cited reference Ahuja fails to teach or suggest "approximating a solution to the convex optimization subproblem that obtains a search direction, to learn a common classifier from training data," as recited in independent claim 1.
Moreover, Applicant respectfully submits that the Examiner's stated rationale for combining Nguyen and Ahuja is insufficient. Specifically, the Examiner's assertion, on page 15 of the Office Action, that combining Nguyen and Ahuja would "improve the art by reducing the number of computations" relies on the intended benefits of Applicant's claimed features, rather than on teachings or suggestions found in the cited references. Such reasoning constitutes impermissible hindsight reconstruction, using Applicant's disclosure as a roadmap for combining unrelated features from the cited references.
EXAMINER RESPONSE: Examiner respectfully disagree; applicant argument is not persuasive. Applicant argues, “Nguyen's discussion of learning rate pertains to algorithm termination and complexity analysis, not to the claimed update rule based on an approximate solution of a convex sub-problem.”, however, the learning rate in Nguyen's Algorithm 3 SARAH++ serves for two distinct roles:
the learning rate serves as an update rule to solve convex problem as shown by the following equation
w
t
+
1
s
=
w
t
s
-
η
v
t
s
, where
w
is the weight vector,
η
is the learning rate ,
v
t
S
is the search direction (see Algorithm 3 SARAH++ line 13).
(ii) the learning rate serve as a stopping criteria for the inner loop in SARAH++ (see pg. 9, Sec: Lemma 2, lines 7-8 & Algorithm 3 SARAH++ line 12).
Thus, Nguyen's Algorithm 3 SARAH++ line 13 explicitly teaches learning rate is used as an update rule to solve convex problem.
In addition, regarding applicant arguments the cited reference “does not relate to global minimum that is achieved by approximating and updating steps with the updated weight vector. In fact, the cited portions or elsewhere in Nguyen does not mention that implementation of the classifier is performed based on the achieved global minimum”, examiner disagree, in particular Nguyen's pg. 8, Assumption 3 teaches an
w
*
as an optimal solution of
F
where
w
*
=
arg
m
ⅈ
n
w
F
w
(see pg. 22 Convex SARAH++ Proof of Lemma 2). A person skilled in the relevant art will recognize that, in convex problem
w
*
is guaranteed to be a global minimum as the function curves upward like a single U shape, thus
w
*
is the global minimizer of
F
(see for example, the graph below illustrating local (or global) minimum in convex function).
PNG
media_image1.png
470
739
media_image1.png
Greyscale
Furthermore, Nguyen pg. 8, sec: Assumption 3 does teach a global minimum (
w
*
) and pg. 11, para. 3 teaches how Algorithm 3 - SARAH++ has a guarantee of theoretical convergence”, thus Algorithm 3 - SARAH++ enable to achieve convergence to the global minimum. Furthermore, Nguyen pg. 11, sec: 3.2 Numerical Experiment, para 1-2 teaches the Algorithm 3 - SARAH ++ being used to perform experiment implementing logistic regression, for which applicant specification para. [0030] and [0031] teaches logistic regression is a predictive model (common classifier)).
Regarding the arguments of Ahuja, the cited reference was applied to teach the limitation that Nguyen lacks. Specifically the teaching of a method, solving an optimization sub-problem and learning a common classifier from training data. In that regard, Ahuja [0004] teaches a method and 0042] teaches solving a sequential optimization problem, as would be familiar to skilled in the art “sequential optimization” involves breaking down an overall objective into sequences of smaller, simpler sub-problems. Moreover, Fig. 3 element 306 teaches learning a machine learning models and [0102] teaches training data is used to learn the models.
Moreover, the motivation to combine the reference was not relied on the intended benefits of Applicant's claimed features. As is clearly shown in pg. 15 of the office action dated 11/12/2025, the motivation to combine was relied on the teaching of found in the cited reference Ahuja. See portion of paragraph [0021] of Ahuja as follows:
PNG
media_image2.png
538
690
media_image2.png
Greyscale
Thus, applicant assertion that examiner relies on the intended benefit of Applicant’s claims feature is erroneous as examiner demonstrated that Nguyen and Ahuja, are analogues art, and clearly emphasize how it would been obvious to a person ordinary skill in the art before the effective filling date of the claimed invention to have modified the invention of Nguyen with the teaching of Ahuja because doing so it will “improve the art by reducing the number of computations to solve sequential optimization problems (i.e., problem instances).Embodiments of the present invention improve the art by reduced costs of querying black-box models, reducing wait-time for subjects, and reduced energy consumption” (see Ahuja [0021]).
Lastly, examiner respectfully disagree, the combination of Nguyen and Ahuja, does not represent “impermissible hindsight”, because Nguyen and Ahuja, are all in the same field of endeavor (i.e., machine learning). Applicants may argue that the examiner’s conclusion of obviousness is based on improper hindsight reasoning. However, “[a]ny judgment on obviousness is in a sense necessarily a reconstruction based on hindsight reasoning, but so long as it takes into account only knowledge which was within the level of ordinary skill in the art at the time the claimed invention was made and does not include knowledge gleaned only from applicant’s disclosure, such a reconstruction is proper.” (See MPEP 2145 subsection X.A)
Therefore for the above reason, claims 1-5 and 12-17 are not directed to patent-eligible
subject matter under 35 U.S.C § 103.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1
Claims 1-11 are a method type claim. Claims 12-14 are a computer program product type claim. Claims 15-20 are an apparatus type claim. Therefore, claims 1-20 are directed to either a process, machine, manufacture or composition of matter.
Regarding claim 1: 2A Prong 1:
selecting an initial weight vector for a convex optimization sub-problem associated with a neural network having a non-convex network architecture loss surface; (mental process – of selecting an initial weight vector for a convex optimization sub-problem associated with a neural network having a non-convex network architecture loss surface can be performed by the human mind with the help of pen and paper (e.g., judgement & evaluation)).
...approximating a solution to the convex optimization sub- problem that obtains a search direction, to learn a classifier from training data; (mathematical concept – of approximating a solution to the convex optimization sub-problem that obtains a search direction, to learn a common classifier from training data. As though by applicant specification [0051] the approximation is derived from basic calculus (e.g., mathematical calculation)).
...updating the initial weight vector by subtracting the approximate solution to the convex optimization sub-problem times a first learning rate; and (mathematical concept – of updating the initial weight vector by subtracting the approximate solution to the convex optimization sub-problem times a first learning rate. Applicant specification equation (8) teaches the updated are performed by the following equation
w
t
+
1
=
w
t
-
η
t
v
t
(e.g., mathematical calculation)).
...repeating the approximating and updating steps, for a plurality of iterations, with the updated weight vector from a given one of the iterations taken as the initial weight vector for a next one of the iterations, to obtain a final weight vector for the neural network, until convergence to a global minimum is achieved, to implement the classifier (mathematical concept – of repeating the approximation and updating steps for a number of iteration such as t=0,...., T-1 until convergence to a global minimum is achieved, to implement the common classifier (e.g., mathematical calculation)).
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
with the at least one processor,... (This is directed to using computers or other machinery merely as a tool to perform an existing process. See MPEP 2106.05(f)).
The additional elements as disclosed above alone or in combination do not integrate the judicial exception into practical application as they are mere generic computer functions being implemented with generic computer elements in a high level of generality to perform the disclosed abstract idea above.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
with the at least one processor,... (This is directed to using computers or other machinery merely as a tool to perform an existing process. See MPEP 2106.05(f)).
The additional elements as disclosed above in combination of the abstract idea are not sufficient to amount to significantly more than the judicial exception as they are mere generic computer functions being implemented with generic computer elements in a high level of generality to perform the disclosed abstract idea above.
Regarding claim 2: Depends on claim 1, thus the rejection of claim 1 is incorporated.2A Prong 1: None.
2A Prong 2 and 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
further comprising carrying out inferencing with the neural network having the final weight vector (This is directed to using computers or other machinery merely as a tool to perform an existing process. See MPEP 2106.05(f)).
Regarding claim 3: Depends on claim 2, thus the rejection of claim 2 is incorporated.2A Prong 1: None.
2A Prong 2 and 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
wherein the inferencing comprises image classification (This is directed to restricting the abstract idea to a particular technological environment. See MPEP 2106.05(h)).
Regarding claim 4: Depends on claim 3, thus the rejection of claim 3 is incorporated.2A Prong 1: None.
2A Prong 2 and 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
further comprising controlling an industrial robot based on the image classification (This is directed to using computers or other machinery merely as a tool to perform an existing process. See MPEP 2106.05(f)).
Regarding claim 5: Depends on claim 3, thus the rejection of claim 3 is incorporated.2A Prong 1: None.
2A Prong 2 and 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
further comprising performing automatically driving an autonomous vehicle based on the image classification (This is directed to using computers or other machinery merely as a tool to perform an existing process. See MPEP 2106.05(f)).
Regarding claim 6: Depends on claim 1, thus the rejection of claim 1 is incorporated.2A Prong 1:
wherein the convex optimization sub-problem that obtains the search direction comprises minimizing an expression including an average, over the training data, of a squared norm of a difference between a first term and a second term, the first term comprising the first learning rate times a first order derivative matrix times the approximate solution, the second term comprising a second learning rate times a gradient of a loss function (mathematical concept – minimizing an expression (e.g., mathematical calculation)).
2A Prong 2 and 2B: None.
Regarding claim 7: Depends on claim 6, thus the rejection of claim 6 is incorporated.2A Prong 1:
wherein the expression comprises:
PNG
media_image3.png
50
703
media_image3.png
Greyscale
wherein a total number of the plurality of iterations is defined by
T
time steps
t
,
H
i
t
comprises the first order derivative matrix,
η
t
>
0
is the first learning rate,
v
t
is the approximate solution for the search direction,
w
t
is the weight vector at time
t
,
α
i
t
is the second learning rate,
∇
z
is the gradient,
ϕ
i
is the loss function,
h
⋅
;
ⅈ
is a classifier neural network,
v
*
(
t
)
is the exact solution for the search direction, and
n
is a count of training samples (mathematical concept – minimizing an expression (e.g., mathematical calculation)).
2A Prong 2 and 2B: None.
Regarding claim 8: Depends on claim 6, thus the rejection of claim 6 is incorporated.2A Prong 1:
wherein approximating the solution to the convex optimization sub-problem that obtains the search direction comprises obtaining an exact solution to a regularized form of the convex optimization sub-problem (mathematical concept – of obtaining an exact solution to a regularized form of the convex optimization sub-problem. As shown in applicant specification [0104] obtaining an exact solution can be done by using the following mathematical equation
PNG
media_image4.png
85
867
media_image4.png
Greyscale
(e.g., mathematical calculation)).
2A Prong 2 and 2B: None.
Regarding claim 9: Depends on claim 8, thus the rejection of claim 8 is incorporated.2A Prong 1:
wherein the exact solution to the regularized form of the convex optimization sub-problem is given by:
PNG
media_image4.png
85
867
media_image4.png
Greyscale
wherein
H
i
t
comprises the first order derivative matrix,
η
t
>
0
is the first learning rate,
w
t
is the weight vector at time
t
,
α
i
t
is the second learning rate,
∇
z
is the gradient,
ϕ
i
is the loss function,
h
⋅
;
ⅈ
is the classifier neural network,
n
is a count of training samples,
ε
predetermined tolerance and
I
is an identity matrix (mathematical concept – of obtaining an exact solution to a regularized form of the convex optimization sub-problem. As shown in applicant specification [0104] obtaining an exact solution can be done by using a mathematical equation (e.g., mathematical calculation)).
2A Prong 2 and 2B: None.
Regarding claim 10: Depends on claim 9, thus the rejection of claim 9 is incorporated.2A Prong 1:
wherein
h
⋅
;
ⅈ
is twice continuously differentiable for all
ⅈ
∈
n
a problem dimension
d
is sufficiently large to interpolate all the data and the predetermined tolerance
ε
(mathematical concept – of a function such as
h
⋅
;
ⅈ
being twice continuously differentiable for all
ⅈ
∈
n
a problem dimension
d
is sufficiently large to interpolate all the data and the tolerance
ε
involves calculus as shown in applicant specification para. [0066] (e.g., mathematical calculation)).
2A Prong 2 and 2B: None.
Regarding claim 11: Depends on claim 6, thus the rejection of claim 6 is incorporated.2A Prong 1:
approximating the solution to the convex optimization sub-problem that obtains the search direction comprises applying gradient descent such that a norm of a difference between the approximate solution and a solution to a regularized form of the convex optimization sub-problem does not exceed a predetermined tolerance
ε
; (mathematical concept – of applying gradient descent such that a norm of a difference between the approximate solution and a solution to a regularized form of the convex optimization problem does not exceed a predetermined tolerance in order to approximating the solution to the convex optimization sub-problem that obtains the search direction. As shown in applicant specification [0080-0084] (e.g., mathematical calculation)).
h
⋅
;
ⅈ
is a classifier neural network and
h
⋅
;
ⅈ
is twice continuously differentiable for all
ⅈ
∈
n
; and (mathematical concept – of a function such as
h
⋅
;
ⅈ
being twice continuously differentiable for all
ⅈ
∈
n
a problem dimension
d
is sufficiently large to interpolate all the data and the tolerance
ε
involves calculus as shown in applicant specification para. [0066] (e.g., mathematical calculation)).
a problem dimension
d
is sufficiently large to interpolate all the data and the predetermined tolerance
ε
(mathematical concept – of data interpolation (e.g., mathematical calculation)).
2A Prong 2 and 2B: None.
Regarding claim 12: is rejected under the same rational of claim 1. Claim 12 only recites the additional elements of A computer program product, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to... which is directed to using computers or other machinery merely as a tool to perform an existing process. See MPEP 2106.05(f).
Regarding claim 13: See rejection of claim 2, same rational applies.
Regarding claim 14: See rejection of claim 3, same rational applies.
Regarding claim 15: it is rejected under the same rationality as claim 1. Claim 12 only recites the additional elements of An apparatus comprising: a memory; and at least one processor, coupled to the memory, and operative to... which is directed to using computers or other machinery merely as a tool to perform an existing process. See MPEP 2106.05(f).
Regarding claim 16: See rejection of claim 2, same rational applies.
Regarding claim 17: See rejection of claim 3, same rational applies.
Regarding claim 18: See rejection of claim 6, same rational applies.
Regarding claim 19: See rejection of claim 8, same rational applies.
Regarding claim 20: See rejection of claim 10, same rational applies.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 12 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Lam M. Nguyen et al. Finite-Sum Smooth Optimization with SARAH (hereinafter Nguyen ) in further view of Ahuja et al. US 2022/0164644 A1 (hereinafter Ahuja).
Regarding claim 1:
selecting an initial weight vector for a convex optimization ( Nguyen pg. 1, sec: Introduction, para. 2 teaches a “finite sum smooth minimization problem” that “covers a wide range of convex and nonconvex models in machine learning” such as neural networks, thus the loss surface of the nonconvex model (i.e., neural network) will be a non-convex network architecture loss surface. Further, Nguyen Algorithm 3 - line 2, teaches a method for choosing (selecting) an initial point
w
~
0
(i.e., initial weight vector)).
with at least one processor, approximating a solution to the convex optimization (
PNG
media_image5.png
717
1092
media_image5.png
Greyscale
Nguyen Algorithm 3 – SARAH ++ line 17, teaches “
v
t
(
s
)
” a search direction that helps approximate to the convex optimization problem).
with the at least one processor, updating the initial weight vector by subtracting the approximate solution to the convex optimization ( Nguyen Algorithm 3 – SARAH ++ teaches update the initial weight vector (line 21) by subtracting the approximate solution to the convex optimization problem (i.e.,
v
t
(
s
)
” ) times the learning rate
η
(line 17)).
with the at least one processor, repeating the approximating and updating steps, for a plurality of iterations, with the updated weight vector from a given one of the iterations taken as the initial weight vector for a next one of the iterations, to obtain a final weight vector for the neural network, until convergence to a global minimum is achieved, to implement the classifier (Nguyen Algorithm 3 – SARAH ++ and pg. 2, para. 3, teaches performing multiple iteration such that “after
m
iteration in the inner loop, the outer loop remembers the last computed
m
m
+
1
s
and starts its loop anew – first with a full gradient computation before again entering the inner loop with updates” to obtain a final weight vector (line 25). In addition, Nguyen pg. 8, sec: Assumption 3 teaches an optimal solution
w
*
(global minimum) and pg. 11, para. 3 teaches how Algorithm 3 - SARAH++ has a guarantee of theoretical convergence”, thus Algorithm 3 - SARAH++ enable to achieve convergence to the global minimum. Furthermore, Nguyen pg. 11, sec: 3.2 Numerical Experiment, para 1-2 teaches the Algorithm 3 - SARAH ++ being used to perform experiment implementing logistic regression, for which applicant specification para. [0030] and [0031] teaches logistic regression is a predictive model (common classifier)).
Nguyen does not specifically teaches a method comprising, an ...optimization sub-problem..., ... learn a classifier from training data;
Nevertheless, Ahuja teaches the following:
A method comprising: (Ahuja [0004]).
solving an ...optimization sub-problem... (Ahuja [0042] teaches solving an a sequential optimization problem, as would be familiar to skilled in the art “sequential optimization” involves breaking down an overall objective into sequences of smaller, simpler sub-problems).
with the at least one processor,... (Ahuja Fig. 16 and [0122] teaches computer system with a processors -element 1601).
...to learn a classifier from training data; (Ahuja Fig. 3 element 306 teaches learning a machine learning models and [0102] teaches training data is used to learn the models ).
Ahuja is also in the same field of endeavor as Nguyen (machine learning). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include the functionality of sequential optimization problem, processor and learning a machine learning model as being disclosed and taught by Ahuja, in the system taught by Nguyen to yield the predictable results of “ improve the art by reducing the number of computations to solve sequential optimization problems (i.e., problem instances). Embodiments of the present invention improve the art by reduced costs of querying black-box models, reducing wait-time for subjects, and reduced energy consumption” (see Ahuja [0021]).
Regarding claim 12: it is rejected under the same rationality as claim 1. Claim 12 only recites the additional elements of A computer program product, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to... , for which Ahuja [0130] teaches.
Regarding claim 15: it is rejected under the same rationality as claim 1. Claim 15 only recites the additional elements of An apparatus comprising: a memory; and at least one processor, coupled to the memory, and operative to... , for which Ahuja [0134] teaches an apparatus (systems) and Fig. 16 teaches a processor (element 1601) coupled with a memory (element 1602).
Claims 2, 13 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Nguyen, Ahuja in further view of Covington et al. US 2022/0037019 A1 (hereinafter Covington).
Regarding claim 2:
Nguyen and Ahuja teach The method of claim 1. Neither Nguyen nor Ahuja teaches further comprising carrying out inferencing with the neural network having the final weight vector.
Nonetheless, Covington teaches the following:
further comprising carrying out inferencing with the neural network having the final weight vector ( Covington [0132] teaches carrying out inference with the neural network “final model parameter data, such as the weight values of the final weight vector”).
Covington is also in the same field of endeavor as Nguyen and Ahuja (machine learning). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include the functionality of carrying out inference with the neural network final model parameter data, as being disclosed and taught by Covington, in the system taught by Nguyen and Ahuja to yield the predictable results of “improve existing inference functions and/or to add new inference functions” (see Covington [0166]).
Regarding claim 13: it is rejected under the same rationality as claim 2.
Regarding claim 16: it is rejected under the same rationality as claim 2.
Claims 3-5, 14, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Nguyen, Ahuja, Covington in further view of Rittel et al. WO 2021/165077 A1 (hereinafter Rittel).
Regarding claim 3:
Nguyen, Ahuja and Covington teach The method of claim 2. Neither over Nguyen, Ahuja or Covington teaches wherein the inferencing comprises image classification.
Nevertheless, Rittel discloses the following:
wherein the inferencing comprises image classification (Rittel pg. 36, sec: Disclosure of invention, lines 66-67, teaches an image classifier that produces image classification output (inference)).
Rittel is also in the same field of endeavor as Nguyen, Ahuja and Covington (machine learning). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include the functionality of image classification, as being disclosed and taught by Rittel, in the system taught by Nguyen, Ahuja and Covington to yield the predictable results of improve performance of the overall system (Rittel pg. 4, lines 172-173 and 176-179).
Regarding claim 4:
Nguyen, Ahuja, Covington and Rittel teach The method of claim 3. Rittel specifically teaches further comprising controlling an industrial robot based on the image classification (Rittel pg. 2, lines 58-59 teaches controlling an at least a partiality autonomous robot (industrial robot) based on the image classifier output).
Regarding claim 5:
Nguyen, Ahuja, Covington and Rittel teach The method of claim 3. Rittel specifically teaches further comprising performing automatically driving an autonomous vehicle based on the image classifications (Rittel pg. 2, lines 49-50 teaches operating a mobile robot such as (i.e., autonomous vehicle see pg. 2 lines 67-68 ) based on the classification output of the image classifier).
Regarding claim 14: it is rejected under the same rationality as claim 3.
Regarding claim 17: it is rejected under the same rationality as claim 3.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GISEL G FACCENDA whose telephone number is (703)756-1919. The examiner can normally be reached Monday - Friday 8:00 am - 4:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abdullah Al Kawsar can be reached at (571) 270-3169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/G.G.F./Examiner, Art Unit 2127
/ABDULLAH AL KAWSAR/Supervisory Patent Examiner, Art Unit 2127
Prosecution Insights