DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 04/06/2026 has been entered.
Response to Amendment
Claims 1-5, 7-15, 17-25, and 27-30 remain pending within the application.
The amendments and supporting remarks filed 04/06/2026 are sufficient to overcome the 35 U.S.C. 101 rejection previously set forth in the Non-Final Office Action mailed 02/04/2026. The rejection has been withdrawn.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 7-9, 11-14, 17-20, 21-24, and 27-30 are rejected under 35 U.S.C. 103 as being unpatentable over Anthony et al. (Pub. No.: US 2022/0261635 A1), hereafter Anthony, in view of Yang et al. (" Joint Active Feature Acquisition and Classification with Variable-Size Set Encoding"), hereafter Yang, in further view of Nag et al. (Pub. No.: US 2020/0065118 A1), hereafter Nag, in further view of Nakata et al. ("An Analysis of Rule Deletion Scheme in XCS on Reinforcement Learning Problem "), hereafter Nakata.
Regarding claim 1, Anthony discloses:
One or more non-transitory computer-readable media storing program instructions that, when executed by one or more processors, cause the one or more processors to perform a method comprising (Anthony, ¶[0095-0103]):
receiving a plurality of actions associated with a reinforcement learning model (Anthony, Fig. 3, ¶[0027] and ¶[0082] teaches receiving possible actions to be performed by agents as receiving a plurality of actions associated with a reinforcement learning model),
generating a plurality of combinations of actions based on the plurality of actions… (Anthony, Fig. 3 element 306 and ¶[0086] teaches sampling a plurality of candidate actions as generating a plurality of combinations of actions based on the plurality of actions),
analyzing the plurality of combinations of actions… (Anthony, Fig. 3 elements 308 and 310, and ¶[0087-0088] teaches analyzing the plurality of combinations of actions),
generating at least one subset of indispensable actions … (Anthony, ¶[0012-0016], ¶[0063] and Fig. 2 and Fig. 3 teaches generating best responses as at least one subset of indispensable actions),
selecting a set of training actions from the plurality of actions based on the at least one subset of indispensable actions (Anthony, Fig. 2 and ¶[0060] teaches selecting training data as a set of training actions based on the indispensable actions),
training the reinforcement learning model based on the set of training actions (Anthony, Fig. 2 and ¶[0058] teaches training the policy neural network as the reinforcement learning model based on the set of training actions).
While Anthony teaches generating a plurality of combinations of actions based on the plurality of actions, they do not explicitly teach the plurality of combinations being a power set of the plurality of actions.
Yang discloses:
generating a plurality of combinations of actions based on the plurality of actions, the plurality of combinations being a power set of the plurality of actions (Yang, page 4, paragraph 6, lines 1-2 “Action. The RL agent selects which features to examine. The set of all possible actions is defined as the power set of {1, . . . ,p} including the empty set ∅,” teaches generating a plurality of combinations of actions based on the plurality of actions, the plurality of combinations being a power set of the plurality of actions).
Anthony and Yang are analogous art because they are from the same field of endeavor, reinforcement learning.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Anthony to include the plurality of combinations being a power set of the plurality of actions, based on the teachings of Yang. One of ordinary skill in the art would have been motivated to make this modification in order to systematically solve the optimization problem that minimizes the prediction error as well as the feature acquisition cost, as suggested by Yang (page 1, introduction, paragraph 2).
While Anthony does not disclose: computing a cut-off cardinality by executing the reinforcement learning model to determine a minimum number of actions required for the reinforcement learning model to satisfy a success threshold within a first number of iterations.
Nag discloses:
… executing the reinforcement learning model to determine … actions required for the reinforcement learning model to satisfy a success threshold within a first number of iterations (Nag, Figs. 25, 36A-C and ¶[0103-105] teaches executing the reinforcement learning model to determine candidate actions required for the reinforcement learning model to satisfy a success threshold, i.e. termination conditions and score thresholds, within a first number of iterations).
Anthony, Yang, and Nag are analogous art because they are from the same field of endeavor, reinforcement learning.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Anthony, in view of Yang, to include executing the reinforcement learning model to determine … actions required for the reinforcement learning model to satisfy a success threshold within a first number of iterations, based on the teachings of Nag. One of ordinary skill in the art would have been motivated to make this modification in order to improve the policy used to select actions, as suggested by Nag (Nag, ¶[0087]).
While Anthony discloses generating at least one subset of indispensable actions, they do not disclose doing so by determining that excluding the subset prevents the reinforcement learning model from satisfying a second success threshold.
Nag discloses:
generating at least one subset of indispensable actions by determining that excluding the subset prevents the reinforcement learning model from satisfying a second success threshold (Nag, Fig. 25, 36A-C and ¶[0105] teaches generating a proposed action as one subset of indispensable actions by determining that excluding the subset prevents the reinforcement learning model from satisfying second success thresholds, i.e., a second termination condition or a second score threshold).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Anthony, in view of Yang, to include generating at least one subset of indispensable actions by determining that excluding the subset prevents the reinforcement learning model from satisfying a second success threshold, based on the teachings of Nag. One of ordinary skill in the art would have been motivated to make this modification in order to improve the policy used to select actions, as suggested by Nag (Nag, ¶[0087]).
While Nag discloses executing the reinforcement learning model to determine actions required for the reinforcement learning model to satisfy a success threshold within a first number of iterations, they do not disclose computing a cut-off cardinality by executing the reinforcement learning model to determine a minimum number of actions.
Nakata discloses:
computing a cut-off cardinality by executing the reinforcement learning model to determine a minimum number of actions (Nakata, page 877, lines above equations 5-6 “Then, the prediction error cl.e and the action set size cl.as of each rule cl are updated as follows,” and equation (6) teaches computing updated action set sizes, i.e. cutoff cardinalities, by executing the reinforcement learning model to determine a minimum number of actions).
Nakata further discloses:
analyzing the plurality of combinations of actions having a cardinality at least equal to the cut-off cardinality (Nakata, equations (7)-(9) teaches analyzing the updated set of actions having a cardinality, i.e. set size, equal to the cutoff determined in equation (5)).
Anthony, Yang, Nag, and Nakata are analogous art because they are from the same field of endeavor, reinforcement learning.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Anthony, in view of Yang, in further view of Nag, to include computing a cut-off cardinality by executing the reinforcement learning model to determine a minimum number of actions, and analyzing the plurality of combinations of actions having a cardinality at least equal to the cut-off cardinality based on the teachings of Nakata. One of ordinary skill in the art would have been motivated to make this modification in order to guarantee that the search space is explored uniformly and that the system does not get stuck, as suggested by Nakata (page 878, right column, paragraph 3).
Regarding claim 2, Anthony, in view of Yang, in further view of Nag, in further view of Nakata, discloses one or more non-transitory computer-readable media of claim 1 (and thus the rejection of claim 1 is incorporated) selecting the set of training actions. Anthony further discloses:
selecting the set of training actions is further based on one or more cumulative rewards associated with the plurality of combinations of actions (Anthony, ¶[0054] and ¶[0037] teaches selection of training actions based on cumulative rewards associated with the plurality of combinations of actions).
Regarding claim 3, Anthony, in view of Yang, in further view of Nag, in further view of Nakata, discloses one or more non-transitory computer-readable media of claim 1 (and thus the rejection of claim 1 is incorporated) analyzing the plurality of combinations of actions. Anthony further discloses:
wherein analyzing the plurality of combinations of actions comprises causing the reinforcement learning model to execute a first number of iterations using a first combination of actions to determine whether the reinforcement learning model satisfies a first threshold condition after the first number of iterations using the first combination of actions (Anthony, Fig. 3 elements 310-314, ¶[0088-0091], ¶[0072] teaches the reinforcement learning model to execute iterations using the first combination of actions to determine whether the model satisfies a threshold condition, i.e. determining the highest overall value estimate).
Regarding claim 4, Anthony, in view of Yang, in further view of Nag, in further view of Nakata, discloses one or more non-transitory computer-readable media of claim 3 (and thus the rejection of claim 3 is incorporated). Anthony further discloses:
wherein generating the at least one subset of indispensable actions is based on determining that the reinforcement learning model does not satisfy the first threshold condition after the first number of iterations using the first combination of actions (Anthony, Fig. 3 elements 310-314, ¶[0088-0091] teaches the reinforcement learning model to determine whether the model does not satisfy a threshold condition, i.e. determining the overall value estimate that is not the highest, and generating the best response as the indispensable action).
Regarding claim 7, Anthony, in view of Yang, in further view of Nag, in further view of Nakata, discloses one or more non-transitory computer-readable media of claim 1 (and thus the rejection of claim 1 is incorporated). Nakata further discloses:
wherein computing the cut-off cardinality comprises evaluating combinations of actions having a given cardinality to determine whether each of the combinations of actions having the given cardinality fail to improve a cumulative reward by a threshold amount (Nakata, page 877, paragraph 2 “When the reward rt is received and the match set [M] with respect to the resulting sensory input is formed, the parameters of the rules in [A] are updated in the following order [18]: prediction, prediction error, action set size, and fitness.” and equations (2)-(3) teaches the successive iteration of reward calculations to evaluate the previous combinations of actions having a given cardinality, i.e. updated set size, to determine whether each of the combinations of actions having the given cardinality fail to improve a cumulative reward by a threshold amount).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Anthony, in view of Yang, in further view of Nag, to include wherein computing the cut-off cardinality comprises evaluating combinations of actions having a given cardinality to determine whether each of the combinations of actions having the given cardinality fail to improve a cumulative reward by a threshold amount based on the teachings of Nakata. One of ordinary skill in the art would have been motivated to make this modification in order to guarantee that the search space is explored uniformly and that the system does not get stuck, as suggested by Nakata (page 878, right column, paragraph 3).
Regarding claim 8, Anthony, in view of Yang, in further view of Nag, in further view of Nakata, discloses one or more non-transitory computer-readable media of claim 1 (and thus the rejection of claim 1 is incorporated).
While Anthony teaches the reinforcement learning model, a plurality of states associated with the reinforcement learning model, and a plurality of actions as disclosed in claim 1, they do not disclose:
wherein the reinforcement learning model is associated with managing one or more components of a computing system,
wherein each state included in a plurality of states associated with the reinforcement learning model corresponds to a different set of resource utilization metrics associated with the computing system,
wherein each action included in the plurality of actions corresponds to a different component management action associated with a different component of the computing system, wherein each component included in the one or more components is one of a virtual machine, an application, a service, or a node.
Nag disclose:
wherein the reinforcement learning model is associated with managing one or more components of a computing system (Nag, Fig. 12 and ¶[0069] teaches a reinforcement learning model that is associated with managing one or more components of a computing system),
wherein each state included in a plurality of states associated with the reinforcement learning model corresponds to a different set of resource utilization metrics associated with the computing system (Nag, Fig. 12 and ¶[0069] teaches states associated with the reinforcement learning model corresponding to a different set of resource utilization metrics associated with the computing system, such as “the amount of memory allocated for applications and/or application instances”),
wherein each action included in the plurality of actions corresponds to a different component management action associated with a different component of the computing system (Nag, Fig. 12 and ¶[0069] and ¶[0071] teaches wherein each action included in the plurality of actions corresponds to a different component management action associated with a different component of the computing system),
wherein each component included in the one or more components is one of a virtual machine, an application, a service, or a node (Nag, Fig. 12, ¶[0069] and ¶[0092]).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Anthony, in view of Yang, to include wherein the reinforcement learning model is associated with managing one or more components of a computing system, wherein each state included in a plurality of states associated with the reinforcement learning model corresponds to a different set of resource utilization metrics associated with the computing system, wherein each action included in the plurality of actions corresponds to a different component management action associated with a different component of the computing system, wherein each component included in the one or more components is one of a virtual machine, an application, a service, or a node, based on the teachings of Nag. One of ordinary skill in the art would have been motivated to make this modification in order to improve the policy used to select actions, as suggested by Nag (Nag, ¶[0087]).
Regarding claim 9, Anthony, in view of Yang, in further view of Nag, in further view of Nakata, discloses one or more non-transitory computer-readable media of claim 1 (and thus the rejection of claim 1 is incorporated).
While Anthony teaches the reinforcement learning model, a plurality of states associated with the reinforcement learning model, and a plurality of actions as disclosed in claim 1, they do not disclose:
wherein the reinforcement learning model is associated with performance tuning for a plurality of microservices that are deployed on a computing system,
wherein each state included in a plurality of states associated with the reinforcement learning model corresponds to a different set of request rates and latencies associated with the plurality of microservices,
wherein each action included in the plurality of actions corresponds to a different create, read, updated, and delete operation on one or more different microservices included in the plurality of microservices.
Nag discloses:
wherein the reinforcement learning model is associated with performance tuning for a plurality of microservices that are deployed on a computing system (Nag, Fig. 27A and ¶[0089] teaches reinforcement learning model to be associated with performance tuning for a plurality of microservices that are deployed on a computing system),
wherein each state included in a plurality of states associated with the reinforcement learning model corresponds to a different set of request rates and latencies associated with the plurality of microservices (Nag, ¶[0068] teaches each state included in a plurality of states associated with the reinforcement learning model corresponds to a different set of request rates and latencies associated with the plurality of microservices),
wherein each action included in the plurality of actions corresponds to a different create, read, updated, and delete operation on one or more different microservices included in the plurality of microservices (Nag, Fig. 27A, ¶[0069] teaches actions to correspond to different operations on the different microservices).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Anthony, in view of Yang, to include wherein the reinforcement learning model is associated with performance tuning for a plurality of microservices that are deployed on a computing system, wherein each state included in a plurality of states associated with the reinforcement learning model corresponds to a different set of request rates and latencies associated with the plurality of microservices, wherein each action included in the plurality of actions corresponds to a different create, read, updated, and delete operation on one or more different microservices included in the plurality of microservices, based on the teachings of Nag. One of ordinary skill in the art would have been motivated to make this modification in order to improve the policy used to select actions, as suggested by Nag (Nag, ¶[0087]).
Claims 11-14 and 21-24 are substantially similar to claims 1-4, and thus are rejected on the same basis as claims 1-4.
Claims 18 and 28 are substantially similar to claim 7, and thus are rejected on the same basis as claim 7.
Regarding claim 17, Anthony, in view of Yang, in further view of Nag, in further view of Nakata, discloses the computer-implemented method of claim 11 (and thus the rejection of claim 11 is incorporated).
Nakata further discloses:
wherein computing the cut- off cardinality comprises evaluating combinations of actions having a given cardinality to determine whether each of the combinations of actions having the given cardinality fail to allow the reinforcement learning model to satisfy a second threshold condition within a second number of iterations (Nakata, page 877, paragraph 2 “When the reward rt is received and the match set [M] with respect to the resulting sensory input is formed, the parameters of the rules in [A] are updated in the following order [18]: prediction, prediction error, action set size, and fitness.” and equations (7)-(8) teaches evaluating the set of combinations of a given cardinality to determine whether each of the combinations of actions having the given cardinality fail to allow the reinforcement learning model to satisfy a second threshold condition, rule fitness and accuracy, within a second number of iterations).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Anthony, in view of Yang, in further view of Nag, to include wherein computing the cut- off cardinality comprises evaluating combinations of actions having a given cardinality to determine whether each of the combinations of actions having the given cardinality fail to allow the reinforcement learning model to satisfy a second threshold condition within a second number of iterations based on the teachings of Nakata. One of ordinary skill in the art would have been motivated to make this modification in order to guarantee that the search space is explored uniformly and that the system does not get stuck, as suggested by Nakata (page 878, right column, paragraph 3).
Regarding claim 19, Anthony, in view of Yang, in further view of Nag, in further view of Nakata, discloses the computer-implemented method of claim 11 (and thus the rejection of claim 11 is incorporated).
Nakata further discloses:
wherein analyzing the plurality of combinations of actions comprises evaluating one or more combinations of actions having a cardinality higher than or equal to the cut-off cardinality and not evaluating one or more combinations of actions having a cardinality lower than the cut-off cardinality (Nakata, equation (6) teaches evaluating one or more combinations of actions having a cardinality equal to the cut-off cardinality and not evaluating actions having a cardinality lower than the cut-off).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Anthony, in view of Yang, in further view of Nag, to include wherein analyzing the plurality of combinations of actions comprises evaluating one or more combinations of actions having a cardinality higher than or equal to the cut-off cardinality and not evaluating one or more combinations of actions having a cardinality lower than the cut-off cardinality, based on the teachings of Nakata. One of ordinary skill in the art would have been motivated to make this modification in order to guarantee that the search space is explored uniformly and that the system does not get stuck, as suggested by Nakata (page 878, right column, paragraph 3).
Regarding claim 20, Anthony, in view of Yang, in further view of Nag, in further view of Nakata, discloses the computer-implemented method of claim 11 (and thus the rejection of claim 11 is incorporated).
While Anthony teaches the reinforcement learning model, a plurality of states associated with the reinforcement learning model, and a plurality of actions as disclosed in claim 11, they do not disclose:
wherein the reinforcement learning model is associated with managing a computing system,
wherein each state included in a plurality of states associated with the reinforcement learning model corresponds to a different state of the computing system, and
wherein each action included in the plurality of actions corresponds to one of: deploying a virtual machine within the computing system, scaling a node pool included in the computing system, deploying a node, deploying an application, removing an application, performing an create, read, update, and delete operation on a microservice associated with the computing system, scaling a microservice associated with the computing system, or scaling network traffic within the computing system.
Nag discloses:
wherein the reinforcement learning model is associated with managing a computing system (Nag, Fig. 12 and ¶[0069] teaches a reinforcement learning model that is associated with managing one or more components of a computing system),
wherein each state included in a plurality of states associated with the reinforcement learning model corresponds to a different state of the computing system (Nag, Fig. 12 and ¶[0069] teaches states associated with the reinforcement learning model corresponding to a different state of the computing system),
wherein each action included in the plurality of actions corresponds to one of: deploying a virtual machine within the computing system, scaling a node pool included in the computing system, deploying a node, deploying an application, removing an application, performing an create, read, update, and delete operation on a microservice associated with the computing system, scaling a microservice associated with the computing system, or scaling network traffic within the computing system (Nag, Fig. 12 and ¶[0069], ¶[0071], ¶[0092] teaches wherein each action included in the plurality of actions corresponds to deploying a virtual machine within the computing system).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Anthony, in view of Yang, to include wherein the reinforcement learning model is associated with managing a computing system, wherein each state included in a plurality of states associated with the reinforcement learning model corresponds to a different state of the computing system, and wherein each action included in the plurality of actions corresponds to one of: deploying a virtual machine within the computing system, scaling a node pool included in the computing system, deploying a node, deploying an application, removing an application, performing an create, read, update, and delete operation on a microservice associated with the computing system, scaling a microservice associated with the computing system, or scaling network traffic within the computing system, based on the teachings of Nag. One of ordinary skill in the art would have been motivated to make this modification in order to improve the policy used to select actions, as suggested by Nag (Nag, ¶[0087]).
Claim 27 is substantially similar to claim 17, and thus is rejected on the same basis as claim 17.
Claim 29 is substantially similar to claim 19, and thus is rejected on the same basis as claim 19.
Claim 30 is substantially similar to claim 20, and thus is rejected on the same basis as claim 20.
Claims 5, 15, and 25 are rejected under 35 U.S.C. 103 as being unpatentable over Anthony et al. (Pub. No.: US 2022/0261635 A1), hereafter Anthony, in view of Yang et al. (" Joint Active Feature Acquisition and Classification with Variable-Size Set Encoding"), hereafter Yang, in further view of Nag et al. (Pub. No.: US 2020/0065118 A1), hereafter Nag, in further view of Nakata et al. ("An Analysis of Rule Deletion Scheme in XCS on Reinforcement Learning Problem "), hereafter Nakata, in further view of Karanik et al. ("Using Combination of Actions in Reinforcement Learning "), hereafter Karanik.
Regarding claim 5, Anthony, in view of Yang, in further view of Nag, in further view of Nakata, discloses one or more non-transitory computer-readable media of claim 1 (and thus the rejection of claim 1 is incorporated) generating the at least one subset of indispensable actions.
While Anthony teaches generating the at least one subset of indispensable actions in claim 1, they do not explicitly teach generating … indispensable actions based on analyzing … actions that does not include the … indispensable actions.
Karanik teaches:
generating … indispensable actions based on analyzing … actions that does not include the … indispensable actions (Karanik, page 20, left column, penultimate paragraph, lines 8-11 “For action combination, a simple method is proposed: start the learning process with basic actions and, incrementally, create new ones using actions which have better values” teaches generating indispensable actions based on analyzing actions which no not include the newly generated actions).
Anthony, Yang, Nag, Nakata, and Karanik are analogous art because they are from the same field of endeavor, reinforcement learning.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Anthony, in view of Yang, in further view of Nag, in further view of Nakata, to include generating … indispensable actions based on analyzing … actions that does not include the … indispensable actions, based on the teachings of Karanik. One of ordinary skill in the art would have been motivated to make this modification in order to acquire useful experience about the possible system states, actions, transitions between states and rewards to operate optimally and to make a better selection of actions, as suggested by Karanik (Karanik, page 19, left column, last 4 lines to right column, first 2 lines).
Claims 15 and 25 are substantially similar to claim 5, and thus are rejected on the same basis as claim 5.
Claims 10 is rejected under 35 U.S.C. 103 as being unpatentable over Anthony et al. (Pub. No.: US 2022/0261635 A1), hereafter Anthony, in view of Yang et al. (" Joint Active Feature Acquisition and Classification with Variable-Size Set Encoding"), hereafter Yang, in further view of Nag et al. (Pub. No.: US 2020/0065118 A1), hereafter Nag, in further view of Nakata et al. ("An Analysis of Rule Deletion Scheme in XCS on Reinforcement Learning Problem "), hereafter Nakata, in further view of Nair et al. (Pub. No.: US 11,757,982 B2), hereafter Nair.
Regarding claim 10, Anthony, in view of Yang, in further view of Nag, in further view of Nakata, discloses one or more non-transitory computer-readable media of claim 1 (and thus the rejection of claim 1 is incorporated).
While Anthony teaches the reinforcement learning model, a plurality of states associated with the reinforcement learning model, and a plurality of actions as disclosed in claim 1, they do not disclose:
wherein the reinforcement learning model is associated with minimizing pairwise latency for a service mesh of a computing system,
wherein each state included in a plurality of states associated with the reinforcement learning model corresponds to a different amount of network traffic within the computing system, and
wherein each action included in the plurality of actions corresponds to a different network traffic management action associated with the computing system.
Nag discloses:
wherein the reinforcement learning model is associated with minimizing pairwise latency for a service … of a computing system (Nag, ¶[0066] teaches a reinforcement learning model associated with minimizing latency for a distributed computing system),
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Anthony, in view of Yang, to include wherein the reinforcement learning model is associated with minimizing pairwise latency for a service … of a computing system, based on the teachings of Nag. One of ordinary skill in the art would have been motivated to make this modification in order to improve the policy used to select actions, as suggested by Nag (Nag, ¶[0087]).
Anthony, in view of Yang, in view of Nag, teaches wherein the reinforcement learning model is associated with minimizing pairwise latency for a service mesh of a computing system, but does not explicitly disclose the service of a computing system to be a mesh.
Furthermore, Anthony, in view of Yang, in view of Nag, in further view of Nakata, does not discloses:
wherein each state included in a plurality of states associated with the reinforcement learning model corresponds to a different amount of network traffic within the computing system,
wherein each action included in the plurality of actions corresponds to a different network traffic management action associated with the computing system.
Nair teaches:
the reinforcement learning model is associated with … a service mesh of a computing system (Nair, col. 3, lines 43-46 teaches a reinforcement learning model is associated with … a service mesh of a computing system),
wherein each state included in a plurality of states associated with the reinforcement learning model corresponds to a different amount of network traffic within the computing system (Nair, col. 3, lines 12-19 and claim 7 teaches of states associated with the reinforcement learning model corresponds to a different amount of network traffic within the computing system),
wherein each action included in the plurality of actions corresponds to a different network traffic management action associated with the computing system (Nair, col. 4, lines 46-54 teaches each action included in the plurality of actions corresponds to a different network traffic management action associated with the computing system).
Anthony, Yang, Nag, Nakata, and Nair are analogous art because they are from the same field of endeavor, reinforcement learning.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Anthony, in view of Yang, in further view of Nag, in further view of Nakata, to include the reinforcement learning model is associated with … a service mesh of a computing system, wherein each state included in a plurality of states associated with the reinforcement learning model corresponds to a different amount of network traffic within the computing system, wherein each action included in the plurality of actions corresponds to a different network traffic management action associated with the computing system, based on the teachings of Nair. One of ordinary skill in the art would have been motivated to make this modification in order to improve performance, as suggested by Nair (Nair, col. 6, line 24).
Response to Arguments
Applicant's arguments filed 04/06/2026 have been fully considered with regards to the 35 U.S.C. 101 rejection, and are persuasive. The 101 rejection is withdrawn.
Applicant's arguments filed 04/06/2026 have been fully considered with regards to the 35 U.S.C. 102/103 rejection, but they are not persuasive.
The applicant asserts on page 14 of the remarks “Anthony would have to disclose generating best responses by systematically excluding them to see if the model fails to satisfy a success threshold. Notably, Anthony is silent in this regard”. As a preliminary matter, the arguments are directed to newly amended limitations that were not previously examined by the examiner, thus, the examiner refers to the rejection under 35 USC § 103 in the current office action for more details. Second, the amended claim 1 does not appear to teach systemic exclusion, but rather determining that excluding a subset would prevent the model from satisfying a success threshold. This determination is not the same as systemically excluding a subset. The amended claims recite “generating at least one subset of indispensable actions by determining that excluding the subset prevents the reinforcement learning model from satisfying a second success threshold”, which is taught by Nag in Fig. 36C and ¶[0105], by generating a proposed action as one subset of indispensable actions by determining that excluding said action prevents the reinforcement learning model from satisfying a second success threshold, i.e., a second termination condition or a second score threshold.
Applicant’s arguments with respect to computing the cut-off cardinality, more specifically the amended limitation “computing a cut-off cardinality by executing the reinforcement learning model to determine a minimum number of actions required for the reinforcement learning model to satisfy a success threshold within a first number of iterations analyzing the plurality of combinations of actions having a cardinality at least equal to the cut-off cardinality”, have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's
disclosure.
Shah et al. (“Finding minimal action sequences with a simple evaluation of actions”) teaches reinforcement learning and determining minimal action sets.
Dulac-Arnold et al. (“Deep Reinforcement Learning in Large Discrete Action Spaces”) teaches reinforcement learning and determining minimal action sets.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HUMAIRA ZAHIN MAUNI whose telephone number is (703)756-5654. The examiner can normally be reached Monday - Friday, 9 am - 5 pm (ET).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, MATT ELL can be reached at (571) 270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/H.Z.M./Examiner, Art Unit 2141
/MATTHEW ELL/Supervisory Patent Examiner, Art Unit 2141
Prosecution Insights