DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This office action is in response to the claimed invention filed on March 14, 2024, in which claims 1-20 are presented for examination.
Information Disclosure Statement
The information disclosure statement filed on March 14, 2024 complies with the provisions of 37 CFR 1.97, 1.98 and MPEP § 609. It has been placed in the application file. The information referred to therein has been considered as to the merits.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract without significantly more.
Step 1, Statutory Category:
Claims 1-7 are directed to a method
Claims 8-14 are directed to a computer system.
Claims 15-20 are directed to a computer program product.
Therefore, claims 1-20 fall into at least one of the four statutory categories of invention, i.e., process, machine, manufacture, or composition of matter.
Step 2A, Prong One (Judicial exception recited)
The limitation “generating a composite reward function based on one or more objective-based reward functions and one or more constraint-based reward functions” in claims 1, 8 and 15, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process as a form of evaluation or judgement and/or a math calculation, but for the recitation of generic computer components. One can manually with the aid of pen and paper generate a composite reward function based on one or more objective-based reward functions. Additionally, dividing numbers is a math calculation.
The limitation “generating, using a first network, candidate action vectors based on the defined current state vector spaces and the defined action space, the candidate action vectors corresponding to action probabilities” in claims 1, 8 and 15, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process as a form of evaluation or judgement and/or a math calculation, but for the recitation of generic computer components. One can manually with the aid of pen and paper generate candidate action vectors based on the defined current state vector spaces and the defined action space.
The limitation “calculating, using a second network, state value functions based on the candidate action vectors” in claims 1, 8 and 15, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process as a form of evaluation or judgement and/or a math calculation, but for the recitation of generic computer components. One can manually with the aid of pen and paper calculate state value functions based on the candidate action vectors.
The limitation “generating, using the optimized first and second networks, grayscale release strategies including a series of optimized actions to be taken” in claims 1, 8 and 15, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process as a form of evaluation or judgement and/or a math calculation, but for the recitation of generic computer components. One can manually with the aid of pen and paper generate , grayscale release strategies.
At Step 2A, Prong Two (Integrated into a practical application):
The claim recites the following additional elements:
That the method is "implemented by a computing system" is a high-level recitation of a generic computer components and represents mere instructions to apply on a computer as in MPEP 2106.05(f), which does not provide integration into a practical application.
The limitation “defining current state vector spaces and an action space for a target system” amounts to data-gathering steps which is considered to be insignificant extra-solution activity, (See MPEP 2106.05(g)).
The limitation “executing, in training iterations, actions corresponding to the candidate action vectors to obtain environment feedback including observed rewards based on the generated composite reward function;” recites insignificant extra-solution activity such as mere outputting of the result. The mere outputting of data does not meaningfully limit the abstract idea. Viewing the additional limitations together and the claim as a whole, nothing provides integration into a practical application. (See MPEP 2106.05 (g)).
The limitation “optimizing the first and second networks based on the observed rewards” amounts to data-gathering steps which is considered to be insignificant extra-solution activity, (See MPEP 2106.05(g)).
The limitation “one or more processors, one or more computer-readable memories, one or more computer-readable tangible storage medium” are recited at a high level of generality such that they amount to on more than mere instructions to apply the exception using a generic component. (see MPEP 2106.05(f)). These limitations can also be viewed as nothing more than an attempt to generally link the use of the judicial exception to the technological environment of a computer (see MPEP 2106.05(h)). Note, the mere instructions to apply an exception on a generic computer cannot integrate a judicial exception into a practical application.
Step 2B (claim provides an inventive concept):
The conclusions for the mere implementation using a computer are carried over and does not provide significantly more.
With respect to the "defining …." identified as insignificant extra-solution activity above when re-evaluated this element is well-understood, routine, and conventional as evidenced by the court cases in MPEP 2106.05(d)(II), "i. Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); … OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network);" and thus remains insignificant extra-solution activity that does not provide significantly more.
With respect to the " executing …. " identified as insignificant extra-solution activity above when re-evaluated this element is well-understood, routine, and conventional in displaying information as evidenced by the court cases in MPEP 2106.05(d)(II), " iv. Presenting offers and gathering statistics, OIP Techs., 788 F.3d at 1362-63, 115 USPQ2d at 1092-93" and "i. … transmitting data over a network, …Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); … OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)".
With respect to the " optimizing …" identified as insignificant extra-solution activity above when re-evaluated this element is well-understood, routine, and conventional as evidenced by the court cases in MPEP 2106.05(d)(II), "iv. Storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334; i. … transmitting data over a network, …Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); … OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)".
With respect to the “one or more processors, one or more computer-readable memories, one or more computer-readable tangible storage medium” amount to elements that have been recognized as well-understood, routine, and conventional activity in particular fields, as demonstrate by: Relevant court decision: the followings are examples of court decisions demonstrating well-understood, routine and conventional activities, see e.g., MPEP 2106.05(d)(II) and MPEP 2106.05(f)(2): Computer readable storage media comprising instructions to implement a method, e.g., see Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015).
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements when considered both individually and as an ordered combination do not amount to significantly more than the abstract idea.
Looking at the claim as a whole does not change this conclusion and the claim appears to be ineligible.
Accordingly, claim 1 is directed to an abstract idea. The remaining independent claim 8 and 15 fall short the 35 USC 101 requirement under the same rationale.
The dependent claims 2-7, 9-14 and 16-20 when analyzed and each taken as a whole are held to be patent ineligible under 35 USC 101 because the additional recited limitations fail to establish that the claims are not directed to an abstract idea.
Claim 2 recites “wherein the one or more objective-based reward functions further include hyperparameters to attribute respective weights to a series of variables in the one or more objective-based reward functions”. This limitation, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process as a form of evaluation or judgement. There is no additional elements recited which tie the abstract idea into a practical application and does not amount to significant more than the identified judicial exception. The same rationale applies to claims 9 and 16.
Claim 3 recites “wherein the state value functions comprise efficiency scores corresponding to the candidate action vectors generated by the first network”. This limitation, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process as a form of evaluation or judgement. There is no additional elements recited which tie the abstract idea into a practical application and does not amount to significant more than the identified judicial exception. The same rationale applies to claims 10 and 17.
Claim 4 recites “wherein optimizing the first and the second networks based on the observed rewards further comprises: in response to an amount of data in an experience pool reaching a predetermined threshold value, randomly sampling a batch of data to update parameters of the first network and the second network respectively”. This additional element is recited at a high level of generality and would function in its ordinary capacity for randomly sampling a batch of data to update parameters of the first network and the second network respectively, this additional element does not integrate the integrate the judicial exception into a practical application and does not amount to significantly more. The same rationale applies to claims 11 and 18.
Claim 5 recites “continuously updating the parameters of the first network and the second network respectively until the first network and the second network converge or reach a predefined maximum number of iterations”. This limitation, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process as a form of evaluation or judgement. There is no additional elements recited which tie the abstract idea into a practical application and does not amount to significant more than the identified judicial exception. The same rationale applies to claims 12 and 19.
Claim 6 recites “wherein the defined current state vector spaces comprise a series of variables related to at least a current health status and traffic conditions at a given time of at least one or more service nodes associated with the target system”. This limitation, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process as a form of evaluation or judgement. There is no additional elements recited which tie the abstract idea into a practical application and does not amount to significant more than the identified judicial exception. The same rationale applies to claims 13 and 20.
Claim 7 recites “wherein the defined action space comprises a series of performable operations corresponding to actions that are achievable by modifying one or more variables in the series of variables included within the defined current state vector”. This limitation, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process as a form of evaluation or judgement. There is no additional elements recited which tie the abstract idea into a practical application and does not amount to significant more than the identified judicial exception. The same rationale applies to claim 14.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20240403381 (involved in learning the value function through reinforcement learning (RL), where the value function is configured to take in an input of a state and an action pair, and provides a set of vectors as output. The request about a total sum of rewards to be achieved is received at an initial stage of a control sequence, and a sequence of actions is determined iteratively based on the output of the function, an observation of the current state, and the request. The function is parameterized by a neural network model that samples multiple random variables from a prefixed probability distribution).
US 20240037393 A1 (involved in training a neural network to implement a value function which for each state of the technical system. A behavior control policy is ascertained that reflects a selection of the previously carried out actions in the respective states. The deviation for the action is weighted more strongly the greater a likelihood is that the action is selected by the control policy. The control policy is trained so that it prioritizes actions that result in states for which the neural network predicts a higher value over actions that result in states for which the neural network predicts a lower value).
US 20230342425 (involved in receive state information that describes a state of a decision making agent in an environment; compute an action vector from an action embedding space based on the state information using a policy neural network of the decision making agent, wherein the policy neural network is trained using reinforcement learning based on a topology loss that constrains changes in a mapping between an action set and the action embedding space; and perform an action that modifies the state of the decision making agent in the environment based on the action vector, wherein the action is selected based on the mapping.)
US 20210181768 A1(involved in defining an action space for an objective-directed controller. A set of feature vectors and the action space are provided as input to a simulation module. The set of feature vectors are associated with a desired objective. The objective-directed controller is trained according to a reward function correlated with the desired objective by a learning module. The trained objective-directed controller is evaluated. The trained objective-directed controller is stored. The action space is defined with continuous directions. The action space is defined with discretized directions. The action space is defined with a set of actions comprising up, down and stay).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JEAN M CORRIELUS whose telephone number is (571)272-4032. The examiner can normally be reached Monday-Friday 6:30a-10p(Midflex).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ann J Lo can be reached at (571)272-9767. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JEAN M CORRIELUS/Primary Examiner, Art Unit 2159 June 27, 2026
Prosecution Insights