Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-20 are presented for examination.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 11 is rejected under 35 U.S.C. 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor regards as the invention. Claims 12-19 are rejected under 35 U.S.C. 112(b) at least by virtue of their dependency from claim 11.
Claim 11 recites, in its first step, “compiling source code that specifies a plurality of locations of a plurality of branchpoints within a program for an agentic workflow into a search space”, thereby introducing “a program for an agentic workflow”. Claim 11 subsequently recites, in its third step, “executing, based on the one or more calls and the compiled search space, a program for the agentic workflow until a primitive specifying a location of a first branchpoint included in the plurality of branchpoints is reached”. The recitation of “a program for the agentic workflow” in the third step employs the indefinite article “a” and therefore lacks proper antecedent basis, rendering it unclear whether the program that is executed is the same program recited in the compiling step, that is, the program into which the source code is compiled and within which the plurality of branchpoints reside, or instead a different program. Because the third step further requires execution to proceed “until a primitive specifying a location of a first branchpoint included in the plurality of branchpoints is reached”, and the plurality of branchpoints are defined as residing within the program of the compiling step, the bounds of claim 11 cannot be determined. The ambiguity is reinforced by parallel independent claim 20, which was amended to recite “executing, based on the one or more calls and the compiled search space, the program for the agentic workflow”, using the definite article.
For the purpose of applying prior art in the rejection under 35 U.S.C. 103 set forth below, the Examiner interprets the phrase “a program for the agentic workflow” recited in the third step of claim 11 as referring to the program recited in the compiling step of claim 11.
Appropriate correction is required.
Dependent claims 12-19 are also rejected under 35 U.S.C. 112(b) as being indefinite for failing to cure the deficiencies of their independent claim.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-16 and 18-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nagaraja (US 2020/0244605 A1) in view of Kartal (US 2020/0143206 A1), further in view of Eilon (US 12,197,560 B1), and further in view of Minka (US 8,103,598 B2).
Regarding Claim 1, Nagaraja (US 2020/0244605 A1) teaches
A computer-implemented method for executing [an agentic workflow], the method comprising:
executing, based on a first plurality of evaluation scores associated with executing the first plurality of branches, a second plurality of branches from at least a portion of the one or more subsequent branchpoints to one or more additional points within [the agentic workflow] (Nagaraja, Paragraph [0090]: “Still in the URTI example, the answers to the respective questions in the diagnostic questionnaire are assigned with a score, and an overall scoring corresponding to the user's answers to one or more questions are computed. Based on comparing an overall scoring with a pre-configured threshold, a diagnosis can be formed in turn. For example, if the user's answer scores equally or greater than a threshold pre-configured in reference to question 4, question 6 and question 7, the user's condition is diagnosed as flu.”) Examiner Comments: The reference teaches the limitation because it uses diagnostic scores from an initial set of questions to determine the further questions or steps that follow, that is, it executes a further set of branches from subsequent points based on the evaluation scores of the first set of branches.
Nagaraja did not specifically teach
an agentic workflow;
receiving, over an interface associated with a search space for the agentic workflow, one or more calls used to control an inference-time search strategy for the agentic workflow
executing, based on the one or more calls, a program for the agentic workflow until a primitive marking a location of a first branchpoint within the program;
and executing one or more additional primitives marking one or more locations of one or more subsequent branchpoints within the program, a first plurality of branches from the first branchpoint and a state of the agentic workflow at the first branchpoint to one or more subsequent branchpoints within the agentic workflow.
However, Kartal (US 2020/0143206 A1) teaches
receiving one or more calls over an interface associated with a search space for [the agentic workflow] (Kartal, Paragraph [0079]: “MCTS is a best-first search algorithm that gained traction after its breakthrough performance in Go. It has been used for many purposes, e.g., for game playing, for playtesting, and for robotics. In MCTS, a search tree is generated where each node in the tree represents a complete state of the domain and each link represents one possible valid action, leading to a child node representing the resulting state after taking an action. The root of the tree is the initial state (for example, the initial configuration of the Pommerman board including the agent location). MCTS proceeds in four phases of: selection, expansion, rollout, and backpropagation. The standard MCTS algorithm proceeds by repeatedly adding one node at a time to the current tree. Given that leaf nodes are likely to be far from terminal states, it uses random actions, a.k.a. rollouts, to estimate state-action values. The rollout policy can also be biased based on information obtained during search, or external domain knowledge. After the rollout phase, the total collected rewards during the episode is back-propagated through the tree branch, updating their empirical state-action values, and visit counts.”) Examiner Comments: Kartal teaches a search space in the form of an MCTS search tree that is accessed through the programmatic phases of the algorithm over an interface to determine the next action at each node, which provides the mechanism for invoking the search over the tree of possible action sequences.
executing, based on the one or more calls, … a program for [the agentic workflow] until a first branchpoint is reached (Kartal, Paragraph [0058]: “In step (a) selection, the UCB equation is used recursively until a node with an unexplored action is selected. Assume that nodes A and B are selected. In step (b) Expansion, node C is added to the tree.”) Examiner Comments: Kartal teaches executing the agent's program through the search tree, using the UCB equation recursively, until a node with an unexplored action is reached, which corresponds to reaching the location of a first branchpoint.
executing … a first plurality of branches from the first branchpoint and a state of [the agentic workflow] at the first branchpoint to one or more subsequent branchpoints within [the agentic workflow] (Kartal, Paragraph [0058]: “In step (b) Expansion, node C is added to the tree. In step (c) Random Rollout, a sequence of random actions is taken from node C to complete the partial game. In step (d) Backpropagation, game after rollout terminates is evaluated and the score is back-propagated from node C to the root.”) Examiner Comments: Kartal teaches that the expansion and rollout phases add child nodes and simulate execution from the first node to subsequent nodes, which corresponds to executing a first plurality of branches from the first branchpoint to one or more subsequent branchpoints.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Nagaraja's teaching into Kartal's in order to incorporate MCTS for exploring a plurality of branches and using evaluation scores to select and expand promising paths, as both references involve AI agents in decision-making processes, and this combination would improve exploration efficiency in sparse-reward environments as taught by Kartal (Kartal, Paragraph [0011]: “In some embodiments, there is provided a computer system for action guidance via Monte Carlo tree set (MCTS)/demonstrations. In some embodiments, there is provided a computer system for blending search with distribution model-free DRL methods such that search and neural network components can be executed simultaneously in an on-policy fashion.”).
Nagaraja and Kartal did not specifically teach
an agentic workflow
one or more calls used to control an inference-time search strategy for the agentic workflow;
a primitive marking a location of a first branchpoint within the program; and
one or more additional primitives marking one or more locations of one or more subsequent branchpoints within the program.
However, Eilon (US 12,197,560 B1) teaches
an agentic workflow (Eilon, Col. 69, ll. 20-67: “the AI block is enabled to use data processed by at least one of the workflow blocks preceding the AI block in a same branch as the AI block; the prompt of the AI block includes data from the source of dynamic data; the AI block is powered by an AI agent; the AI agent is selectable from a plurality of available AI agents.”) Examiner Comments: Eilon teaches a workflow in which an AI block, powered by an AI agent selectable from a plurality of available AI agents, processes data within a branch of the workflow, which corresponds to an agentic workflow.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Nagaraja and Kartal's teaching into Eilon's in order to enable facilitating efficient and organized project management, allowing users to strategize effectively, carry out tasks, and track progress so as to accomplish a specific objective, and to enable a dynamic data retrieval process that improves the functionality and accuracy of the workflow (Eilon, Summary).
Nagaraja, Kartal, and Eilon did not specifically teach
one or more calls used to control an inference-time search strategy for the agentic workflow;
a primitive marking a location of a first branchpoint within the program; and
one or more additional primitives marking one or more locations of one or more subsequent branchpoints within the program.
However, Minka (US 8,103,598 B2) teaches
one or more calls used to control an inference-time search strategy for the agentic workflow (Col 4, lines 20-52, “The iterative process transform (block 305) converts an inference method into a class which implements the IterativeProcess interface, enabling the inference procedure to be executed in a consistent way from external code.”; Col 3, ln 1-20, “The inference engine 200 also has a number of settings, which may be user specified, such as the choice of inference algorithm 207 to be used and any inference settings 208 such as the number of iterations to perform.”) Examiner Comments: Minka teaches an interface, namely the IterativeProcess interface, through which the inference procedure is executed and through which user-specified calls and settings, including the choice of inference algorithm and the inference settings, control the inference-time search strategy applied to the compiled representation.
a primitive marking a location of a first branchpoint within the program (Col 4, line 20-52, “The model analysis transform (block 301) comprises determining which variables in the input program are random and which are ordinary.”; Col 4, line 53-67, “Special statements may be used to directly define attributes on variables. These statements are removed from the output code.”) Examiner Comments: Minka teaches that the input program contains statements that are determined to be random and that special statements directly mark variables in the program, such that a statement marking a stochastic variable is a primitive that marks the location of a nondeterministic branchpoint within the program.
one or more additional primitives marking one or more locations of one or more subsequent branchpoints within the program (Col 4, lines 20-52, “The model analysis transform (block 301) comprises determining which variables in the input program are random and which are ordinary.”; Col 15, lines 1-10 “the gate transform 801 handles if and switch statements with stochastic arguments.”) Examiner Comments: Minka teaches that each random variable in the program is identified and that conditional branching is handled at if and switch statements having stochastic arguments, such that the additional stochastic statements are additional primitives that mark the locations of the subsequent branchpoints within the program.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Nagaraja, Kartal, and Eilon with Minka in order to compile the agentic workflow program, including the locations marked by its branchpoint primitives, into a search space and to define and control the inference-time search strategy through a consistent interface independently of the workflow logic, because Minka expressly teaches that doing so allows different inference algorithms to be used and even changed without requiring the model to be re-written, thereby reducing the time and engineering effort needed to implement, modify, and reuse search strategies, which yields the predictable benefit of more efficiently exploring and optimizing inference-time search strategies for the agentic workflow as set forth below.
Regarding Claim 2, Nagaraja, Kartal, Eilon, and Minka teach
The computer-implemented method of claim 1, further comprising determining a final output of the agentic workflow based on a second plurality of evaluation scores associated with executing the second plurality of branches
determining a final output of the agentic workflow based on a second plurality of evaluation scores associated with executing the second plurality of branches (Nagaraja, Paragraph [0090]: “Still in the URTI example, the answers to the respective questions in the diagnostic questionnaire are assigned with a score, and an overall scoring corresponding to the user's answers to one or more questions are computed. Based on comparing an overall scoring with a pre-configured threshold, a diagnosis can be formed in turn. For example, if the user's answer scores equally or greater than a threshold pre-configured in reference to question 4, question 6 and question 7, the user's condition is diagnosed as flu. For another example, if the user's answer scores equally or greater than a threshold pre-configured in reference to question 3, question 2 and question 5, the user's condition is diagnosed as sore throat.”) Examiner Comments: The reference teaches the limitation because it determines the final diagnosis, which is the output, based on the evaluation scores accumulated across the workflow steps.
Regarding Claim 3, Nagaraja, Kartal, Eilon, and Minka teach
The computer-implemented method of claim 2, wherein determining the final output comprises: determining that a point at which a branch included in the second plurality of branches terminates corresponds to an endpoint in the agentic workflow; and determining the final output based on a subsequent state of the agentic workflow at the point (Nagaraja, Paragraph [0077]: “A workflow is a collection of steps corresponding to a chain of events and processes/actions starting from a first engagement (e.g., an intake) of a user who sends a request for a service, until the requested service is fulfilled and the user released from the workflow. Taking medical or clinical services for example, a medical workflow may include a set of one or more steps (ordered or not, or partially ordered) that encompass a patient intake with pertinent diagnostic processes (e.g., questions, lab orders, modality orders, and the like); diagnosis of a condition; treatment (e.g., prescription of medication treatment, physical treatment, and the like) of a condition; follow-ups; etc.”) Examiner Comments: The reference teaches the limitation because the workflow terminates when the requested service is fulfilled, which is the endpoint, and the final output is determined from the state at that point.
Regarding Claim 4, Nagaraja, Kartal, Eilon, and Minka teach
The computer-implemented method of claim 1.
Nagaraja did not specifically teach
further comprising compiling the program for the agentic workflow into the search space that includes the first branchpoint, the one or more subsequent branchpoints, and the one or more additional points, wherein the first plurality of branches and the second plurality of branches are executed based on a search of the search space.
However, Minka teaches
compiling the program for the agentic workflow into the search space that includes the first branchpoint, the one or more subsequent branchpoints, and the one or more additional points (Col 1, line 44-55, “A compiler for probabilistic programs is described. The inputs to the compiler are a definition of a model and a set of inference queries. The model definition is written as a probabilistic program which describes a system of interest. The compiler transforms statements in the probabilistic program to generate source code which performs the specified queries on the model.”; Minka, Detailed Description (block 302): “The channel transform (block 302) performs the equivalent of creating a factor graph in code where edges are variables and functions are nodes.”) Examiner Comments: Minka teaches a compiler that transforms the statements of the input program into a factor graph in code whose nodes and edges form the search space that includes the branchpoints and additional points.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Nagaraja, Kartal, and Eilon with Minka in order to compile the agentic workflow program, including the locations marked by its branchpoint primitives, into a search space and to define and control the inference-time search strategy through a consistent interface independently of the workflow logic, because Minka expressly teaches that doing so allows different inference algorithms to be used and even changed without requiring the model to be re-written, thereby reducing the time and engineering effort needed to implement, modify, and reuse search strategies, which yields the predictable benefit of more efficiently exploring and optimizing inference-time search strategies for the agentic workflow as set forth below.
Kartal further teaches
wherein the first plurality of branches and the second plurality of branches are executed based on a search of the search space (Kartal, Paragraph [0079]: “MCTS is a best-first search algorithm that gained traction after its breakthrough performance in Go. It has been used for many purposes, e.g., for game playing, for playtesting, and for robotics. In MCTS, a search tree is generated where each node in the tree represents a complete state of the domain and each link represents one possible valid action, leading to a child node representing the resulting state after taking an action. The root of the tree is the initial state (for example, the initial configuration of the Pommerman board including the agent location). MCTS proceeds in four phases of: selection, expansion, rollout, and backpropagation. The standard MCTS algorithm proceeds by repeatedly adding one node at a time to the current tree. Given that leaf nodes are likely to be far from terminal states, it uses random actions, a.k.a. rollouts, to estimate state-action values. The rollout policy can also be biased based on information obtained during search, or external domain knowledge. After the rollout phase, the total collected rewards during the episode is back-propagated through the tree branch, updating their empirical state-action values, and visit counts.”) Examiner Comments: Kartal teaches that the branches are executed by performing an MCTS search of the search tree, which corresponds to executing the branches based on a search of the search space.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Nagaraja, Kartal, and Eilon with Minka in order to compile the agentic workflow program, including the locations marked by its branchpoint primitives, into a search space and to define and control the inference-time search strategy through a consistent interface independently of the workflow logic, because Minka expressly teaches that doing so allows different inference algorithms to be used and even changed without requiring the model to be re-written, thereby reducing the time and engineering effort needed to implement, modify, and reuse search strategies, which yields the predictable benefit of more efficiently exploring and optimizing inference-time search strategies for the agentic workflow as set forth below.
Regarding Claim 5, Nagaraja, Kartal, Eilon, and Minka teach
The computer-implemented method of claim 4.
Nagaraja did not specifically teach
wherein the program for the agentic workflow further specifies a first set of locations of the one or more additional points; and a second set of locations at which the first plurality of evaluation scores are recorded.
However, Kartal teaches
wherein the program for the agentic workflow further specifies a first set of locations of the one or more additional points; and a second set of locations at which the first plurality of evaluation scores are recorded (Kartal, Paragraph [0079]: “MCTS is a best-first search algorithm that gained traction after its breakthrough performance in Go. It has been used for many purposes, e.g., for game playing, for playtesting, and for robotics. In MCTS, a search tree is generated where each node in the tree represents a complete state of the domain and each link represents one possible valid action, leading to a child node representing the resulting state after taking an action. The root of the tree is the initial state (for example, the initial configuration of the Pommerman board including the agent location). MCTS proceeds in four phases of: selection, expansion, rollout, and backpropagation. The standard MCTS algorithm proceeds by repeatedly adding one node at a time to the current tree. Given that leaf nodes are likely to be far from terminal states, it uses random actions, a.k.a. rollouts, to estimate state-action values. The rollout policy can also be biased based on information obtained during search, or external domain knowledge. After the rollout phase, the total collected rewards during the episode is back-propagated through the tree branch, updating their empirical state-action values, and visit counts.”; Kartal, Paragraph [0053]: “A model based DRL architecture based on Deep-Q-Network may predict: Q-values, next frame, rewards, and a binary terminal flag that predicts whether the episode will end or not.”) Examiner Comments: The reference teaches the limitation because Q-values, visit counts, and rewards are recorded at the nodes of the tree, which represent the locations of the additional points and the locations at which the evaluation scores are recorded.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Nagaraja's teaching into Kartal's in order to incorporate MCTS for exploring a plurality of branches and using evaluation scores to select and expand promising paths, as both references involve AI agents in decision-making processes, and this combination would improve exploration efficiency in sparse-reward environments as taught by Kartal (Kartal, Paragraph [0011]: “In some embodiments, there is provided a computer system for action guidance via Monte Carlo tree set (MCTS)/demonstrations. In some embodiments, there is provided a computer system for blending search with distribution model-free DRL methods such that search and neural network components can be executed simultaneously in an on-policy fashion.”).
Regarding Claim 6, Nagaraja, Kartal, Eilon, and Minka teach
The computer-implemented method of claim 4.
Nagaraja did not specifically teach
wherein the search of the search space is performed based on a search policy associated with the agentic workflow.
However, Kartal teaches
wherein the search of the search space is performed based on a search policy associated with the agentic workflow (Kartal, Paragraph [0066]: “AlphaGo defeated one of the strongest human Go players in the world. It uses imitation learning by pretraining RL's policy network from human expert games with supervised learning. Then, its policy and value networks keep improving by selfplay games via DRL. Finally, an MCTS search is employed where a policy network narrows down move selection (i.e., effectively reducing the branching factor) and a value network helps with leaf evaluation (i.e., reducing the number of costly rollouts to estimate state-value of leaf nodes).”) Examiner Comments: The reference teaches the limitation because the MCTS search uses a policy network as the search policy that narrows down move selection during the search.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Nagaraja's teaching into Kartal's in order to incorporate MCTS for exploring a plurality of branches and using evaluation scores to select and expand promising paths, as both references involve AI agents in decision-making processes, and this combination would improve exploration efficiency in sparse-reward environments as taught by Kartal (Kartal, Paragraph [0011]: “In some embodiments, there is provided a computer system for action guidance via Monte Carlo tree set (MCTS)/demonstrations. In some embodiments, there is provided a computer system for blending search with distribution model-free DRL methods such that search and neural network components can be executed simultaneously in an on-policy fashion.”).
Regarding Claim 7, Nagaraja, Kartal, Eilon, and Minka teach
The computer-implemented method of claim 1.
Nagaraja did not specifically teach
wherein the one or more calls are associated with at least one of: searching over the first plurality of branches and the second plurality of branches; a set of costs associated with the first plurality of branches and the second plurality of branches; and a number of branches included in the first plurality of branches.
However, Kartal teaches
wherein the one or more calls are associated with at least one of: searching over the first plurality of branches and the second plurality of branches; a set of costs associated with the first plurality of branches and the second plurality of branches; and a number of branches included in the first plurality of branches (Kartal, Paragraph [0066]: “AlphaGo defeated one of the strongest human Go players in the world. It uses imitation learning by pretraining RL's policy network from human expert games with supervised learning. Then, its policy and value networks keep improving by selfplay games via DRL. Finally, an MCTS search is employed where a policy network narrows down move selection (i.e., effectively reducing the branching factor) and a value network helps with leaf evaluation (i.e., reducing the number of costly rollouts to estimate state-value of leaf nodes).”) Examiner Comments: The reference teaches the limitation under the broadest reasonable interpretation because the branching factor specifies the number of branches and the costly rollouts represent a set of costs associated with the branches.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Nagaraja's teaching into Kartal's in order to incorporate MCTS for exploring a plurality of branches and using evaluation scores to select and expand promising paths, as both references involve AI agents in decision-making processes, and this combination would improve exploration efficiency in sparse-reward environments as taught by Kartal (Kartal, Paragraph [0011]: “In some embodiments, there is provided a computer system for action guidance via Monte Carlo tree set (MCTS)/demonstrations. In some embodiments, there is provided a computer system for blending search with distribution model-free DRL methods such that search and neural network components can be executed simultaneously in an on-policy fashion.”).
Regarding Claim 8, Nagaraja, Kartal, Eilon, and Minka teach
The computer-implemented method of claim 1.
Nagaraja did not specifically teach
further comprising storing, in a search tree associated with the agentic workflow: a first node representing the first branchpoint and a first state of the agentic workflow at the first branchpoint; and a plurality of nodes that are children of the first node, wherein each of the plurality of nodes represents (i) a branchpoint included in the one or more subsequent branchpoints and (ii) an additional state of the agentic workflow after a corresponding execution branch that is included in the first plurality of branches has terminated at the branchpoint.
However, Kartal teaches
storing, in a search tree associated with the agentic workflow: a first node representing the first branchpoint and a first state at the first branchpoint; and a plurality of nodes that are children of the first node, each representing a subsequent branchpoint and an additional state after a corresponding branch has terminated (Kartal, Paragraph [0079]: “MCTS is a best-first search algorithm that gained traction after its breakthrough performance in Go. It has been used for many purposes, e.g., for game playing, for playtesting, and for robotics. In MCTS, a search tree is generated where each node in the tree represents a complete state of the domain and each link represents one possible valid action, leading to a child node representing the resulting state after taking an action. The root of the tree is the initial state (for example, the initial configuration of the Pommerman board including the agent location). MCTS proceeds in four phases of: selection, expansion, rollout, and backpropagation. The standard MCTS algorithm proceeds by repeatedly adding one node at a time to the current tree. Given that leaf nodes are likely to be far from terminal states, it uses random actions, a.k.a. rollouts, to estimate state-action values. The rollout policy can also be biased based on information obtained during search, or external domain knowledge. After the rollout phase, the total collected rewards during the episode is back-propagated through the tree branch, updating their empirical state-action values, and visit counts.”) Examiner Comments: The reference teaches the limitation because the root node of the MCTS tree represents the initial state and first branchpoint and the child nodes represent the resulting states after actions, which are the subsequent branchpoints and additional states.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Nagaraja's teaching into Kartal's in order to incorporate MCTS for exploring a plurality of branches and using evaluation scores to select and expand promising paths, as both references involve AI agents in decision-making processes, and this combination would improve exploration efficiency in sparse-reward environments as taught by Kartal (Kartal, Paragraph [0011]: “In some embodiments, there is provided a computer system for action guidance via Monte Carlo tree set (MCTS)/demonstrations. In some embodiments, there is provided a computer system for blending search with distribution model-free DRL methods such that search and neural network components can be executed simultaneously in an on-policy fashion.”).
Regarding Claim 9, Nagaraja, Kartal, Eilon, and Minka teach
The computer-implemented method of claim 1.
Nagaraja did not specifically teach
wherein the first plurality of branches is executed based on at least one of a branching factor associated with the first branchpoint or a beam width associated with the first branchpoint.
However, Kartal teaches
wherein the first plurality of branches is executed based on at least one of a branching factor associated with the first branchpoint or a beam width associated with the first branchpoint (Kartal, Paragraph [0066]: “AlphaGo defeated one of the strongest human Go players in the world. It uses imitation learning by pretraining RL's policy network from human expert games with supervised learning. Then, its policy and value networks keep improving by selfplay games via DRL. Finally, an MCTS search is employed where a policy network narrows down move selection (i.e., effectively reducing the branching factor) and a value network helps with leaf evaluation (i.e., reducing the number of costly rollouts to estimate state-value of leaf nodes).”) Examiner Comments: The reference teaches the limitation because MCTS executes branches based on the branching factor that the policy network reduces, and a beam width is an analogous search parameter that limits expansions.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Nagaraja's teaching into Kartal's in order to incorporate MCTS for exploring a plurality of branches and using evaluation scores to select and expand promising paths, as both references involve AI agents in decision-making processes, and this combination would improve exploration efficiency in sparse-reward environments as taught by Kartal (Kartal, Paragraph [0011]: “In some embodiments, there is provided a computer system for action guidance via Monte Carlo tree set (MCTS)/demonstrations. In some embodiments, there is provided a computer system for blending search with distribution model-free DRL methods such that search and neural network components can be executed simultaneously in an on-policy fashion.”).
Regarding Claim 10, Nagaraja, Kartal, Eilon, and Minka teach
The computer-implemented method of claim 1, wherein the state of the agentic workflow comprises at least one of a set of variables, a set of values for the set of variables, or a final output of the agentic workflow (Nagaraja, Paragraph [0090]: “In some embodiments, the scores are weighted based on other factors such as age, geo-locations, other known medical conditions, financial status, and the like. In this scenario, an overall scoring is computed based on both the score and its respective weight.”) Examiner Comments: The reference teaches the limitation because the state includes variables such as age and known medical conditions and corresponding values such as the computed scores, which lead to a final output such as the diagnosis.
Regarding Claim 11, Nagaraja (US 2020/0244605 A1) teaches
One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of: executing, based on a first plurality of evaluation scores associated with executing the first plurality of branches, a second plurality of branches from at least a portion of the one or more subsequent branchpoints to one or more additional points within [the agentic workflow] (Nagaraja, Paragraph [0090]: “Still in the URTI example, the answers to the respective questions in the diagnostic questionnaire are assigned with a score, and an overall scoring corresponding to the user's answers to one or more questions are computed. Based on comparing an overall scoring with a pre-configured threshold, a diagnosis can be formed in turn. For example, if the user's answer scores equally or greater than a threshold pre-configured in reference to question 4, question 6 and question 7, the user's condition is diagnosed as flu.”) Examiner Comments: The reference teaches the limitation because it uses the evaluation scores of the first set of branches to determine and execute the further set of branches from the subsequent points.
Nagaraja did not specifically teach
compiling source code that specifies a plurality of locations of a plurality of branchpoints within a program for an agentic workflow into a search space that includes (i) a first plurality of operations applied to inputs into the plurality of branchpoints and (ii) a second plurality of operations applied to results associated with executing the plurality of branchpoints;
receiving one or more calls over an interface associated with the search space;
executing based on the one or more calls and the compiled search space, a program for the agentic workflow until a primitive specifying a location of a first branchpoint included in the plurality of branchpoints is reached; and
executing based on one or more additional primitives specifying one or more locations of one or more subsequent branchpoints included in the plurality of branchpoints a first plurality of branches from the first branchpoint and a state of the agentic workflow at the first branchpoint to the one or more subsequent branchpoints within the agentic workflow.
However, Kartal (US 2020/0143206 A1) teaches
receiving one or more calls over an interface associated with the [compiled] search space (Kartal, Paragraph [0079]: “MCTS is a best-first search algorithm that gained traction after its breakthrough performance in Go. It has been used for many purposes, e.g., for game playing, for playtesting, and for robotics. In MCTS, a search tree is generated where each node in the tree represents a complete state of the domain and each link represents one possible valid action, leading to a child node representing the resulting state after taking an action. The root of the tree is the initial state (for example, the initial configuration of the Pommerman board including the agent location). MCTS proceeds in four phases of: selection, expansion, rollout, and backpropagation. The standard MCTS algorithm proceeds by repeatedly adding one node at a time to the current tree. Given that leaf nodes are likely to be far from terminal states, it uses random actions, a.k.a. rollouts, to estimate state-action values. The rollout policy can also be biased based on information obtained during search, or external domain knowledge. After the rollout phase, the total collected rewards during the episode is back-propagated through the tree branch, updating their empirical state-action values, and visit counts.”) Examiner Comments: Kartal teaches a search space in the form of an MCTS search tree that is accessed through the programmatic phases of the algorithm over an interface, which provides the mechanism for receiving the calls that invoke the search over the compiled tree of possible action sequences.
executing … a program for the agentic workflow … until a first branchpoint included in the plurality of branchpoints is reached (Kartal, Paragraph [0058]: “In step (a) selection, the UCB equation is used recursively until a node with an unexplored action is selected. Assume that nodes A and B are selected. In step (b) Expansion, node C is added to the tree.”) Examiner Comments: Kartal teaches executing the agent's program through the search tree, using the UCB equation recursively, until a node with an unexplored action is reached, which corresponds to reaching the location of a first branchpoint.
executing … a first plurality of branches from the first branchpoint and a state of the agentic workflow at the first branchpoint to the one or more subsequent branchpoints within the agentic workflow (Kartal, Paragraph [0058]: “In step (b) Expansion, node C is added to the tree. In step (c) Random Rollout, a sequence of random actions is taken from node C to complete the partial game. In step (d) Backpropagation, game after rollout terminates is evaluated and the score is back-propagated from node C to the root.”) Examiner Comments: Kartal teaches that the expansion and rollout phases add child nodes and simulate execution from the first node to subsequent nodes, which corresponds to executing a first plurality of branches from the first branchpoint to the one or more subsequent branchpoints.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Nagaraja's teaching into Kartal's in order to incorporate MCTS for exploring a plurality of branches and using evaluation scores to select and expand promising paths, as both references involve AI agents in decision-making processes, and this combination would improve exploration efficiency in sparse-reward environments as taught by Kartal (Kartal, Paragraph [0011]: “In some embodiments, there is provided a computer system for action guidance via Monte Carlo tree set (MCTS)/demonstrations. In some embodiments, there is provided a computer system for blending search with distribution model-free DRL methods such that search and neural network components can be executed simultaneously in an on-policy fashion.”).
Nagaraja and Kartal did not specifically teach
an agentic workflow
compiling source code that specifies a plurality of locations of a plurality of branchpoints within a program for an agentic workflow into a search space that includes (i) a first plurality of operations applied to inputs into the plurality of branchpoints and (ii) a second plurality of operations applied to results associated with executing the plurality of branchpoints;
a primitive specifying a location of a first branchpoint included in the plurality of branchpoints; and
one or more additional primitives specifying one or more locations of one or more subsequent branchpoints included in the plurality of branchpoints.
However, Eilon (US 12,197,560 B1) teaches
an agentic workflow (Eilon, Col. 69, ll. 20-67: “the AI block is enabled to use data processed by at least one of the workflow blocks preceding the AI block in a same branch as the AI block; the prompt of the AI block includes data from the source of dynamic data; the AI block is powered by an AI agent; the AI agent is selectable from a plurality of available AI agents.”) Examiner Comments: Eilon teaches a workflow in which an AI block, powered by an AI agent selectable from a plurality of available AI agents, processes data within a branch of the workflow, which corresponds to an agentic workflow.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Nagaraja and Kartal's teaching into Eilon's in order to enable facilitating efficient and organized project management, allowing users to strategize effectively, carry out tasks, and track progress so as to accomplish a specific objective, and to enable a dynamic data retrieval process that improves the functionality and accuracy of the workflow (Eilon, Summary).
Nagaraja, Kartal, and Eilon did not specifically teach
compiling source code that specifies a plurality of locations of a plurality of branchpoints within a program for an agentic workflow into a search space that includes (i) a first plurality of operations applied to inputs into the plurality of branchpoints and (ii) a second plurality of operations applied to results associated with executing the plurality of branchpoints;
a primitive specifying a location of a first branchpoint included in the plurality of branchpoints; and
one or more additional primitives specifying one or more locations of one or more subsequent branchpoints included in the plurality of branchpoints.
However, Minka (US 8,103,598 B2) teaches
compiling source code that specifies a plurality of locations of a plurality of branchpoints within a program for an agentic workflow into a search space that includes (i) a first plurality of operations applied to inputs into the plurality of branchpoints and (ii) a second plurality of operations applied to results associated with executing the plurality of branchpoints (Col 1, line 44-55, “A compiler for probabilistic programs is described. The inputs to the compiler are a definition of a model and a set of inference queries. The model definition is written as a probabilistic program which describes a system of interest. The compiler transforms statements in the probabilistic program to generate source code which performs the specified queries on the model.”; Col 4, lines 20-52 “The model analysis transform (block 301) comprises determining which variables in the input program are random and which are ordinary.”; Minka, Detailed Description (block 302): “The channel transform (block 302) performs the equivalent of creating a factor graph in code where edges are variables and functions are nodes.”; Col 6, line 60-67, “The message passing transform (block 303) converts channels into pairs of forward/backwards messages and inserts message operators to compute these messages, given other messages and deterministic variables.”; Col 8, lne 36-47, “Each coin variable is treated as a message operator which takes in and generates messages. The forward and backward messages are shown and labeled.”) Examiner Comments: Minka teaches a compiler that transforms the statements of a probabilistic program, which specifies at their locations which variables are random branch-inducing variables, into a factor graph search space in which forward messages are the operations applied to the inputs into each variable and backward messages are the operations applied to the results of each variable, which corresponds to compiling source code that specifies the locations of the branchpoints into a search space including a first plurality of operations applied to inputs into the branchpoints and a second plurality of operations applied to results associated with executing the branchpoints.
a primitive specifying a location of a first branchpoint included in the plurality of branchpoints (Col 4, lines 20-52, “The model analysis transform (block 301) comprises determining which variables in the input program are random and which are ordinary.”; Col 4, line53-67, “Special statements may be used to directly define attributes on variables. These statements are removed from the output code.”) Examiner Comments: Minka teaches that the input program contains statements determined to be random and that special statements directly mark variables in the program, such that a statement marking a stochastic variable is a primitive that specifies the location of a first branchpoint within the program.
one or more additional primitives specifying one or more locations of one or more subsequent branchpoints included in the plurality of branchpoints (Col 4, line 20-52, “The model analysis transform (block 301) comprises determining which variables in the input program are random and which are ordinary.”; Col 15, line 1-5, “the gate transform 801 handles if and switch statements with stochastic arguments.”) Examiner Comments: Minka teaches that each random variable in the program is identified and that conditional branching is handled at if and switch statements having stochastic arguments, such that the additional stochastic statements are additional primitives that specify the locations of the subsequent branchpoints within the program.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Nagaraja, Kartal, and Eilon with Minka in order to compile the agentic workflow program, including the locations marked by its branchpoint primitives, into a search space and to define and control the inference-time search strategy through a consistent interface independently of the workflow logic, because Minka expressly teaches that doing so allows different inference algorithms to be used and even changed without requiring the model to be re-written, thereby reducing the time and engineering effort needed to implement, modify, and reuse search strategies, which yields the predictable benefit of more efficiently exploring and optimizing inference-time search strategies for the agentic workflow as set forth below.
Regarding Claim 12, Nagaraja, Kartal, Eilon, and Minka teach
The one or more non-transitory computer-readable media of claim 11, wherein the instructions further cause the one or more processors to perform the step of determining a final output of the agentic workflow based on a second plurality of evaluation scores associated with executing the second plurality of branches (Nagaraja, Paragraph [0090]: “Still in the URTI example, the answers to the respective questions in the diagnostic questionnaire are assigned with a score, and an overall scoring corresponding to the user's answers to one or more questions are computed. Based on comparing an overall scoring with a pre-configured threshold, a diagnosis can be formed in turn. For example, if the user's answer scores equally or greater than a threshold pre-configured in reference to question 4, question 6 and question 7, the user's condition is diagnosed as flu. For another example, if the user's answer scores equally or greater than a threshold pre-configured in reference to question 3, question 2 and question 5, the user's condition is diagnosed as sore throat.”) Examiner Comments: The reference teaches the limitation because it determines the final diagnosis, which is the output, based on the evaluation scores accumulated across the workflow steps.
Regarding Claim 13, Nagaraja, Kartal, Eilon, and Minka teach
The one or more non-transitory computer-readable media of claim 12.
Nagaraja did not specifically teach
wherein determining the final output comprises: determining that a second branchpoint at which an execution branch included in the second plurality of branches terminates (i) corresponds to an endpoint in the agentic workflow and (ii) is associated with a highest score included in the second plurality of evaluation scores; and determining the final output based on a subsequent state of the agentic workflow at the second branchpoint.
However, Kartal teaches
determining that a second branchpoint at which an execution branch terminates corresponds to an endpoint and is associated with a highest score, and determining the final output based on a subsequent state at the second branchpoint (Kartal, Paragraph [0088]: “We compute the loss term for the terminal state prediction head, TP, by using mean squared error between the predicted value of closeness to a terminal state of any given state (i.e., y^p) and the target values approximately computed from completed episodes (i.e., y). We assume that the target for ith state can be approximated with y_i = i/N implying y_N = 1 for the actual terminal state and y_0 = 0 for the initial state for each episode, and intermediate values are linearly interpolated between [0, 1].”) Examiner Comments: The reference teaches the limitation because closeness to a terminal state is computed for the nodes and the child associated with the highest value at termination is selected to determine the output.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Nagaraja's teaching into Kartal's in order to incorporate MCTS for exploring a plurality of branches and using evaluation scores to select and expand promising paths, as both references involve AI agents in decision-making processes, and this combination would improve exploration efficiency in sparse-reward environments as taught by Kartal (Kartal, Paragraph [0011]: “In some embodiments, there is provided a computer system for action guidance via Monte Carlo tree set (MCTS)/demonstrations. In some embodiments, there is provided a computer system for blending search with distribution model-free DRL methods such that search and neural network components can be executed simultaneously in an on-policy fashion.”).
Regarding Claim 14, Nagaraja, Kartal, Eilon, and Minka teach
The one or more non-transitory computer-readable media of claim 11.
Nagaraja did not specifically teach
wherein executing the first plurality of branches comprises: generating, based on the state of the agentic workflow at the first branchpoint, a plurality of execution threads corresponding to the first plurality of branches; and executing each execution thread in the plurality of execution threads until a corresponding branchpoint included in the one or more subsequent branchpoints is reached.
However, Kartal teaches
generating a plurality of execution threads corresponding to the first plurality of branches and executing each thread until a corresponding subsequent branchpoint is reached (Kartal, Paragraph [0084]: “Within A3C's asynchronous distributed architecture, all the CPU workers perform agent-environment interactions with their policy networks, see FIG. 3A. In the PI-A3C framework 750, k≥1 CPU workers are assigned (i.e., experiments are performed with different k values) to perform MCTS-based planning for agent-environment interaction based on the agent's observations, while also keeping track of how its policy network would perform for those cases, see FIG. 7.”) Examiner Comments: The reference teaches the limitation because multiple CPU workers, which are execution threads, perform agent-environment interactions that execute the branches from the state until the next node is reached.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Nagaraja's teaching into Kartal's in order to incorporate MCTS for exploring a plurality of branches and using evaluation scores to select and expand promising paths, as both references involve AI agents in decision-making processes, and this combination would improve exploration efficiency in sparse-reward environments as taught by Kartal (Kartal, Paragraph [0011]: “In some embodiments, there is provided a computer system for action guidance via Monte Carlo tree set (MCTS)/demonstrations. In some embodiments, there is provided a computer system for blending search with distribution model-free DRL methods such that search and neural network components can be executed simultaneously in an on-policy fashion.”).
Regarding Claim 15, Nagaraja, Kartal, Eilon, and Minka teach
The one or more non-transitory computer-readable media of claim 11.
Nagaraja did not specifically teach
wherein executing the first plurality of branches comprises performing a search of the search space that includes the first branchpoint and the one or more subsequent branchpoints based on one or more search parameters associated with the first branchpoint.
However, Kartal teaches
performing a search of the search space that includes the first branchpoint and the one or more subsequent branchpoints based on one or more search parameters associated with the first branchpoint (Kartal, Paragraph [0079]: “MCTS is a best-first search algorithm that gained traction after its breakthrough performance in Go. It has been used for many purposes, e.g., for game playing, for playtesting, and for robotics. In MCTS, a search tree is generated where each node in the tree represents a complete state of the domain and each link represents one possible valid action, leading to a child node representing the resulting state after taking an action. The root of the tree is the initial state (for example, the initial configuration of the Pommerman board including the agent location). MCTS proceeds in four phases of: selection, expansion, rollout, and backpropagation. The standard MCTS algorithm proceeds by repeatedly adding one node at a time to the current tree. Given that leaf nodes are likely to be far from terminal states, it uses random actions, a.k.a. rollouts, to estimate state-action values. The rollout policy can also be biased based on information obtained during search, or external domain knowledge. After the rollout phase, the total collected rewards during the episode is back-propagated through the tree branch, updating their empirical state-action values, and visit counts.”) Examiner Comments: The reference teaches the limitation because executing branches in MCTS is performed by searching the tree, which includes the root and child nodes, based on search parameters such as the rollout policy.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Nagaraja's teaching into Kartal's in order to incorporate MCTS for exploring a plurality of branches and using evaluation scores to select and expand promising paths, as both references involve AI agents in decision-making processes, and this combination would improve exploration efficiency in sparse-reward environments as taught by Kartal (Kartal, Paragraph [0011]: “In some embodiments, there is provided a computer system for action guidance via Monte Carlo tree set (MCTS)/demonstrations. In some embodiments, there is provided a computer system for blending search with distribution model-free DRL methods such that search and neural network components can be executed simultaneously in an on-policy fashion.”).
Regarding Claim 16, Nagaraja, Kartal, Eilon, and Minka teach
The one or more non-transitory computer-readable media of claim 15, wherein the one or more search parameters comprise at least one of a branching factor or a beam width (Kartal, Paragraph [0066]: “AlphaGo defeated one of the strongest human Go players in the world. It uses imitation learning by pretraining RL's policy network from human expert games with supervised learning. Then, its policy and value networks keep improving by selfplay games via DRL. Finally, an MCTS search is employed where a policy network narrows down move selection (i.e., effectively reducing the branching factor) and a value network helps with leaf evaluation (i.e., reducing the number of costly rollouts to estimate state-value of leaf nodes).”) Examiner Comments: The reference teaches the limitation because the policy network reduces the branching factor, which is a search parameter associated with the search.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Nagaraja's teaching into Kartal's in order to incorporate MCTS for exploring a plurality of branches and using evaluation scores to select and expand promising paths, as both references involve AI agents in decision-making processes, and this combination would improve exploration efficiency in sparse-reward environments as taught by Kartal (Kartal, Paragraph [0011]: “In some embodiments, there is provided a computer system for action guidance via Monte Carlo tree set (MCTS)/demonstrations. In some embodiments, there is provided a computer system for blending search with distribution model-free DRL methods such that search and neural network components can be executed simultaneously in an on-policy fashion.”).
Regarding Claim 18, Nagaraja, Kartal, Eilon, and Minka teach
The one or more non-transitory computer-readable media of claim 11.
Nagaraja did not specifically teach
wherein the one or more calls are associated with at least one of: executing one or more branches included in the first plurality of branches; determining the state of the agentic workflow at the first branchpoint; or searching over the first plurality of branches.
However, Kartal teaches
wherein the one or more calls are associated with at least one of: executing one or more branches; determining the state at the first branchpoint; or searching over the first plurality of branches (Kartal, Paragraph [0058]: “FIG. 2 illustrates, in a sequence of tree graphs, an example of a Monte Carlo Tree Search (MCTS) 200, in accordance with some embodiments. FIG. 2 illustrates and overview of MCTS. In step (a) selection, the UCB equation is used recursively until a node with an unexplored action is selected. Assume that nodes A and B are selected. In step (b) Expansion, node C is added to the tree. In step (c) Random Rollout, a sequence of random actions is taken from node C to complete the partial game. In step (d) Backpropagation, game after rollout terminates is evaluated and the score is back-propagated from node C to the root.”) Examiner Comments: The reference teaches the limitation because the MCTS phase invocations are associated with executing branches through expansion and rollout, determining node states, and searching over branches through recursive selection.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Nagaraja's teaching into Kartal's in order to incorporate MCTS for exploring a plurality of branches and using evaluation scores to select and expand promising paths, as both references involve AI agents in decision-making processes, and this combination would improve exploration efficiency in sparse-reward environments as taught by Kartal (Kartal, Paragraph [0011]: “In some embodiments, there is provided a computer system for action guidance via Monte Carlo tree set (MCTS)/demonstrations. In some embodiments, there is provided a computer system for blending search with distribution model-free DRL methods such that search and neural network components can be executed simultaneously in an on-policy fashion.”).
Regarding Claim 19, Nagaraja, Kartal, Eilon, and Minka teach
The one or more non-transitory computer-readable media of claim 11.
Nagaraja did not specifically teach
wherein the first plurality of evaluation scores is computed based on at least one of an objective associated with the agentic workflow, a reinforcement learning reward, or a final output of the agentic workflow.
However, Kartal teaches
wherein the first plurality of evaluation scores is computed based on at least one of an objective, a reinforcement learning reward, or a final output of the agentic workflow (Kartal, Paragraph [0050]: “Reinforcement Learning (RL) seeks to maximize the sum of discounted rewards an agent collects by interacting with an environment. RL approaches mainly fall under three categories: value based methods such as tabular Q-learning or Deep-Q Network, policy based methods such as REINFORCE, and combination of value and policy based techniques, i.e., actor-critic RL methods.”) Examiner Comments: The reference teaches the limitation because the evaluation scores are state-action values computed based on the sum of discounted reinforcement learning rewards collected by the agent.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Nagaraja's teaching into Kartal's in order to incorporate MCTS for exploring a plurality of branches and using evaluation scores to select and expand promising paths, as both references involve AI agents in decision-making processes, and this combination would improve exploration efficiency in sparse-reward environments as taught by Kartal (Kartal, Paragraph [0011]: “In some embodiments, there is provided a computer system for action guidance via Monte Carlo tree set (MCTS)/demonstrations. In some embodiments, there is provided a computer system for blending search with distribution model-free DRL methods such that search and neural network components can be executed simultaneously in an on-policy fashion.”).
Regarding Claim 20, Nagaraja (US 2020/0244605 A1) teaches
A system, comprising: one or more memories that store instructions, and one or more processors that are coupled to the one or more memories and, when executing the instructions, are configured to perform the steps of:
executing, based on a first plurality of evaluation scores associated with executing the first plurality of branches, a second plurality of branches from at least a portion of the one or more subsequent branchpoints to one or more additional points within [the agentic workflow] (Nagaraja, Paragraph [0090]: “Still in the URTI example, the answers to the respective questions in the diagnostic questionnaire are assigned with a score, and an overall scoring corresponding to the user's answers to one or more questions are computed. Based on comparing an overall scoring with a pre-configured threshold, a diagnosis can be formed in turn. For example, if the user's answer scores equally or greater than a threshold pre-configured in reference to question 4, question 6 and question 7, the user's condition is diagnosed as flu.”) Examiner Comments: The reference teaches the limitation because it uses the evaluation scores of the first set of branches to determine and execute the further set of branches from the subsequent points.
Nagaraja did not specifically teach
compiling source code that comprises a plurality of primitives marking a plurality of locations of a plurality of branchpoints within a program for an agentic workflow into a search space that includes (i) a first plurality of operations applied to inputs into the plurality of branchpoints and (ii) a second plurality of operations applied to results associated with executing the plurality of branchpoints
receiving, over an interface associated with the search space, one or more calls used to control an inference-time search strategy for the agentic workflow;
executing, based on the one or more calls and the compiled search space, the program for the agentic workflow until a primitive that is included in the plurality of primitives and marks a location of a first branchpoint included in the plurality of branchpoints;
and
executing based on one or more additional primitives that are included in the plurality of primitives and mark one or more locations of one or more subsequent branchpoints included in the plurality of branchpoints a first plurality of branches from the first branchpoint and a state of the agentic workflow at the first branchpoint to the one or more subsequent branchpoints within the agentic workflow.
However, Kartal (US 2020/0143206 A1) teaches
receiving, over an interface associated with the search space, one or more calls… (Kartal, Paragraph [0079]: “MCTS is a best-first search algorithm that gained traction after its breakthrough performance in Go. It has been used for many purposes, e.g., for game playing, for playtesting, and for robotics. In MCTS, a search tree is generated where each node in the tree represents a complete state of the domain and each link represents one possible valid action, leading to a child node representing the resulting state after taking an action. The root of the tree is the initial state (for example, the initial configuration of the Pommerman board including the agent location). MCTS proceeds in four phases of: selection, expansion, rollout, and backpropagation. The standard MCTS algorithm proceeds by repeatedly adding one node at a time to the current tree. Given that leaf nodes are likely to be far from terminal states, it uses random actions, a.k.a. rollouts, to estimate state-action values. The rollout policy can also be biased based on information obtained during search, or external domain knowledge. After the rollout phase, the total collected rewards during the episode is back-propagated through the tree branch, updating their empirical state-action values, and visit counts.”) Examiner Comments: Kartal teaches a search space in the form of an MCTS search tree that is accessed through the programmatic phases of the algorithm over an interface, which provides the mechanism for receiving the calls that invoke the search over the tree of possible action sequences.
executing … the program for the agentic workflow until a first branchpoint included in the plurality of branchpoints is reached (Kartal, Paragraph [0058]: “In step (a) selection, the UCB equation is used recursively until a node with an unexplored action is selected. Assume that nodes A and B are selected. In step (b) Expansion, node C is added to the tree.”) Examiner Comments: Kartal teaches executing the agent's program through the search tree, using the UCB equation recursively, until a node with an unexplored action is reached, which corresponds to reaching the location of a first branchpoint.
executing … a first plurality of branches from the first branchpoint and a state of the agentic workflow at the first branchpoint to the one or more subsequent branchpoints within the agentic workflow (Kartal, Paragraph [0058]: “In step (b) Expansion, node C is added to the tree. In step (c) Random Rollout, a sequence of random actions is taken from node C to complete the partial game. In step (d) Backpropagation, game after rollout terminates is evaluated and the score is back-propagated from node C to the root.”) Examiner Comments: Kartal teaches that the expansion and rollout phases add child nodes and simulate execution from the first node to subsequent nodes, which corresponds to executing a first plurality of branches from the first branchpoint to the one or more subsequent branchpoints.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Nagaraja's teaching into Kartal's in order to incorporate MCTS for exploring a plurality of branches and using evaluation scores to select and expand promising paths, as both references involve AI agents in decision-making processes, and this combination would improve exploration efficiency in sparse-reward environments as taught by Kartal (Kartal, Paragraph [0011]: “In some embodiments, there is provided a computer system for action guidance via Monte Carlo tree set (MCTS)/demonstrations. In some embodiments, there is provided a computer system for blending search with distribution model-free DRL methods such that search and neural network components can be executed simultaneously in an on-policy fashion.”).
Nagaraja and Kartal did not specifically teach
an agentic workflow
compiling source code that comprises a plurality of primitives marking a plurality of locations of a plurality of branchpoints within a program for an agentic workflow into a search space that includes (i) a first plurality of operations applied to inputs into the plurality of branchpoints and (ii) a second plurality of operations applied to results associated with executing the plurality of branchpoints;
one or more calls used to control an inference-time search strategy for the agentic workflow;
a primitive that is included in the plurality of primitives and marks a location of a first branchpoint included in the plurality of branchpoints; and
one or more additional primitives that are included in the plurality of primitives and mark one or more locations of one or more subsequent branchpoints included in the plurality of branchpoints.
However, Eilon (US 12,197,560 B1) teaches
an agentic workflow (Eilon, Col. 69, ll. 20-67: “the AI block is enabled to use data processed by at least one of the workflow blocks preceding the AI block in a same branch as the AI block; the prompt of the AI block includes data from the source of dynamic data; the AI block is powered by an AI agent; the AI agent is selectable from a plurality of available AI agents.”) Examiner Comments: Eilon teaches a workflow in which an AI block, powered by an AI agent selectable from a plurality of available AI agents, processes data within a branch of the workflow, which corresponds to an agentic workflow.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Nagaraja and Kartal's teaching into Eilon's in order to enable facilitating efficient and organized project management, allowing users to strategize effectively, carry out tasks, and track progress so as to accomplish a specific objective, and to enable a dynamic data retrieval process that improves the functionality and accuracy of the workflow (Eilon, Summary).
Nagaraja, Kartal, and Eilon did not specifically teach
compiling source code that comprises a plurality of primitives marking a plurality of locations of a plurality of branchpoints within a program for an agentic workflow into a search space that includes (i) a first plurality of operations applied to inputs into the plurality of branchpoints and (ii) a second plurality of operations applied to results associated with executing the plurality of branchpoints;
one or more calls used to control an inference-time search strategy for the agentic workflow;
a primitive that is included in the plurality of primitives and marks a location of a first branchpoint included in the plurality of branchpoints; and
one or more additional primitives that are included in the plurality of primitives and mark one or more locations of one or more subsequent branchpoints included in the plurality of branchpoints.
However, Minka (US 8,103,598 B2) teaches
one or more memories that store instructions, and one or more processors that are coupled to the one or more memories and, when executing the instructions, are configured to perform the steps (Col 24, lines 39-48, “Computing-based device 1000 comprises one or more processors 1001 which may be microprocessors, controllers or any other suitable type of processors for processing computing executable instructions to control the operation of the device.”) Examiner Comments: Minka teaches a computing-based device having one or more processors that process executable instructions stored in a memory, which corresponds to the recited one or more memories that store instructions and one or more processors coupled thereto and configured to perform the steps.
compiling source code that comprises a plurality of primitives marking a plurality of locations of a plurality of branchpoints within a program for an agentic workflow into a search space that includes (i) a first plurality of operations applied to inputs into the plurality of branchpoints and (ii) a second plurality of operations applied to results associated with executing the plurality of branchpoints (Col 1, lines 44-54, “A compiler for probabilistic programs is described. The inputs to the compiler are a definition of a model and a set of inference queries. The model definition is written as a probabilistic program which describes a system of interest. The compiler transforms statements in the probabilistic program to generate source code which performs the specified queries on the model.”; Col 4, lines 20-52, “The model analysis transform (block 301) comprises determining which variables in the input program are random and which are ordinary.”; Col 6, line 59-67, “The channel transform (block 302) performs the equivalent of creating a factor graph in code where edges are variables and functions are nodes.”; Col 8, line 27-37, “The message passing transform (block 303) converts channels into pairs of forward/backwards messages and inserts message operators to compute these messages, given other messages and deterministic variables.”; Col 8, lines 37-44, FIG. 4: “Each coin variable is treated as a message operator which takes in and generates messages. The forward and backward messages are shown and labeled.”) Examiner Comments: Minka teaches a compiler that transforms the statements of a probabilistic program, which comprises stochastic statements marking at their locations which variables are random branch-inducing variables, into a factor graph search space in which forward messages are operations applied to the inputs into each variable and backward messages are operations applied to the results of each variable, which corresponds to compiling source code comprising a plurality of primitives marking the locations of the branchpoints into a search space including the recited first and second pluralities of operations.
one or more calls used to control an inference-time search strategy for the agentic workflow (Col 4, lines 20-52 “The iterative process transform (block 305) converts an inference method into a class which implements the IterativeProcess interface, enabling the inference procedure to be executed in a consistent way from external code.”; Col 3, lines 1-20, “The inference engine 200 also has a number of settings, which may be user specified, such as the choice of inference algorithm 207 to be used and any inference settings 208 such as the number of iterations to perform.”) Examiner Comments: Minka teaches an interface, namely the IterativeProcess interface, through which the inference procedure is executed and through which user-specified calls and settings, including the choice of inference algorithm and the inference settings, control the inference-time search strategy applied to the compiled representation.
a primitive that is included in the plurality of primitives and marks a location of a first branchpoint included in the plurality of branchpoints (Col 4, lines 20-52, “The model analysis transform (block 301) comprises determining which variables in the input program are random and which are ordinary.”; Col 4, lines 52-67, “Special statements may be used to directly define attributes on variables. These statements are removed from the output code.”) Examiner Comments: Minka teaches that the program comprises statements determined to be random and that special statements directly mark variables in the program, such that a primitive included in the program marks the location of a first branchpoint within the program.
one or more additional primitives that are included in the plurality of primitives and mark one or more locations of one or more subsequent branchpoints included in the plurality of branchpoints (Col 4, lines 20-52, “The model analysis transform (block 301) comprises determining which variables in the input program are random and which are ordinary.”; Col 15, lines 1-5, “the gate transform 801 handles if and switch statements with stochastic arguments.”) Examiner Comments: Minka teaches that each random variable in the program is identified and that conditional branching is handled at if and switch statements having stochastic arguments, such that the additional stochastic statements are additional primitives that mark the locations of the subsequent branchpoints within the program.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Nagaraja, Kartal, and Eilon with Minka in order to compile the agentic workflow program, including the locations marked by its branchpoint primitives, into a search space and to define and control the inference-time search strategy through a consistent interface independently of the workflow logic, because Minka expressly teaches that doing so allows different inference algorithms to be used and even changed without requiring the model to be re-written, thereby reducing the time and engineering effort needed to implement, modify, and reuse search strategies, which yields the predictable benefit of more efficiently exploring and optimizing inference-time search strategies for the agentic workflow as set forth below. The same rationale supporting the combination of Nagaraja, Kartal, and Eilon set forth above regarding claim 1 applies to claim 20.
Claim(s) 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nagaraja (US 2020/0244605 A1) in view of Kartal (US 2020/0143206 A1), Eilon (US 12,197,560 B1), and Minka (US 8,103,598 B2), further in view of Chen (US 2024/0020096 A1).
Regarding Claim 17, Nagaraja, Kartal, Eilon, and Minka teach
The one or more non-transitory computer-readable media of claim 11.
Nagaraja, Kartal, Eilon, and Minka did not specifically teach
wherein each execution branch included in the first plurality of branches comprises an invocation of a language model.
However, Chen (US 2024/0020096 A1) teaches
wherein each execution branch included in the first plurality of branches comprises an invocation of a language model (Chen, Abstract: “a method may comprise one or more of: receiving a docstring representing natural language text specifying a digital programming result; generating, using a trained machine learning model, and based on the docstring, a computer code sample configured to produce respective candidate results; causing the computer code sample to be executed; identifying, based on the executing, a computer code sample configured to produce a particular candidate result associated with the digital programming result.”) Examiner Comments: The reference teaches the limitation because it invokes a trained machine learning language model in each generation step, which corresponds to a branch, to produce a computer code sample.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Nagaraja, Kartal, Eilon, and Minka's teaching with Chen in order to incorporate language model invocations in workflow branches, as Chen teaches using trained language models for code generation and execution verification, which would enhance the AI agents in the combined system with advanced reasoning and generation capabilities (Chen, Paragraph [0053]: “Consistent with some disclosed embodiments, the trained machine learning model may be developed by applying training data comprising annotated computer code to a precursor model comprising a machine learning model trained on natural language prompts.”).
Response to Arguments
Applicant’s arguments with respect to claims 1-20 have been considered but are moot because the arguments do not apply to the previous cited sections of the references used in the previous office action. The current office action is now citing additional paragraphs to address the newly added claimed limitations.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AMIR SOLTANZADEH whose telephone number is (571)272-3451. The examiner can normally be reached M-F, 9am - 5pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Wei Mui can be reached at (571) 272-3708. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AMIR SOLTANZADEH/Examiner, Art Unit 2191 /WEI Y MUI/Supervisory Patent Examiner, Art Unit 2191
Prosecution Insights