Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 10/27/2023 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, 5-8, 10, and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Bo Liu (LLM+P: Empowering Large Language Models with Optimal Planning Proficiency) in view of Terry Yue Zhuo (On Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model: An Empirical Study on Codex) and further in view of Shunyu Yao (ReAct: Synergizing Reasoning And Acting In Language Models).
Regarding claim 1, Liu in view of Zhuo and further in view of Yao-ReAct, teach a method of prompt processing, comprising:
“generating ” – Liu teaches this limitation in part. Specifically, Liu teaches receiving a natural-language description of a planning problem using an LLM, where the description specifies an initial state and goal conditions:
“LLM+P takes in a natural language description of a planning problem, then returns a correct (or optimal) plan for solving that problem in natural language.” (Liu, § Abstract)
“LLM+P does so by first converting the language description into a file written in the planning domain definition language (PDDL), then leveraging classical planners to quickly find a solution, and then translating the found solution back into natural language.” (Liu, § Abstract)
“The problem PDDL file provides a list of objects to ground the domain, the problem’s initial state
S
i
n
i
t
and goal conditions
S
G
” (Liu, § II. Background)
“In this work, we aim to take a natural language prompt which describes the initial state
S
i
n
i
t
and goal conditions
S
G
formulate it in PDDL,” (Liu, § II. Background)
Thus, Liu teaches an LLM receiving and processing a prompt that requests transformation from an initial state to a desired goal state, but does not expressly teach the separately recited semantic-feature representation.
“generating a task description in planning domain definition language (PDDL) using ” – Liu teaches this limitation in part. Specifically, Liu teaches using an LLM to generate a PDDL problem file corresponding to the natural-language planning problem and supplying that problem file, together with a provided PDDL domain file, to a classical planner:
“The PDDL representation of a planning problem
P
is separated into two files: a domain file and a problem file.” (Liu, § II. Background)
“The domain PDDL file provides a lifted representation of the underlying rules of the world. It includes a set of predicates that define the state space
S
and the actions (i.e.,
A
) with their preconditions and effects” (Liu, § II. Background)
“The LLM then uses the in-context learning to infer the problem PDDL file corresponding to
P
. Once the problem PDDL fil is generated, we feed it into any classical planner, together with the provided domain PDDL file, to generate a PDDL plan” (Liu, § III. Method)
Thus, Liu teaches generating a PDDL task or problem description using a natural-language prompt and a PDDL domain description, but does not teach using the claimed separate semantic representation as the input to that generation.
“generating an execution plan for the task description in the PDDL using an artificial intelligence (AI) planner, the execution plan comprising a sequence of steps used to transform the initial state to the desired goal state;” – Liu teaches this limitation. Specifically, Liu teaches supplying the generated PDDL problem file and the provided PDDL domain file to a classical symbolic planner to generate a PDDL plan comprising a sequence of symbolic actions that transforms the initial state into a state satisfying the goal conditions:
“Once the problem PDDL fil is generated, we feed it into any classical planner, together with the provided domain PDDL file, to generate a PDDL plan” (Liu, § III. Method)
“A solution to a planning problem
P
is a symbolic plan
π
in the form
a
1
,
a
2
,
…
,
a
N
, such that the preconditions of
a
1
hold in
S
i
n
i
t
, the preconditions of
a
2
hold in the state that results from applying
a
1
, and so on, with the goal conditions all holding in the state that results after applying
a
N
.” (Liu, § II. Background)
Thus, Liu’s classical planner constitutes the recited AI planner and generates a sequence of steps that transforms the modeled initial state into a state satisfying the desired goal conditions.
“and generating a natural language response to the prompt state.” – Liu teaches this limitation in part. Liu teaches using the LLM to translate the planner-generated PDDL plan into natural language, thereby producing a natural-language response associated with the plan for achieving the specified goal:
“LLM+P makes use of a large language model (LLM) to produce the PDDL description of the given problem, then leverages a classical planner for finding an optimal plan, then translates the raw plan back to natural language using the LLM again.” (Liu, Fig. 1)
“In the end, the LLM translates the PDDL plan back into the natural language to finish up the LLM+P pipeline.” (Liu, § III. Method)
Thus, Liu teaches generating a natural-language response corresponding to the plan for achieving the desired goal state but does not expressly teach generating the response after execution has been completed.
Liu does not teach these limitations and/or portions of:
“… a representation of … representation comprising semantic features of the prompt,”
“… representation of the prompt …”
“executing the sequence of steps of the execution plan;”
“… after completing the execution of the sequence of steps, …”
Zhuo, however, teaches these limitations and/or portions of:
“… a representation of … representation comprising semantic features of the prompt,” and “… representation of the prompt …” – Zhou defines semantic parsing as constructing a structured representation of the meaning of a natural-language question and teaches prompt-based semantic parsers implemented using large pretrained language models, including Codex and GPT-J:
“Semantic parsing is a technique aimed at constructing a structured representation of the meaning of a natural-language question.” (Zhou, § Abstract)
“Recent advances in language models trained on code have shown superior performance in generating these representations compared to language models trained solely on natural language text.” (Zhou, § Abstract)
“Semantic parsing is a technique that transforms natural-language utterances (NLs) into machine readable logical forms (LFs)” (Zhou, § 1 Introduction)
“prompt-based semantic parsers based on large pre-trained language models, such as Codex (Chen et al., 2021) and GPT-J (Wang and Komatsuzaki, 2021), have become a new choice for semantic parsing applications.” (Zhou, § 1 Introduction)
Thus, Zhou teaches using a large language model to generate a machine-readable structured representation that captures the semantic meaning or semantic features of a natural-language prompt.
Neither Zhuo nor Liu teach these remaining limitations and/or portions of:
“executing the sequence of steps of the execution plan;”
“… after completing the execution of the sequence of steps, …”
Yao-ReAct, however, teaches these remaining limitations and/or portions of:
“executing the sequence of steps of the execution plan;” – Yao-ReAct teaches an LLM generating and performing task-specific actions that interface with external knowledge bases or environments, with the actions forming part of the executed task-solving trajectory:
“we explore the use of LLMs to generate both reasoning traces and task-specific actions in an interleaved manner” (Yao-ReAct, pg. 1, § Abstract)
“actions allow it to interface with and gather additional information from external sources such as knowledge bases or environments.” (Yao-ReAct, pg. 1, § Abstract)
“ReAct prompts LLMs to generate both verbal reasoning traces and actions pertaining to a task in an in manner, which allows the model to perform dynamic reasoning to create, maintain, and adjust high-level plans for acting (reason to act), while also interact with the external environments” (Yao-ReAct, pg. 2, § 1 Introduction)
Thus, Yao-ReAct teaches actually carrying out task-specific action steps through interaction with an external environment rather than merely generating or displaying a plan.
“… after completing the execution of the sequence of steps, …” – Yao-ReAct teaches a sequence of thought, action, and observation steps followed by a terminal finish[answer] action that completes the task with an answer based on the information gathered through execution:
“We design a simple Wikipedia web API with three types of actions to support interactive information retrieval: (1) search[entity] … (2) lookup[string] … (3) finish[answer], which would finish the current task with answer.” (Yao-ReAct, pg. 4, § 3.1 Setup)
“Specifically, we use a combination of thoughts that decompose questions (“I need to search x, find y, then find z”), extract information from Wikipedia observations … guide search reformulation … and synthesize the final answer (“...so the answer is x”).” (Yao-ReAct, pg. 4-5, § 3.2 Methods)
Thus, Yao-ReAct teaches generating a final natural-language answer after carrying out an action-observation sequence, where the answer corresponds to completion of the task requested by the input prompt.
It would have been obvious to one of ordinary skill in the art to implement Liu’s expressly contemplated action executors using Yao-ReAct’s known action-execution and final-answer arrangement. Liu already suggests connecting the generated PDDL plan to robot action executors. Yao-ReAct teaches a predictable implementation in which task-specific actions interact with an external environment, resulting observations are obtained, and a final answer is generated upon completion of the task. The combination would have enabled Liu’s planner-generated sequence to be carried out and would have provided the user with a response grounded in the completed execution rather than merely a textual restatement of the proposed plan.
Accordingly, it would have been obvious to modify Liu to: (1) generate an intermediate structured semantic representation of the prompt and use the representation to generate the PDDL task description, as taught by Zhuo; and (2) execute the planner-generated action sequence and generate a final natural-language response after completion of the action sequence, as taught by Yao-ReAct. The combination would have predictably improved the accuracy of natural-language-to-PDDL conversion and provided a grounded response reelecting completion of the requested state-changing task.
Regarding claim 2, Liu in view of Zhuo and further in view of Yao-ReAct, teach the method of Claim 1, further comprising, during the execution of the sequence of steps:
“determining execution of at least one step of the sequence of steps has failed;” – Liu does not teach this limitation. Yao-ReAct, however, teaches this limitation. Yao-ReAct teaches an agent that executes an action in an external environment and receives an observation indicating the result of the action. The agent uses the observation to determine that the attempted action was unsuccessful:
“At time step
t
, an agent receives an observation
o
t
∈
O
from the environment and takes an action
a
t
∈
A
following some policy
π
(
a
t
|
c
t
)
, where
c
t
=
(
o
1
,
a
1
,
…
,
o
t
-
1
,
a
t
-
1
,
o
t
)
is the context to the agent.” (Yao-ReAct, § 2 ReAct: Synergizing Reasoning + Acting)
Yao-ReAct provides an example in which an executed search action fails to locate the requested subject:
“
PNG
media_image1.png
86
526
media_image1.png
Greyscale
” (Yao-ReAct, pg. 2, Figure 1)
Thus, Yao-ReAct teaches determining from environment feedback received during execution that an attempted step did not successfully produce the intended result.
“adjusting the execution plan to create an adjusted execution plan;” – Liu does not teach this limitation. Yao-ReAct, however, teaches this limitation. Yao-ReAct teaches using reasoning traces and observations to track, update, and adjust action plans, including handling exceptions encountered during execution:
“reasoning traces help the model induce, track, and update action plans as well as handle exceptions” (Yao-ReAct, § Abstract)
“ReAct prompts LLMs to generate both verbal reasoning traces and actions pertaining to a task in an in manner, which allows the model to perform dynamic reasoning to create, maintain, and adjust high-level plans for acting (reason to act), while also interact with the external environments” (Yao-ReAct, pg. 2, § 1 Introduction)
“there could be various types of useful thoughts, e.g. decomposing task goals and create action plans … extracting important parts from observations … track progress and transit action plans … handle exceptions and adjust action plans” (Yao-ReAct, pg. 3, § 2 ReAct: Synergizing Reasoning + Acting)
In response to the failed search for “Front Row”, Yao-ReAct expressly teaches revising the contemplated search action:
“Thought 3: Front Row is not found. I need to search Front Row (software).” (Yao-ReAct, pg. 2, Figure 1)
Thus, Yao-ReAct teaches adjusting the plan based on the observed failure by replacing or supplementing the failed search step with a revised search step directed to “Front Row (software)”.
“and re-executing the adjusted execution plan.” – Liu does not teach this limitation. Yao-ReAct, however, teaches this limitation. Specifically, after determining that the original search step failed and adjusting the plan to use a revised search query, Yao-ReAct executes the revised action and receives a corresponding observation from the external environment:
“Thought 3: Front Row is not found. I need to search Front Row (software).” (Yao-ReAct, pg. 2, Figure 1)
“Obs 3: Front Row is a discontinued media center software …” (Yao-ReAct, pg. 2, Figure 1)
Yao-ReAct further explains that its task-solving trajectories include repeated thought, action, and observation steps:
“Each in-context example is a human trajectory of actions, thoughts, and environment observations to solve a task instance” (Yao-ReAct, pg. 3, § 2 ReAct: Synergizing Reasoning + Acting)
“For the tasks where reasoning is of primary importance (Figure 1(1)), we alternate the generation of thoughts and actions so that the task-solving trajectory consists of multiple thought-action-observation steps.” (Yao-ReAct, pg. 3, § 2 ReAct: Synergizing Reasoning + Acting)
Thus, after detecting the failure and adjusting the plan, Yao-ReAct teaches carrying out the adjusted action plan based on the observed failure, and executing the adjusted action plan. As set forth with respect to claim 1, Yao-ReAct is combined with Liu to implement Liu’s planner-generated sequence though an interactive action-execution arrangement that receives environmental feedback and dynamically responds to execution results.
Regarding claim 5, Liu in view of Zhuo and further in view of Yao-ReAct teach the method of Claim 1, further comprising, prior to executing the sequence of steps:
“determining the generation of the execution plan for the task description in the PDDL has failed due to one or more errors associated with the task description in the PDDL;” – Liu teaches this limitation. Specifically, Liu teaches supplying an LLM-generated PDDL problem file to a classical planner to generate a PDDL plan. Liu further teaches that errors in the generated PDDL problem file may make the planning problem unsolvable, thereby causing generation of the execution plan to fail:
“Once the problem PDDL fil is generated, we feed it into any classical planner, together with the provided domain PDDL file, to generate a PDDL plan” (Liu, § III. Method)
Liu provides an example in which the LLM-generated task description includes an undefined predicate and omits a required initial condition:
“As we see, the generated file appears to have the correct PDDL syntax but uses a made-up predicate (empty) and misses the initial condition that b1 is on the table.” (Liu, § III. Method)
Liu further explains that the failures of the LLM-plus-planner pipeline result from errors in the generated PDDL problem file:
“Most failed cases are due to mis-specified problem files, such as missing one of the initial conditions (e.g. leaving the tiles disconnected in FLOORTILE), causing the planning problem to be unsolvable.” (Liu, § V. Experiments)
Thus, Liu teaches determining that generation of the execution plan has failed because one or more errors in the PDDL task description render the planning problem unsolvable.
“and generating a new task description in the PDDL to fix the one or more errors.” – Liu teaches this limitation. Liu teaches that after identifying that an initially generated PDDL problem file contains an undefined predicate and omits a required initial condition, Liu supplies an example natural-language/PDDL pair as context and causes the LLM to generate a new PDDL problem file that corrects the errors and is directly solvable by the planner:
“Here is a pair of problem descriptions in natural language and in PDDL provided to GPT-4 as context.” (Liu, § III. Method)
“When the context is included with the prompt from the example above, the resulting PDDL problem file is directly solvable by the planner.” (Liu, § III. Method)
Liu then shows the newly generated PDDL problem file and the execution successfully returned by the planner:
“
PNG
media_image2.png
381
417
media_image2.png
Greyscale
” (Liu, § III. Method, An Example PDDL Problem File Written by GPT-4 with Context)
The newly generated PDDL problem file replaces the made-up predictable “empty” with the valid predicate “arm-empty” and includes the omitted initial condition “on-table b1”. The corrected PDDL task description is therefore accepted by the planner and results in generation of a valid execution plan.
Thus, Liu teaches generating a new task description in PDDL that fixes errors in an earlier task description and enables the planner to generate the execution plan.
Although Liu presents the corrected PDDL generation as a comparison using an in-context example, Liu expressly identifies mis-specified PDDL files as causing planner failure and demonstrates that contextual prompting produces a corrected, planner-solvable PDDL file. It therefore would have been obvious to regenerate the PDDL task description using Liu’s disclosed contextual procedure when planner failure indicates that the original description is invalid or incomplete, thereby correcting the errors and enabling generation of the execution plan.
Regarding claim 6, Liu in view of Zhuo and further in view of Yao-ReAct teach the method of Claim 1, wherein:
“the sequence of steps comprises at least one step for making an application programming interface (API) call to an application, and executing the sequence of steps of the execution plan comprises ” – Liu teaches this limitation in part. Specifically, Liu teaches generating an execution plan comprising a sequence of actions and connecting the planner output to action executors:
“A solution to a planning problem
P
is a symbolic plan
π
in the form
a
1
,
a
2
,
…
,
a
N
…” (Liu, § II. Background)
“converts the output of the planner back to natural language (or connects to action executors of a robot).” (Liu, § 1. Introduction)
Thus, Liu teaches a planner-generated sequence applied to action executors, but does not expressly teach making an API call to an application.
Liu does not teach these limitations and/or portions of:
“… translating information included in the execution plan to one or more input parameters understood by the API.”
Yao-ReAct, however, teaches these remaining limitations and/or portions of:
“… translating information included in the execution plan to one or more input parameters understood by the API.” – Yao-ReAct defines the API-call actions using parameterized formats in which information selected during planning or reasoning is inserted into an API-understood input field:
“We design a simple Wikipedia web API with three types of actions to support interactive information retrieval: (1) search[entity] … (2) lookup[string] … (3) finish[answer], which would finish the current task with answer.” (Yao-ReAct, pg. 4, § 3.1 Setup)
“Thought 3: Front Row is not found. I need to search Front Row (software).” (Yao-ReAct, pg. 2, Figure 1)
“Act 3: Search[Front Row (software)]” (Yao-ReAct, pg. 2, Figure 1)
Thus, the information “Front Row (software)” identified in the adjusted action plan is formatted as the entity input parameter understood by the API and used to execute the search[entity] API call.
It would have been obvious to one of ordinary skill in the art to implement Liu’s planner-generated actions using Yao-ReAct’s parameterized API-call arrangement. Liu expressly contemplates connecting the generated plan to action executors, while Yao-ReAct teaches executing planned actions by formatting action information as parameters of defined API operations. Applying Yao-ReAct’s arrangement would have provided a predictable mechanism for carrying out Liu’s symbolic plan through external applications and tools.
Regarding claim 7, Liu in view of Zhuo and further in view of Yao-ReAct teach the method of Claim 1, wherein the sequence of steps comprise steps for at least one of:
“making one or more application programming interface (API) calls to one or more applications; initiating one or more database queries for one or more databases; or triggering one or more functions.” – Liu does not teach this limitation. Yao-ReAct, however, teaches this limitation. Yao-ReAct teaches an agent executing actions through a Wikipedia web API to interact with an external information source:
“actions allow it to interface with and gather additional information from external sources such as knowledge bases or environments.” (Yao-ReAct, pg. 1, § Abstract)
“We design a simple Wikipedia web API with three types of actions to support interactive information retrieval: (1) search[entity], which returns the first 5 sentences from the corresponding wiki page if it exists, or else suggests top-5 similar entities from the entity Wikipedia search engine, (2) lookup[string], which would return the next sentence in the page containing string, simulating Ctrl+F functionality on the browser. (3) finish[answer], which would finish the current task with answer.” (Yao-ReAct, pg. 4, § 3.1 Setup)
Yao-ReAct further provides an execution trajectory containing multiple API-call actions:
“Action 1 Search[Colorado orogeny] … Action 2 Lookup[eastern sector] … Action 3 Search[High Plains] … Action 4 Search[High Plains (United States)]” (Yao-ReAct, pg. 16, § C Prompts)
Thus, Yao-ReAct teaches a sequence of steps comprising multiple calls to a web API associated with an external application or knowledge source.
Because claim 7 requires steps for “at least one of” the listed alternatives, Yao-ReAct’s teaching of making API calls is sufficient; the prior art need not additionally teach database queries or triggered functions.
It would have been obvious to implement Liu’s planner-generated execution sequence using Yao-ReAct’s API-based action arrangement. Liu expressly contemplates connecting the planner output to action executors, while Yao-ReAct teaches a known mechanism for carrying out planned actions through parameterized API calls to external applications and information sources. The modification would have predictably enabled Liu’s generated plan to interact with external systems and obtain information needed to complete the requested task.
Regarding claim 8, Liu in view of Zhuo and further in view of Yao-ReAct teach the method of Claim 1, wherein
“generating the execution plan for the task description in the PDDL comprises making one or more conclusions based on data included in the task description in the PDDL,” – Liu teaches this limitation. Liu teaches that the PDDL task description defines the initial state and goal conditions, while the PDDL domain defines predicates, actions, preconditions, and effects. The symbolic planner evaluates this information to determine which actions are applicable in each state and which sequence of actions reaches the goal state:
“Formally, the input of a planning problem P is defined by
S
,
S
i
n
i
t
,
S
G
,
A
,
f
” (Liu, § II. Background)
“The domain PDDL file provides a lifted representation of the underlying rules of the world. It includes a set of predicates that define the state space
S
and the actions (i.e.,
A
) with their preconditions and effects (i.e., the transition function f). The problem PDDL file provides a list of objects to ground the domain, the problem’s initial state
s
i
n
i
t
and goal conditions
S
G
.” (Liu, § II. Background)
Thus, in generating the execution plan, Liu’s planner makes conclusions regarding action applicability and resulting states based on the objects, initial-state facts, predicates, and goal conditions including PDDL task description.
“wherein at least one of the sequence of steps included in the execution plan is associated with the one or more conclusions.” – Liu teaches this limitation. Specifically, Liu teaches that each action selected for the plan is associated with the planner’s determination that the action’s preconditions hold in the applicable state and that applying the action produces a resulting state used to select the next action:
“A solution to a planning problem
P
is a symbolic plan
π
in the form
a
1
,
a
2
,
…
,
a
N
, such that the preconditions of
a
1
hold in
S
i
n
i
t
, the preconditions of
a
2
hold in the state that results from applying
a
1
, and so on, with the goal conditions all holding in the state that results after applying
a
N
.” (Liu, § II. Background)
Thus, each selected action step is associated with the planner’s conclusion that its preconditions are satisfied in the relevant state and that its effects advance the state toward satisfaction of the goal conditions.
Liu teaches generating an execution plan by drawing conclusions from the facts and conditions represented in the PDDL task description and including action steps associated with those conclusions.
Regarding claim 10, Liu in view of Zhuo and further in view of Yao-ReAct teach the method of Claim 1, wherein:
“the desired goal state requested by the prompt comprises possession of information associated with at least one of a user, a product, a client, or an organization,” – Liu does not teach this limitation. Yao-ReAct, however, teaches this limitation. Yao-ReAct teaches question-answering tasks in which the user’s prompt requests information about an identified person, entity, or organization, and completion of the task requires obtaining that information:
“Hot-PotQA (Yang et al., 2018), a multi-hop question answering benchmark that requires reasoning over two or more Wikipedia passages,” (Yao-ReAct, pg. 4, § 3.1 Setup)
Yao-ReAct provides exemplary prompts requesting information associated with identified entities:
“What profession does Nicholas Ray and Elia Kazan have in common?” (Yao-ReAct, pg. 16, § C.1 HotpotQA)
Thus, the requested goal is possession of information associated with identified persons or entities.
“the sequence of steps included in the execution plan comprises at least a call to a database comprising the information associated with at least one of the user, the product, the client, or the organization,” – Yao-ReAct teaches executing calls to the Wikipedia knowledge base through a web API using search[entity] and lookup[string] actions:
“actions allow it to interface with and gather additional information from external sources such as knowledge bases or environments.” (Yao-ReAct, pg. 1, § Abstract)
“We design a simple Wikipedia web API with three types of actions to support interactive information retrieval: (1) search[entity], which returns the first 5 sentences from the corresponding wiki page if it exists, or else suggests top-5 similar entities from the entity Wikipedia search engine, (2) lookup[string], which would return the next sentence in the page containing string, simulating Ctrl+F functionality on the browser.” (Yao-ReAct, pg. 4, § 3.1 Setup)
Yao-ReAct further provides an execution trajectory containing multiple API-call actions:
“Action 1 Search[Colorado orogeny] … Action 2 Lookup[eastern sector] … Action 3 Search[High Plains] … Action 4 Search[High Plains (United States)]” (Yao-ReAct, pg. 16, § C Prompts)
Thus, Yao-ReAct teaches a sequence of execution steps containing calls to a database or knowledge base that stores information associated with identified persons, products, clients, organizations, or other entities.
“and the natural language response comprises the information associated with at least one of the user, the product, the client, or the organization.” – Yao-ReAct teaches extracting information from observations returned by the knowledge base and synthesizing that information into the final answer:
“Specifically, we use a combination of thoughts that decompose questions (“I need to search x, find y, then find z”), extract information from Wikipedia observations … guide search reformulation … and synthesize the final answer (“...so the answer is x”).” (Yao-ReAct, pg. 4-5, § 3.2 Methods)
“finish[answer], which would finish the current task with answer.” (Yao-ReAct, pg. 4, § 3.1 Setup)
Thus, Yao-ReAct teaches generating a natural-language response containing the information retrieved from the database concerning the entity identified in the prompt.
It would have been obvious to implement Liu’s planner-generated sequence using Yao-ReAct’s knowledge-base retrieval actions when the requested goal state requires obtaining information about a particular person, product, client, organization, or other entity. Liu teaches generating a plan to achieve a prompt-defined goal state, while Yao-ReAct teaches executing database calls to retrieve requested entity information and incorporating the retrieved information into a final answer. The combination would have predictably enabled Liu’s system to satisfy information-retrieval prompts using external stored data.
Regarding claim 12, Liu in view of Zhuo and further in view of Yao-ReAct teach the method of Claim 1, wherein
“the desired goal state requested by the prompt comprises a completion of one or more tasks including information retrieval.” – Liu does not teach this limitation. Yao-ReAct, however, teaches this limitation. Specifically, Yao-ReAct applies its reasoning-and-acting system to question-answering tasks that require retrieving information from multiple Wikipedia passages:
“Hot-PotQA (Yang et al., 2018), a multi-hop question answering benchmark that requires reasoning over two or more Wikipedia passages,” (Yao-ReAct, pg. 4, § 3.1 Setup)
Yao-ReAct teaches that the agent completes the requested task by performing information-retrieval actions through a Wikipedia web API:
“We design a simple Wikipedia web API with three types of actions to support interactive information retrieval: (1) search[entity], which returns the first 5 sentences from the corresponding wiki page if it exists, or else suggests top-5 similar entities from the entity Wikipedia search engine, (2) lookup[string], which would return the next sentence in the page containing string, simulating Ctrl+F functionality on the browser. (3) finish[answer], which would finish the current task with answer.” (Yao-ReAct, pg. 4, § 3.1 Setup)
Yao-ReAct further teaches decomposing the requested task into information-retrieval steps and synthesizing the retrieved information into a final answer:
“Specifically, we use a combination of thoughts that decompose questions (“I need to search x, find y, then find z”), extract information from Wikipedia observations … guide search reformulation … and synthesize the final answer (“...so the answer is x”).” (Yao-ReAct, pg. 4-5, § 3.2 Methods)
Thus, Yao-ReAct teaches a prompt-defined task whose completion includes retrieving information from an external knowledge base and using that retrieved information to complete the task.
It would have been obvious to use Yao-ReAct’s information-retrieval actions in Liu’s planner-generated execution sequence when the prompt-defined goal requires obtaining information. Liu teaches generating a sequence of actions to achieve a requested goal state, while Yao-ReAct teaches executing search and lookup actions against an external knowledge base to retrieve information and complete the requested task. The combination would have predictably enabled Liu’s system to process prompts whose requested tasks include information retrieval.
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Liu in view of Zhou in further view of Yao-ReAct and further in view of Shreyas Sundara Raman (Planning with Large Language Models via Corrective Re-prompting).
Regarding claim 3, Liu in view of Zhuo further in view of Yao-ReAct and further in view of Raman, teach the method of Claim 2, wherein determining the execution of the at least one step of the sequence of steps has failed comprises:
“identifying an outage or an error when invoking an application programming interface (API) associated with the at least one step;” – Liu does not teach this limitation. Raman, however, teaches this limitation. Raman teaches that actions are executable only in contexts in which their preconditions are satisfied and that an embodied agent determines whether an action is executable in the current context by detecting a precondition error:
“Our approach assumes that actions are only afforded execution in certain contexts, i.e., implicit preconditions must be met for an action to execute (e.g., a door must be unlocked to open it), and that the embodied agent has the ability to determine if the action is/is not executable in the current context (e.g., detect if a precondition error is present).” (Raman, pg. 1, § Abstract)
Raman further teaches that the agent determines whether an action is currently afforded and may identify the particular precondition that renders the action infeasible:
“Preconditions come from the notion that actions are not executable in all circumstances, and that certain conditions must first be satisfied.” (Raman, pg. 2, § 1 Introduction)
“In certain scenarios, these conditions are factorized and explicit to the agent (e.g., to follow a cooking recipe, the agent must access the requisite ingredients), while in other scenarios, there may be no easy or accessible way to explicitly decompose the pre-conditions, and so the agent only has the ability to determine whether an action is currently afforded or not” (Raman, pg. 2, § 1 Introduction)
“In the latter case, the agent only has the ability to report a precondition error stating that an action is currently infeasible, whereas in the former case the precondition error can contain more information about why that action is currently not feasible.” (Raman, pg. 2, § 1 Introduction)
Thus, Raman teaches evaluating the conditions associated with an action against the current runtime context and determining that the action is not executable because one or more required conditions are not satisfied.
Raman expressly applies this determination during execution of a generated sequence of action steps:
“Our approach takes a natural language description of a task and uses a LLM to produce a sequence of actions for the robot to execute.” (Raman, pg. 2, § 1 Introduction)
“When the robot fails to execute a skill in the plan and reports a precondition error, we use a template-based prompt strategy to query new actions from the LLM to produce new plans for the embodied agent.” (Raman, pg. 2, § 1 Introduction)
Raman further explains that its approach covers systems having explicit knowledge of the particular unmet preconditions:
“Our approach is adaptable to situations where agents are only able to evaluate whether an action is currently afforded or not, and to more structured settings where explicit knowledge about what preconditions are currently unmet are known.” (Raman, pg. 2-3, § 1 Introduction)
Raman also teaches expressly evaluating which preconditions of the failed action are not currently satisfied:
“leveraging cause of error assumes the agent has a model of the preconditions for the action and the ability to explicitly evaluate what preconditions are not currently satisfied” (Raman, pg. 6, § 4.1 Plan Generation via Re-prompting)
Raman’s VirtualHome implementation generates execution errors when an action is attempted while its required preconditions are unsatisfied:
“When errors occur during program execution, the VirtualHome environment throws descriptive messages containing the step information above along with the cause of the error.” (Raman, pg. 7, § 5.1 Environment)
“The agent attempts to execute an action whose pre-conditions are not satisfied” (Raman, pg. 13, § A.1 Error Types in VirtualHome)
Thus, Raman teaches determining during execution of a particular step that the current runtime state does not satisfy one or more preconditions required for execution of that step.
Because claim 3 recites the API-error condition, the unmet-runtime-precondition condition, and the database-query-error condition as alternatives, Raman’s teaching of the unmet-runtime-precondition alternatives satisfies the claimed limitation.
It would have been obvious to one of ordinary skill in the art to modify the interactive plan-execution of Liu and Yao-ReAct to evaluate whether the current runtime context satisfies the preconditions associated with each planned action, as taught by Raman.
Liu already teaches that actions in a PDDL plan are defined by preconditions and effects and that a valid plan requires the preconditions of each successive action to hold in the state produced by proceeding actions. Yao-ReAct teaches executing action steps against an external environment, receiving observations regarding the results of those steps, and adjusting the plan in response to execution failures. Raman teaches the predictable execution-monitoring technique of evaluating the current runtime context to determine whether the preconditions required for a proposed action are satisfied.
A person of ordinary skill in the art would have been motivated to apply Raman’s runtime precondition evaluation to the plan-execution arrangement of Liu and Yao-ReAct to identify planned actions that cannot be executed in the current environment, provide a specific cause of the execution failure, and facilitate the corrective replanning and re-execution taught by Yao-ReAct. The modification would have used a known execution-monitoring technique according to its established function and would have predictably improved the executability and reliability of the generated action plans.
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Liu in view of Zhou in further view of Yao-ReAct and further in view of Yao-MISP (Model-based Interactive Semantic Parsing: A Unified Framework and A Text-to-SQL Case Study).
Regarding claim 4, Liu in view of Zhou and further in view of Yao-ReAct and further in view of Yao-MISP teach the method of Claim 1, further comprising, prior to generating the execution plan:
“determining the generation of the task description in the PDDL has failed;” – Liu teaches this limitation. Liu teaches that an LLM-generated PDDL task description may contain errors or omissions that render the planning problem unsolvable and prevent generation of an execution plan:
“As we see, the generated file appears to have the correct PDDL syntax but uses a made-up predicate (empty) and misses the initial condition that b1 is on the table.” (Liu, § III. Method)
“Most failed cases are due to mis-specified problem files, such as missing one of the initial conditions … causing the planning problem to be unsolvable.” (Liu, § V. Experiments)
Thus, Liu teaches determining that generation of a usable PDDL task description has failed because errors in the generated task description prevent the planner from generating an execution plan.
Liu does not teach these limitations and/or portions of:
“and requesting, from a user that submitted the prompt, additional information for the prompt.”
Yao-MISP, however, teaches detecting an error
“and requesting, from a user that submitted the prompt, additional information for the prompt.” – Yao-MISP teaches receiving an initial natural-language utterance from a user and requesting clarification from that same user when the generated semantic parse may contain an error:
“Taking an utterance (e.g., a natural language question) as input, the agent forms the semantic parse (e.g., a SQL query) in steps, potentially soliciting user feedback in some steps to correct parsing errors.” (Yao-MISP, § 1 Introduction)
Yao-MISP teaches detecting whether and where the generated formal representation is erroneous and generating a question requesting additional information from the user:
“a MISP agent maintains its state as the current semantic parse and, via an error detector, decides whether and where human intervention is needed (the action).” (Yao-MISP, § Introduction)
“This action is performed by a question generator (the actuator), which generates and presents to the user a human understandable question.” (Yao-MISP, § Introduction)
Yao-MISP further teaches that the user who provided the original utterance responds to the classification request:
“Based on this intent, the user gives an initial natural language utterance
u
0
to start a semantic parsing session and responds to any clarification question from the agent with feedback
u
t
at interaction turn
t
.” (Yao-MISP, § 3 Model-based Interactive Semantic Parsing)
The error detector identifies a portion of the generated formal representation that is likely erroneous and provides that information to the component that formulates the clarification request:
“The error detector serves this role. Given the current state
s
t
… it decides on an action
a
t
… otherwise, it determines a span in the current semantic parse that is likely erroneous and passes it, along with necessary context information needed to make sense of the error span, to the actuator.” (Yao-MISP, § 3 Model-based Interactive Semantic Parsing)
Yao-MISP then uses the additional information supplied by the user to update the formal semantic representation:
“A core component of the agent is a world model … which incorporates user feedback from the environment and transitions to a new agent state (e.g., an updated semantic parse).” (Yao-MISP, § 1 Introduction)
Thus, Yao-MISP teaches requesting additional information from the user who submitted the natural-language utterance after determining that the generated formal representation likely contains an error.
It would have been obvious to apply Yao-MISP’s interactive clarification procedure to Liu’s natural-language-to-PDDL generation process. Liu teaches that errors or omissions in a generated PDDL task description may prevent plan generation, while Yao-MISP teaches detecting errors in a generated formal representation, requesting clarification from the originating user, and using the feedback to update the representation. Applying Yao-MISP’s procedure would have predictably enabled Liu’s system to obtain information needed to correct or complete a failed PDDL task description before generating the execution plan.
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Liu in view of Zhou in further view of Yao-ReAct and further in view of Zhaoheng Huang (MCP: Self-supervised Pre-training for Personalized Chatbots with Multi-level Contrastive Sampling)
Regarding claim 9, Liu in view of Zhuo further in view of Yao-ReAct and further in view of Huang teach the method of Claim 1, wherein:
“the natural language response is generated by the LLM,” – Liu teaches this limitation. Liu teaches using the LLM to translate the planner-generated PDDL plan into a natural-language response:
“LLM+P makes use of a large language model (LLM) to produce the PDDL description of the given problem, then leverages a classical planner for finding an optimal plan, then translates the raw plan back to natural language using the LLM again.” (Liu, Fig. 1)
“In the end, the LLM translates the PDDL plan back into the natural language to finish up the LLM+P pipeline.” (Liu, § III. Method)
Thus, Liu teaches generating the natural-language response using the LLM.
Liu does not teach these limitations and/or portions of:
“and the LLM is trained to generate the natural language response as a personalized response for a user associated with the prompt.”
Huang, however, teaches these remaining limitations and/or portions of:
“and the LLM is trained to generate the natural language response as a personalized response for a user associated with the prompt.” – Huang teaches that personalized chatbots use personal information associated with a user to generate user-specific responses:
“When equipped with personal information (either given by a predefine profile or learning from dialogue history), personalized chatbots can generate more user-specific and informative responses.” (Liu, § 1. Introduction)
Huang defines its task as generating a personalized response to a new query for a particular user using personal information extracted form that user’s dialogue history:
“With the above notations, our task is defined as generating a personalized response
r
for the user
u
i
to reply to a new query
q
with the personalized information extracted from the user’s dialogue history
H
i
.” (Huang, § 3.1 Problem Statement and Overview)
Huang further teaches providing the user-specific history representation and current query to a Transformer-based language-generation model:
“The obtained history representation (containing personal information) and the input query are fed into a Transformer-based Seq2Seq structure to generate a personalized response.” (Huang, § 3.1 Problem Statement and Overview)
Huang also teaches using the user profile to control personalized response generation:
“The user profiles are fed into the encoder decoder model to drive the personalized response generation process.” (Huang, § 1. Introduction)
Huang expressly teaches training the personalized chatbot model:
“We propose a two-stage framework for applying self-supervised learning to train personalized chatbots for better data representation.” (Huang, § 1. Introduction)
“At the fine-tuning stage, the model is trained to maximize the generation probability of the ground-truth response
y
.” (Huang, § 3.6 Training and Optimizing)
Thus, Huang teaches training a Transformer-based language-generation model to generate a personalized response to a current prompt for the particular user associated with that prompt.
It would have been obvious to modify Liu’s LLM-based natural-language response generator according to Huang’s personalization training technique. Liu already uses an LLM to generate the natural-language response, while Huang teaches training a Transformer-based response generator to use a current query and user-specific profile information to produce a personalized response for that user. Applying Huang’s technique to Liu’s LLM would have predictably tailored the generated response to the user associated with the prompt.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Liu in view of Zhou in further view of Yao-ReAct and further in view of Timo Schick (Toolformer: Language Models Can Teach Themselves to Use Tools)
Regarding claim 11, Liu in view of Zhuo further in view of Yao-ReAct and further in view of Schick teach the method of Claim 1, wherein:
“the desired goal state requested by the prompt comprises a completion of one or more tasks,” – Liu teaches this limitation. Specifically, Liu teaches that the prompt defines goal conditions and that the planner generates a sequence of actions that transforms the initial state into a state in which the goal conditions are satisfied:
“In this work, we aim to take a natural language prompt which describes the initial state
S
i
n
i
t
and goal conditions
S
G
formulate it in PDDL,” (Liu, § II. Background)
“A solution to a planning problem
P
is a symbolic plan
π
in the form
a
1
,
a
2
,
…
,
a
N
, … with the goal conditions all holding in the state that results after applying
a
N
.” (Liu, § II. Background)
Thus, Liu teaches a desired goal state representing completion of the task specified by the prompt.
“and the sequence of steps included in the execution plan to transform the initial state to the desired goal state comprises at least one step for: ” – Liu teaches this limitation in part. Liu teaches an execution plan comprising a sequence of steps that transforms the initial state to the desired goal state:
“Once the problem PDDL fil is generated, we feed it into any classical planner, together with the provided domain PDDL file, to generate a PDDL plan” (Liu, § III. Method)
“A solution to a planning problem
P
is a symbolic plan
π
in the form
a
1
,
a
2
,
…
,
a
N
, such that the preconditions of
a
1
hold in
S
i
n
i
t
, the preconditions of
a
2
hold in the state that results from applying
a
1
, and so on, with the goal conditions all holding in the state that results after applying
a
N
.” (Liu, § II. Background)
Thus, Liu teaches generating a sequence of steps that transforms the modeled initial state into a state satisfying the desired goal conditions.
Liu does not teach these limitations and/or portions:
“the one or more tasks excluding information retrieval,”
“generating an invoice, sending an invoice, sending an email, updating information in a database, submitting a payment, accepting a payment, calculating an average, calculating a maximum, calculating a minimum, or setting a reminder.”
Schick, however, teaches these remaining limitations and/or portions of:
“the one or more tasks excluding information retrieval,” – Schick distinguishes calculator operations from retrieval tools and teaches using a calculator to perform numeric computations rather than retrieve stored information:
“We incorporate a range of tools, including a calculator, a Q&A system, a search engine, a translation system, and a calendar.” (Schick, § Abstract)
“As a second tool, we use a calculator that can perform simple numeric calculations; we only support the four basic arithmetic operations.” (Schick, § 3 Tools)
Thus, Schick’s calculator performs a computational task rather than an information-retrieval task.
“generating an invoice, sending an invoice, sending an email, updating information in a database, submitting a payment, accepting a payment, calculating an average, calculating a maximum, calculating a minimum, or setting a reminder.” – Schick teaches that the language model determines when to invoke a calculator, supplies the arithmetic expression as an input argument, and executes the calculator call to obtain the result:
“We introduce Toolformer, a model trained to decide which APIs to call, when to call them, what arguments to pass, and how to best incorporate the results into future token prediction.” (Schick, § Abstract)
“As a next step, we execute all API calls generated by
M
to obtain the corresponding results.” (Schick, § 2 Approach)
“As a second tool, we use a calculator that can perform simple numeric calculations; we only support the four basic arithmetic operations.” (Schick, § 3 Tools)
Calculating an average requires adding the relevant values and dividing the sum by the number of values, both of which are basic arithmetic operations supported by Schick’s calculator. Thus, Schick teaches or renders obvious a calculator-execution step for calculating an average.
Because claim 11 requires at least one of the listed operations, the teaching of calculating an average is sufficient.
It would have been obvious to implement Liu’s planner-generated execution sequence using Schick’s calculator when the prompt requests completion of a calculation task. Liu teaches generating a sequence of actions to achieve a prompt-defined goal state, Yao-ReAct teaches executing planned actions through external tools, and Schick teaches selecting, parameterizing, and executing a calculator API. Applying Schick’s calculator would have predictably enabled completion of a non-retrieval task requiring calculation of an average.
Regarding claims 13-20
Claims 13-20 are the apparatus counterparts of method claims 1-8, respectively. Claim 13 recites one or more memories storing processor-executable instructions and one or more processors configured to execute the instructions to perform substantially the same operations recited in claim 1. Claims 14-20 further configure the processors to perform substantially the same additional operations recited in claims 2-8, respectively.
Liu, Zhuo, and Yao-ReAct teach the underlying LLM, semantic-representation, PDDL task-generation, AI-planning, plan-execution, and natural-language-response operations for the reasons discussed above with respect to claim 1. The recited references implement those operations using computer-based language models, planners, external tools, APIs, and execution environments. It would have been obvious to store instructions for performing the disclosed operations in one or more memories and execute the instructions using one or more processors because the disclosed operations are computer-implemented. Expressing the previously established method operations as processor-configured apparatus functions does not, without more, impart a patentable distinction over the prior-art combination.
Accordingly, claim 13 is obvious over Liu in view of Zhuo and Yao-ReAct for the same reasons discussed above with respect to claim 1.
Claim 14 recites processor configurations corresponding to the failure detection, execution-plan adjustment, and re-execution operations of claim 2. Yao-ReAct teaches detecting an unsuccessful action result, dynamically adjusting the action plan based on the observed failure, and executing the adjusted action plan for the reasons discussed above with respect to claim 2.
Claim 15 recites processor configurations corresponding to the alternative failure conditions of claim 3. Raman teaches determining at runtime that one or more preconditions associated with a planned action have not been met for the reasons discussed above with respect to claim 3. Accordingly, claim 15 is obvious over Liu in view of Zhuo further in view of Yao-ReAct and further in view of Raman.
Claim 16 recites processor configurations corresponding to the failed PDDL task-description generation and user-classification operations of claim 4. Liu teaches that an erroneous or incomplete PDDL task description may prevent successful plan generation, and Yao-MISP teaches detecting an error in a formal representation, requesting clarification from the originating user, and updating the representation using the user’s feedback, for the reasons discussed above with respect to claim 4. Accordingly, claim 16 is obvious over Liu in view of Zhuo and Yao-ReAct, and further in view of Yao-MISP.
Claim 17 recites processor configurations corresponding to the planner-failure detection and corrected PDDL task-generation operations of claim 5. Liu teaches that mis-specified PDDL problem files may render a planning problem unsolvable and demonstrates generating a corrected, planner-solvable PDDL problem file using contextual prompting for the reasons discussed above with respect to claim 5. Accordingly, claim 17 is obvious over Liu in view of Zhuo and Yao-ReAct.
Claim 18 recites apparatus limitations corresponding to the API-call and API-parameter translation operations of claim 6. Yao-ReAct teaches parameterized API actions, including supplying information selected during planning as an input parameter understood by the API, for the reasons discussed abo e with respect to claim 6. Accordingly, claim 18 is obvious over Liu in view of Zhuo and further in view of Yao-ReAct.
Claim 19 recites apparatus limitations corresponding to the API-call, database-query, or function-triggering alternatives of claim 7. Yao-ReAct teaches an execution sequence comprising one or more calls to a web API for the reasons discussed above with respect to claim 7. Because claim 19 requires at least one of the listed alternatives, Yao-ReAct’s API-call teaching is sufficient. Accordingly, claim 19 is obvious over Liu in view of Zhuo and Yao-ReAct.
Claim 20 recites processor configurations corresponding to the conclusion-making and associated-plan-step operations of claim 8. Liu teaches a symbolic planner that evaluates PDDL objects, predicates, initial-state facts, action preconditions, action effects, and goal conditions to determine applicable action steps and resulting states for the reasons discussed above with respect to claim 8. Accordingly, claim 20 is obvious over Liu in view of Zhuo and Yao-ReAct.
Therefore,
Claims 13, 14, and 17-20 are rejected under 35 U.S.C. § 103 as being unpatentable over Liu in view of Zhuo and further in view of Yao-ReAct.
Claim 15 is rejected under 35 U.S.C. § 103 as being unpatentable over Liu in view of Zhuo and Yao-ReAct, and further in view of Raman.
Claim 16 is rejected under 35 U.S.C. § 103 as being unpatentable over Liu in view of Zhuo and Yao-ReAct, and further in view of Yao-MISP.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Paul Coleman whose telephone number is (571) 272-4687. The examiner can normally be reached Mon-Fri.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David Yi can be reached at (571) 270-7519. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PAUL COLEMAN/ Examiner, Art Unit 2126
/DAVID YI/ Supervisory Patent Examiner, Art Unit 2126
Prosecution Insights