Detailed Action
This communication is in response to the Application filed on 6/17/2024.
Claims 1-20 are pending and have been examined.
Independent Claims 1, 8 and 15 are parallel method, system and computer program product claims, respectively.
Apparent priority: 6/17/2024.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 9/30/2025 and 6/17/2024 have been considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Independent claim 1 recites,
“1. A computer-implemented method comprising: receiving a task described at least in part with natural language; (This relates to a human receiving a task using the auditory or visual systems)
instructing a public language model to split the task into a plurality of sub-tasks based on a capability of an operation pool which includes a plurality of private services, and to pair the plurality of sub-tasks with respective private services; (This relates to splitting up tasks and pairing tasks using logic and reasoning in the human mind or with pen and paper.)
and instructing the respective private services to perform the plurality of sub-tasks so as to complete the task. (This relates to a human giving instructions to perform a task using voice or pen and paper.)
The Dependent Claims do not include additional limitations that could incorporate the abstract idea into a practical application or cause the Claim as a whole to amount to significantly more than the underlying abstract idea.
Regarding Independent Claim 8, claim 8 is a System claim with limitations similar to that of Claim 1 and is rejected under the same rational.
Regarding Independent Claim 15, claim 15 is a computer program product claim with limitations similar to that of Claim 1 and is rejected under the same rational.
This judicial exception is not integrated into a practical application. In particular, claims 8 and 15 recite additional elements of “processor” and “memory”. For example, in [0021] of the as filed specification, there is description of using computer processors … Processing circuitry 120 may implement multiple processor threads and/or multiple processor cores. Cache 121 is memory that is located in the processor chip package(s)... Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claims are directed to an abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element of using a computer is noted as a general computer. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Further, the additional limitation in the claims noted above are directed towards insignificant solution activity. The claims are not patent eligible.
Dependent claim 2 recites,
“2. The computer-implemented method of claim 1, wherein the task is further described with multimodal information, and wherein the public language model includes a Multimodal Large Language Model (MLLM). (This relates to a task a human can receive using auditory or visual systems.) no additional limitations present.
Dependent claim 3 recites,
“3. The computer-implemented method of claim 1, wherein the capability of the operation pool is composed of capabilities of the plurality of private services, and the capabilities of the plurality of private services are not overlapped from each other. (This relates to a human performing capabilities of an operation pool using pen and paper or in the human mind.) no additional limitations present.
Dependent claim 4 recites,
“4. The computer-implemented method of claim 3, wherein the method further comprises: indicating the capability of the operation pool to the public language model by describing functions of each of the plurality of private services with natural language.” (This relates to human indicating capability using pen and paper.) no additional limitations present.
Dependent claim 5 recites,
“5. The computer-implemented method of claim 1, wherein the method further comprises: instructing the public language model to generate an execution order of the respective private services based on dependency relations of the private services; and wherein the plurality of sub-tasks are performed by the respective private services based on the execution order. (This relates to a human giving instructions verbally or using pen and paper.)
Dependent claim 6 recites,
“6. The computer-implemented method of claim 5, wherein the execution order is represented by a Directed Acyclic Graph (DAG) generated by the public language model, each node of the DAG corresponding to a private service with a state variable indicating an execution state of the private service, and an edge between two nodes of the DAG corresponding to a dependency relation between two respective private services. (This relates to a human creating a Directed Acyclic Graph using pen and paper.)
Dependent claim 7 recites,
“7. The computer-implemented method of claim 6, wherein the DAG is generated based on a Bayesian network model.” (This relates to a human creating a Directed Acyclic Graph using pen and paper.)
As to Claim 9, Claim 9 is a system claim with limitations similar to that of claim 2 and is rejected under the same rationale.
As to Claim 10, Claim 10 is a system claim with limitations similar to that of claim 3 and is rejected under the same rationale.
As to Claim 11, Claim 11 is a system claim with limitations similar to that of claim 4 and is rejected under the same rationale.
As to Claim 12, Claim 12 is a system claim with limitations similar to that of claim 5 and is rejected under the same rationale.
As to Claim 13, Claim 13 is a system claim with limitations similar to that of claim 6 and is rejected under the same rationale.
As to Claim 14, Claim 14 is a system claim with limitations similar to that of claim 7 and is rejected under the same rationale.
As to Claim 16, Claim 16 is a computer program product claim with limitations similar to that of claim 2 and is rejected under the same rationale.
As to Claim 17, Claim 17 is a computer program product claim with limitations similar to that of claim 3 and is rejected under the same rationale.
As to Claim 18, Claim 18 is a computer program product claim with limitations similar to that of claim 4 and is rejected under the same rationale.
As to Claim 19, Claim 19 is a computer program product claim with limitations similar to that of claim 5 and is rejected under the same rationale.
As to Claim 20, Claim 20 is a computer program product claim with limitations similar to that of claim 6 and is rejected under the same rationale.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-5 are rejected under 35 U.S.C. 102a2 as being unpatentable over Song (NPL Adaptive In-conversation Team Building for Language Model Agents).
Regarding independent Claim 1, Song teaches 1. A computer-implemented method comprising: receiving a task described at least in part with natural language; (see Song Section 2.1 “The overall workflow of Captain Agent is illustrated in Figure 2. Given a task, Captain Agent is prompted to derive a plan before task execution. According to the plan, Captain Agent will repeat the following two steps until it thinks the task is done and output the results: (Step 1) Captain Agent will first identify a subtask instructed by our prompt, list several roles needed for this subtask, and then create a team of agents accordingly by retrieval, selection, and generation. Each of these will be equipped with predefined tools retrieved from the tool library (Section 2.2); (Step 2) this team of agents will attempt to solve the subtask via conversation with the free-form tool using. Once it’s done, a reflector LLM will provide Captain Agent with a reflection report for it to decide whether to adjust the team or subtask instruction or to terminate and output the results (Section 2.3).”) (see Song Introduction Paragraph 4, Line 6 “e.g., Please solve the following math problems” (examiner interprets natural language as “e.g. please solve…”) (See Song Figure 2 “user proxy”) instructing a public language model (see Song Table 2 “Comparison results on different real-world scenarios. We record each scenario’s accuracy for each baseline and Captain Agent, and mark the best results in bold. We adopt gpt-4-0125-preview as the backbone LLM model for all baselines and Captain Agent.”) to split the task into a plurality of sub-tasks based on a capability of an operation pool which includes a plurality of private services, and to pair the plurality of sub-tasks with respective private services; (See Song Figure 1 “instruct” “task decomposition”) (see Song Section 2.1 “The overall workflow of Captain Agent is illustrated in Figure 2. Given a task, Captain Agent is prompted to derive a plan before task execution. According to the plan, Captain Agent will repeat the following two steps until it thinks the task is done and output the results: (Step 1) Captain Agent will first identify a subtask instructed by our prompt, list several roles needed for this subtask, and then create a team of agents accordingly by retrieval, selection, and generation. Each of these will be equipped with predefined tools retrieved from the tool library (Section 2.2); (Step 2) this team of agents will attempt to solve the subtask via conversation with the free-form tool using. Once it’s done, a reflector LLM will provide Captain Agent with a reflection report for it to decide whether to adjust the team or subtask instruction or to terminate and output the results (Section 2.3).”) (see Song Figure 3, Agent and Tool Retrieval ““Model”: [gpt-4…]” (examiner interprets private services as ““Model”: [gpt-4…]” examiner further notes Song uses AutoGen (see appendex F) to call any of "gpt -4", " 0125 ", " 1106 ", " claude3 ", " sonnet ", " haiku ", 5 6 "gemini -1.5 ", " llama3 ", "8b", "70b", " mixtral ", "8 x22b ", "8x7b"] ) (examiner notes Song explicitly teaches a plurality of subtasks in section 4. Paragraph 2, Lines 8 and 9) “involves a meta-model decomposing tasks and assigning subtasks to different LLMs for completion and aggregation.”) and instructing the respective private services to perform the plurality of sub-tasks so as to complete the task. (see song Figure 3 (bottom right) “General Task and Coding Instructions”) (see Song Figure 3, Agent and Tool Retrieval ““Model”: [gpt-4…]” (examiner interprets private services as ““Model”: [gpt-4…]” examiner further notes Song uses AutoGen (see appendex F) to call any of "gpt -4", " 0125 ", " 1106 ", " claude3 ", " sonnet ", " haiku ", 5 6 "gemini -1.5 ", " llama3 ", "8b", "70b", " mixtral ", "8 x22b ", "8x7b"] )
As to Claim 2, Song teaches 2. The computer-implemented method of claim 1,
Furthermore, Song teaches wherein the task is further described with multimodal information, and wherein the public language model includes a Multimodal Large Language Model (MLLM). (see Song Figure 3, Agent and Tool Retrieval ““Model”: [gpt-4…]” see Song appendex F AutoGen to call any of "gpt -4", " 0125 ", " 1106 ", " claude3 ", " sonnet ", " haiku ", 5 6 "gemini -1.5 ", " llama3 ", "8b", "70b", " mixtral ", "8 x22b ", "8x7b"] (examiner notes gpt 4, gemini 1.5 are Multimodal Large Language Model (MLLM)).
As to Claim 3, Song teaches 3. The computer-implemented method of claim 1,
Furthermore, Song teaches wherein the capability of the operation pool is composed of capabilities of the plurality of private services, and the capabilities of the plurality of private services are not overlapped from each other. (see Song Figure 3 Agent Generation “skill” “expert” (examiner interprets capabilities as “skill”))(see Song Section 3, Paragraph 2 “For mathematics problems, programming, data analysis, and scientific scenarios,… Vanilla LLM (prompt an LLM once for an answer), AutoAgents [1], Meta-prompting [34], and a two-agent system (a system involving an Assistant agent with an Executor agent) realized with AutoGen [33]… For meta-prompting, we improve the code execution ability of meta-prompting by reproducing it with the AutoGen framework… For world information retrieval scenarios, we compare Captain Agent with the top-5 baselines (with reference) reported to the GAIA validation leaderboard, which includes AutoGen: GAIA _Orchestrator (a specific three-agent setting organized by an Orchestrator agent designed for GAIA ) [59], FRIDAY [60], Warm-up Act4 , and HuggingFace Agent [61]. All these baselines have a gpt-4-1106-preview backbone, except the HuggingFace Agent equipped with an LLaMA-3-70B as the backbone.”) (see Song Table 2)
As to Claim 4, Song teaches 4. The computer-implemented method of claim 3,
Furthermore, Song teaches wherein the method further comprises: indicating the capability of the operation pool to the public language model by describing functions of each of the plurality of private services with natural language. (see Song Table 1: Scenario, Dataset, and Samples: 1-6. Examiner notes natural language is used in Samples 1-6 as word problems and questions.)
As to Claim 5, Song teaches 5. The computer-implemented method of claim 1,
Furthermore, Song teaches wherein the method further comprises: instructing the public language model to generate an execution order of the respective private services based on dependency relations of the private services; (see Song Figures 1-3) (See Song Figure 3 “It will generate the agent’s name and task-specific instructions, (examiner interprets order as “task-specific instructions”) combined with general task and coding skills and group chat instructions as the final system message. “) and wherein the plurality of sub-tasks are performed by the respective private services based on the execution order. (see Song Section 2.1 “The overall workflow of Captain Agent is illustrated in Figure 2. Given a task, Captain Agent is prompted to derive a plan before task execution. According to the plan, Captain Agent will repeat the following two steps until it thinks the task is done and output the results: (Step 1) Captain Agent will first identify a subtask instructed by our prompt, list several roles needed for this subtask, and then create a team of agents accordingly by retrieval, selection, and generation. Each of these will be equipped with predefined tools retrieved from the tool library (Section 2.2); (Step 2) this team of agents will attempt to solve the subtask via conversation with the free-form tool using. Once it’s done, a reflector LLM will provide Captain Agent with a reflection report for it to decide whether to adjust the team or subtask instruction or to terminate and output the results (Section 2.3).”)
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 6-20 are rejected under 35 U.S.C. 103 as being unpatentable over Song (NPL Adaptive In-conversation Team Building for Language Model Agents) in view of Stergioudis (U.S. Patent Number US 20220230095 A1).
Regarding Independent Claim 8, Song teaches receiving a task described at least in part with natural language; (see Song Section 2.1 “The overall workflow of Captain Agent is illustrated in Figure 2. Given a task, Captain Agent is prompted to derive a plan before task execution. According to the plan, Captain Agent will repeat the following two steps until it thinks the task is done and output the results: (Step 1) Captain Agent will first identify a subtask instructed by our prompt, list several roles needed for this subtask, and then create a team of agents accordingly by retrieval, selection, and generation. Each of these will be equipped with predefined tools retrieved from the tool library (Section 2.2); (Step 2) this team of agents will attempt to solve the subtask via conversation with the free-form tool using. Once it’s done, a reflector LLM will provide Captain Agent with a reflection report for it to decide whether to adjust the team or subtask instruction or to terminate and output the results (Section 2.3).”) (see Song Introduction Paragraph 4, Line 6 “e.g., Please solve the following math problems” (examiner interprets natural language as “e.g. please solve…”) (See Song Figure 2 “user proxy”) instructing a public language model (see Song Table 2 “Comparison results on different real-world scenarios. We record each scenario’s accuracy for each baseline and Captain Agent, and mark the best results in bold. We adopt gpt-4-0125-preview as the backbone LLM model for all baselines and Captain Agent.”) to split the task into a plurality of sub-tasks based on a capability of an operation pool which includes a plurality of private services, and to pair the plurality of sub-tasks with respective private services; (See Song Figure 1 “instruct” “task decomposition”) (see Song Section 2.1 “The overall workflow of Captain Agent is illustrated in Figure 2. Given a task, Captain Agent is prompted to derive a plan before task execution. According to the plan, Captain Agent will repeat the following two steps until it thinks the task is done and output the results: (Step 1) Captain Agent will first identify a subtask instructed by our prompt, list several roles needed for this subtask, and then create a team of agents accordingly by retrieval, selection, and generation. Each of these will be equipped with predefined tools retrieved from the tool library (Section 2.2); (Step 2) this team of agents will attempt to solve the subtask via conversation with the free-form tool using. Once it’s done, a reflector LLM will provide Captain Agent with a reflection report for it to decide whether to adjust the team or subtask instruction or to terminate and output the results (Section 2.3).”) (see Song Figure 3, Agent and Tool Retrieval ““Model”: [gpt-4…]” (examiner interprets private services as ““Model”: [gpt-4…]” examiner further notes Song uses AutoGen (see appendex F) to call any of "gpt -4", " 0125 ", " 1106 ", " claude3 ", " sonnet ", " haiku ", 5 6 "gemini -1.5 ", " llama3 ", "8b", "70b", " mixtral ", "8 x22b ", "8x7b"] ) (examiner notes Song explicitly teaches a plurality of subtasks in section 4. Paragraph 2, Lines 8 and 9) “involves a meta-model decomposing tasks and assigning subtasks to different LLMs for completion and aggregation.”) and instructing the respective private services to perform the plurality of sub-tasks so as to complete the task. (see song Figure 3 (bottom right) “General Task and Coding Instructions”) (see Song Figure 3, Agent and Tool Retrieval ““Model”: [gpt-4…]” (examiner interprets private services as ““Model”: [gpt-4…]” examiner further notes Song uses AutoGen (see appendex F) to call any of "gpt -4", " 0125 ", " 1106 ", " claude3 ", " sonnet ", " haiku ", 5 6 "gemini -1.5 ", " llama3 ", "8b", "70b", " mixtral ", "8 x22b ", "8x7b"] )
Song does not specifically teach 8. A system comprising: one or more processors; a memory coupled to at least one of the one or more processors; a set of computer program instructions stored in the memory and executed by at least one of the one or more processors in order to perform actions of: However, Stergioudis does teach this limitation (see Stergioudis [0035] “As shown in FIG. 1A, computer 100 may include one or more processors 105, volatile memory 110 (e.g., random access memory (RAM)), non-volatile memory 120 (e.g., one or more hard disk drives (HDDs) or other magnetic or optical storage media,…”)
Song and Stergioudis are in the same field of endeavor of signal processing, therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system of Song to incorporate A system comprising: one or more processors; a memory coupled to at least one of the one or more processors; a set of computer program instructions stored in the memory and executed by at least one of the one or more processors in order to perform actions of Stergioudis. This allows for system elements to be deployed in environments as recognized by Stergioudis [0034].
Regarding Independent Claim 15, claim 15 is a storage medium claim with limitations similar to that of Claim 8 and is rejected under the same rational. Furthermore, Stergioudis teaches 15. A computer program product comprising a computer readable storage medium having program instructions embodied therewith, wherein the program instructions being executable by a device to perform a method comprising: (see Stergioudis [0017] At least one aspect is directed to a non-transitory computer-readable medium including instructions that, when executed by one or more processors, train a model for a virtual assistant that interfaces with one or more virtual applications hosted on one or more servers
As to Claim 6, Song teaches 6. The computer-implemented method of claim 5,
Song does not specifically teach wherein the execution order is represented by a Directed Acyclic Graph (DAG) generated by the public language model, However, Stergioudis does teach this limitation. (see Stergioudis [0069] “… Bayesian network can refer to a probabilistic graphical model that can represent a set of variables and their conditional dependencies via a direct acyclic graph...”) each node of the DAG corresponding to a private service with a state variable indicating an execution state of the private service, and an edge between two nodes of the DAG corresponding to a dependency relation between two respective private services. (see Stergioudis [0069] The 3P system 228 can be provided, managed or otherwise maintained by any third-party entity, organization or company. The 3P system 228 can provide one or more task or function based on a machine learning technique. The 3P system 228 can interface with the data processing system 202 via an interface, such as an application programming interface. The 3P system 228 can include a 3P model generator 230 designed, constructed and operational to generate a 3P model 238. The 3P model generator 230 can receive training data from a data processing system 202 or other source, and train the 3P model 238 using one or more machine learning technique. For example, the 3P model generator 230 can train the 3P model 238 using one or more of neural networks, decision trees, support vector machines, regression analysis, or Bayesian networks. A neural network can refer to or include a model based on a collection of connected units or nodes that can be based on the neurons in a biological brain. Example neural networks can include convolution neural network, long short-term memory; or generative adversarial networks. Decision tree learning can utilize a predictive model to go from observations about an item to conclusions about the item's target value. The decision tree model can include classification trees or regression trees. A support vector machine can refer to a supervised learning model configured to analyze data for classification or regression analysis. For example, given a set of training examples that are marked as belonging to one of two categories, a support-vector machine can build a model that assigns new examples to one category or the other. Regression analysis can include a set of statistical processes for estimating relationships between dependent variables (e.g., outcome variable) and one or more independent variables (e.g., predictors, covariates, or features). Bayesian network can refer to a probabilistic graphical model that can represent a set of variables and their conditional dependencies via a direct acyclic graph. A genetic function can refer to a metaheuristic based on the process of natural selection. Thus, the 3P model generator 230 can be configured with any type of machine learning technique or artificial intelligence technique.”) (see Stergioudis [0070-0072] The 3P model generator 230 can train a 3P model 238 based on received training data. The 3P model generator 230 can maintain the 3P model 238 by updating or re-training the 3P model 238 based on subsequently received data or feedback data. The 3P model generator 230 can store the 3P model 238 in a database or other data repository accessible to the 3P system 228. [0071] In some cases, the 3P system 228 can include a 3P ML-based service 232. The 3P ML-based service 232 can perform a function, task or provide a service based on or using the 3P model 238 trained by the 3P model generator 230. The 3P ML-based service 232 can include, for example, a virtual assistant, autonomous driving functionality, natural language understanding, conversational dialogue generator, computer vision, object detection, etc. In some cases, the 3P ML-based service 232 can interface with an intermediary ML-based agent 212 of the data processing system 202 to perform a task or function. For example, the client computing device 234 can receive voice input from a user of the client computing device 234. The voice input can include a hotword, wakeup word or other trigger keyword. The client computing device 234 can transmit the voice input to the data processing system 202 or intermediary ml-based agent 212 for processing. The intermediary ML-based agent 212 can forward the voice input to the 3P ML-based service 232 for natural language processing. The ML-based service 232 can transmit the output or a command back to the intermediary ML-based agent 212 to perform a function. The function can include, for example, performing a task via a virtual application accessible via the client application 236. In some cases, the client application 236 can be configured to forward the voice input to the 3P ML-based service 232.”)
Song in view of Stergioudis are in the same field of endeavor of signal processing, therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified method of Song to incorporate wherein the execution order is represented by a Directed Acyclic Graph (DAG) generated by the public language model, each node of the DAG corresponding to a private service with a state variable indicating an execution state of the private service, and an edge between two nodes of the DAG corresponding to a dependency relation between two respective private services of Stergioudis. This allows for knowledge distillation and promotes active learning and improves the machine learning model as recognized by Stergioudis [0003].
As to Claim 7, Song in view of Stergioudis teaches 7. The computer-implemented method of claim 6,
Furthermore, Stergioudis teaches wherein the DAG is generated based on a Bayesian network model. (see Stergioudis [0069] “… Bayesian network can refer to a probabilistic graphical model that can represent a set of variables and their conditional dependencies via a direct acyclic graph...”)
Song in view of Stergioudis are in the same field of endeavor of signal processing, therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified method of Song and Stergioudis to incorporate wherein the DAG is generated based on a Bayesian network model of Stergioudis. This allows for knowledge distillation and promotes active learning and improves the machine learning model as recognized by Stergioudis [0003].
As to Claim 9, Song in view of Stergioudis teaches 9. The system of claim 8,
Furthermore, Song teaches wherein the task is further described with multimodal information, and wherein the public language model includes a Multimodal Large Language Model (MLLM). (see Song Figure 3, Agent and Tool Retrieval ““Model”: [gpt-4…]” see Song appendex F AutoGen to call any of "gpt -4", " 0125 ", " 1106 ", " claude3 ", " sonnet ", " haiku ", 5 6 "gemini -1.5 ", " llama3 ", "8b", "70b", " mixtral ", "8 x22b ", "8x7b"] (examiner notes gpt 4, gemini 1.5 are Multimodal Large Language Model (MLLM)).
As to Claim 10, Song in view of Stergioudis teaches 10. The system of claim 8,
Furthermore, Song teaches wherein the capability of the operation pool is composed of capabilities of the plurality of private services, and the capabilities of the plurality of private services are not overlapped from each other. (see Song Figure 3 Agent Generation “skill” “expert” (examiner interprets capabilities as “skill”))(see Song Section 3, Paragraph 2 “For mathematics problems, programming, data analysis, and scientific scenarios,… Vanilla LLM (prompt an LLM once for an answer), AutoAgents [1], Meta-prompting [34], and a two-agent system (a system involving an Assistant agent with an Executor agent) realized with AutoGen [33]… For meta-prompting, we improve the code execution ability of meta-prompting by reproducing it with the AutoGen framework… For world information retrieval scenarios, we compare Captain Agent with the top-5 baselines (with reference) reported to the GAIA validation leaderboard, which includes AutoGen: GAIA _Orchestrator (a specific three-agent setting organized by an Orchestrator agent designed for GAIA ) [59], FRIDAY [60], Warm-up Act4 , and HuggingFace Agent [61]. All these baselines have a gpt-4-1106-preview backbone, except the HuggingFace Agent equipped with an LLaMA-3-70B as the backbone.”) (see Song Table 2)
As to Claim 11, Song in view of Stergioudis teaches 11. The system of claim 10,
Furthermore, Song teaches wherein the actions further comprise: indicating the capability of the operation pool to the public language model by describing functions of each of the plurality of private services with natural language. (see Song Table 1: Scenario, Dataset, and Samples: 1-6. Examiner notes natural language is used in Samples 1-6 as word problems and questions.)
As to Claim 12, Song in view of Stergioudis teaches The system of claim 8,
Furthermore, Song teaches wherein the actions further comprise: instructing the public language model to generate an execution order of the respective private services based on dependency relations of the private services; and wherein the plurality of sub-tasks are performed by the respective private services based on the execution order. (see Song Figures 1-3) (See Song Figure 3 “It will generate the agent’s name and task-specific instructions,(examiner interprets order as “task-specific instructions”) combined with general task and coding skills and group chat instructions as the final system message. “) (see Song Section 2.1 “The overall workflow of Captain Agent is illustrated in Figure 2. Given a task, Captain Agent is prompted to derive a plan before task execution. According to the plan, Captain Agent will repeat the following two steps until it thinks the task is done and output the results: (Step 1) Captain Agent will first identify a subtask instructed by our prompt, list several roles needed for this subtask, and then create a team of agents accordingly by retrieval, selection, and generation. Each of these will be equipped with predefined tools retrieved from the tool library (Section 2.2); (Step 2) this team of agents will attempt to solve the subtask via conversation with the free-form tool using. Once it’s done, a reflector LLM will provide Captain Agent with a reflection report for it to decide whether to adjust the team or subtask instruction or to terminate and output the results (Section 2.3).”)
As to Claim 13, Claim 13 is a system claim with limitations similar to that of claim 6 and is rejected under the same rationale.
As to Claim 14, Claim 14 is a system claim with limitations similar to that of claim 7 and is rejected under the same rationale.
As to Claim 16, Claim 16 is a computer program product claim with limitations similar to that of claim 9 and is rejected under the same rationale.
As to Claim 17, Claim 17 is a computer program product claim with limitations similar to that of claim 10 and is rejected under the same rationale.
As to Claim 18, Claim 18 is a computer program product claim with limitations similar to that of claim 11 and is rejected under the same rationale.
As to Claim 19, Claim 19 is a computer program product claim with limitations similar to that of claim 12 and is rejected under the same rationale.
As to Claim 20, Claim 20 is a computer program product claim with limitations similar to that of claim 13 and is rejected under the same rationale.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KRISTEN MICHELLE MASTERS whose telephone number is (703)756-1274. The examiner can normally be reached M-F 8:30 AM - 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pierre Louis Desir can be reached at 571-272-7799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KRISTEN MICHELLE MASTERS/Examiner, Art Unit 2659
/PIERRE LOUIS DESIR/Supervisory Patent Examiner, Art Unit 2659