Detailed Action
This communication is in response to the Application filed on 4/9/2026.
Claims 1-20 are pending and have been examined.
Claims 1-20 are rejected. Hence, this action has been made Final.
Independent Claims 1, 8 and 15 are parallel method, system and computer program product claims, respectively.
Apparent priority: 6/17/2024.
Any previous objection/rejection not mentioned in this Office Action has been withdrawn by the Examiner.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The Applicant has amended the claims to include “unavailable to the public; receiving the plurality of sub-tasks paired with respective private services:”
Regarding the Claim Rejections - 35 USC § 101
Applicant notes Under the USPTO's 2019 Guidance, for a claim to be directed to an abstract idea, it must recite the abstract idea (Prong One), and the abstract idea must not be integrated into a practical application (Prong Two). Applicant respectfully submits, for reasons which will now be discussed, using claim 1 as an example, that the present claims do not satisfy the requirements of Prong One or Prong Two, and that therefore the claims are not directed to an abstract idea.
Claim 1, as amended, recites:
1. (Currently Amended) A computer-implemented method comprising:
receiving a task described at least in part with natural language;
instructing a public language model to split the task into a plurality of sub-tasks based
on a capability of an operation pool which includes a plurality of private services, and to pair
the plurality of sub-tasks with respective private services unavailable to the public;
receiving the plurality of sub-tasks paired with respective private services; and
instructing the respective private services to perform the plurality of sub-tasks so as to
complete the task.
First, regarding Prong One, the Office Action states, on page 3, that claim 1 is directed to the abstract ideas of "a human receiving a task using the auditory or visual systems," "splitting up tasks and pairing the tasks using logic and reasoning in the human mind or with pen and paper," and "a human giving instructions to perform a task using voice or pen and paper." While Applicant does not concede to this interpretation of claim 1, Applicant submits that even if claim 1 were directed to such a concept, the alleged abstract idea does not fall within any of the subject matter groupings of abstract ideas enumerated in the 2019 Guidance (i.e. "Mathematical concepts," "Certain methods of organizing human activity," or "Mental Processes"). Therefore, for at least this reason, claim 1 does not recite an abstract idea, failing Prong One of the USPTO's required analysis.
Examiner notes "a human receiving a task using the auditory or visual systems," "splitting up tasks and pairing the tasks using logic and reasoning in the human mind or with pen and paper," and "a human giving instructions to perform a task using voice or pen and paper." Falls under mental processes and organizing human activity.
Applicant notes Regarding Prong Two, under the USPTO's 2019 Guidance, if a claim recites an abstract idea, it must then be analyzed to determine whether the recited judicial exception is integrated into a practical application of that exception. While Applicant does not concede that the claims, as amended, recite a judicial exception (as stated above), Applicant submits that the amended claims clearly integrate any alleged abstract ideas into practical applications, and that therefore the claims are not directed to any abstract ideas.
In describing examples of such an integration, the 2019 Guidance describes "an
additional [claim] element [that] reflects an improvement in the functioning of a computer, or an improvement to other technology or technical field," (2019 Guidance, page 20, emphasis added). In the present case, Applicant respectfully submits that claim 1 integrates its features into a practical application of a public and private server integration system at least by providing improvements to existing public and private server integration system technology.
As noted above, the claimed invention integrates the benefits of public LLMs and private language models by using the public language model to act as a brain for splitting a proposed task into a series of sub-tasks and selecting corresponding private services (small-scale LMs) to pair with the sub-tasks ([0044]). Such a solution obviates the flaws identified above in the current public/private language model integration landscape and additionally improves upon other areas as noted by at least paragraphs [0073]-[0077] of the as-filed Specification (emphasis added):
Therefore, because claim 1 integrates its features into practical applications (specifically, practical applications of a public and private language model integration system) that apply, rely on, or use any alleged abstract ideas in a manner that imposes meaningful limits, claim 1 fails Prong Two of the USPTO's required analysis.
Examiner notes Applicant argues the claim integrates the exception into a practical application by solving flaws identified above in the current public/private language model integration landscape and by reciting a specific combination of public LLMs and private language models by using the public language model to act as a brain for splitting a proposed task into a series of sub-tasks and selecting corresponding private services (small-scale LMs) to pair with the sub-tasks.
Examiner notes the eligibility inquiry is primarily directed to the claim language as a whole and whether the claim recites particular technical means that show an improvement in computer functionality or other technology.
On the present claim wording, the limitations are largely functional and outcome oriented (receiving a task, instructing, pairing subtasks, receiving subtasks, instructing services) The claims lack sufficient computational detail of how the arrangements materially improve the functioning of the computer system itself (e.g., speed/latency reductions, memory or computational efficiency, novel data representations that reduce error by a measurable metric, or specific unconventional network architectures constrained in a way that produces the improvement).
The applicants’ arguments have been fully considered but do not overcome the 35 USC § 101 claim rejections.
Claim Rejections - Anticipation -- 35 USC @ 102
Applicant notes Song fails to disclose all elements of the independent claims. For example, Applicant respectfully submits that Song fails to disclose "private services unavailable to the public."
For example, the Office Action indicates on p. 8 that "examiner interprets private
services as ""Model": [gpt-4]" and "examiner further notes Song uses Autogen (see appendex F) to call any of "gpt-4", "0125", "1106", "claude 3", "sonnet", "haiku", 5 6 "gemini -1.5", "llama", "8b", "70b", "mixtral", "8x22b", "8x7b", "8x7b"J). Applicant respectfully submits that the LLMs disclosed by Song are public and, in fact, the Office Action also equates "the public language model" recited by the claims to gpt-4 (OA, p. 8). Thus, Applicant respectfully submits that the Office Action cites public language models for "private services."
Examiner notes “gpt-4”, "claude 3", "sonnet", "haiku", 5 6 "gemini -1.5" as proprietary models (offered often through subscription services) that contain closed-weights that are not available to the public. By contrast, “Llama” and “mixtral” are open-weight models that can be downloaded and run locally.
By contrast, and as noted above, embodiments of the claimed invention utilize non- public, "private services" to execute the subtasks. Applicant respectfully submits that because the claimed invention utilizes non-public language models to perform the subtasks instead of public models as disclosed by Song, the claims are distinguished from the teachings of Song. Examiner applicants amendment private services unavailable to the public caused a new interpretation of the claims.
Examiner interprets public language model as “hand-crafted ConversableAgent”
Claim Rejections - Obviousness - 35 USC @ 103
Claims 6-20 are rejected under 35 U.S.C. 103 as being unpatentable over Song in view of Stergioudis (U.S. Patent Number US 20220230095 A1, hereinafter "Stergioudis").
B. Argument
Applicant respectfully submits that Stergioudis fails to cure the deficiencies of Song with respect to utilizing non-public language models. For example, while Stergioudis may disclose "one or more processors" in [0035], Applicant nonetheless respectfully submits that Stergioudis fails to disclose "instructing a public language model to split the task into a plurality of sub-tasks based on a capability of an operation pool which includes a plurality of private services, and to pair the plurality of sub-tasks with respective private services unavailable to the public."
Examiner notes song teaches this particular limitation, as noted in the office action below.
Applicant's arguments have been fully considered but they are not persuasive.
Updated mappings to reflect the amendments to the independent claims have been provided below.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Independent claim 1 recites,
“1. A computer-implemented method comprising: receiving a task described at least in part with natural language; (This relates to a human receiving a task using the auditory or visual systems)
instructing a public language model to split the task into a plurality of sub-tasks based on a capability of an operation pool which includes a plurality of private services, and to pair the plurality of sub-tasks with respective private services unavailable to the public; (This relates to splitting up tasks and pairing tasks using logic and reasoning in the human mind or with pen and paper.) receiving the plurality of sub-tasks paired with respective private services: (This relates to receiving tasks using pen and paper.)
and instructing the respective private services to perform the plurality of sub-tasks so as to complete the task. (This relates to a human giving instructions to perform a task using voice or pen and paper.)
The Dependent Claims do not include additional limitations that could incorporate the abstract idea into a practical application or cause the Claim as a whole to amount to significantly more than the underlying abstract idea.
Regarding Independent Claim 8, claim 8 is a System claim with limitations similar to that of Claim 1 and is rejected under the same rational.
Regarding Independent Claim 15, claim 15 is a computer program product claim with limitations similar to that of Claim 1 and is rejected under the same rational.
This judicial exception is not integrated into a practical application. In particular, claims 8 and 15 recite additional elements of “processor” and “memory”. For example, in [0021] of the as filed specification, there is description of using computer processors … Processing circuitry 120 may implement multiple processor threads and/or multiple processor cores. Cache 121 is memory that is located in the processor chip package(s)... Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claims are directed to an abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element of using a computer is noted as a general computer. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Further, the additional limitation in the claims noted above are directed towards insignificant solution activity. The claims are not patent eligible.
Dependent claim 2 recites,
“2. The computer-implemented method of claim 1, wherein the task is further described with multimodal information, and wherein the public language model includes a Multimodal Large Language Model (MLLM). (This relates to a task a human can receive using auditory or visual systems.) no additional limitations present.
Dependent claim 3 recites,
“3. The computer-implemented method of claim 1, wherein the capability of the operation pool is composed of capabilities of the plurality of private services, and the capabilities of the plurality of private services are not overlapped from each other. (This relates to a human performing capabilities of an operation pool using pen and paper or in the human mind.) no additional limitations present.
Dependent claim 4 recites,
“4. The computer-implemented method of claim 3, wherein the method further comprises: indicating the capability of the operation pool to the public language model by describing functions of each of the plurality of private services with natural language.” (This relates to human indicating capability using pen and paper.) no additional limitations present.
Dependent claim 5 recites,
“5. The computer-implemented method of claim 1, wherein the method further comprises: instructing the public language model to generate an execution order of the respective private services based on dependency relations of the private services; and wherein the plurality of sub-tasks are performed by the respective private services based on the execution order. (This relates to a human giving instructions verbally or using pen and paper.)
Dependent claim 6 recites,
“6. The computer-implemented method of claim 5, wherein the execution order is represented by a Directed Acyclic Graph (DAG) generated by the public language model, each node of the DAG corresponding to a private service with a state variable indicating an execution state of the private service, and an edge between two nodes of the DAG corresponding to a dependency relation between two respective private services. (This relates to a human creating a Directed Acyclic Graph using pen and paper.)
Dependent claim 7 recites,
“7. The computer-implemented method of claim 6, wherein the DAG is generated based on a Bayesian network model.” (This relates to a human creating a Directed Acyclic Graph using pen and paper.)
As to Claim 9, Claim 9 is a system claim with limitations similar to that of claim 2 and is rejected under the same rationale.
As to Claim 10, Claim 10 is a system claim with limitations similar to that of claim 3 and is rejected under the same rationale.
As to Claim 11, Claim 11 is a system claim with limitations similar to that of claim 4 and is rejected under the same rationale.
As to Claim 12, Claim 12 is a system claim with limitations similar to that of claim 5 and is rejected under the same rationale.
As to Claim 13, Claim 13 is a system claim with limitations similar to that of claim 6 and is rejected under the same rationale.
As to Claim 14, Claim 14 is a system claim with limitations similar to that of claim 7 and is rejected under the same rationale.
As to Claim 16, Claim 16 is a computer program product claim with limitations similar to that of claim 2 and is rejected under the same rationale.
As to Claim 17, Claim 17 is a computer program product claim with limitations similar to that of claim 3 and is rejected under the same rationale.
As to Claim 18, Claim 18 is a computer program product claim with limitations similar to that of claim 4 and is rejected under the same rationale.
As to Claim 19, Claim 19 is a computer program product claim with limitations similar to that of claim 5 and is rejected under the same rationale.
As to Claim 20, Claim 20 is a computer program product claim with limitations similar to that of claim 6 and is rejected under the same rationale.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-5 are rejected under 35 U.S.C. 102a2 as being unpatentable over Song (NPL Adaptive In-conversation Team Building for Language Model Agents).
Regarding independent Claim 1, Song teaches 1. A computer-implemented method comprising: receiving a task described at least in part with natural language; (see Song Section 2.1 “The overall workflow of Captain Agent is illustrated in Figure 2. Given a task, Captain Agent is prompted to derive a plan before task execution. According to the plan, Captain Agent will repeat the following two steps until it thinks the task is done and output the results: (Step 1) Captain Agent will first identify a subtask instructed by our prompt, list several roles needed for this subtask, and then create a team of agents accordingly by retrieval, selection, and generation. Each of these will be equipped with predefined tools retrieved from the tool library (Section 2.2); (Step 2) this team of agents will attempt to solve the subtask via conversation with the free-form tool using. Once it’s done, a reflector LLM will provide Captain Agent with a reflection report for it to decide whether to adjust the team or subtask instruction or to terminate and output the results (Section 2.3).”) (see Song Introduction Paragraph 4, Line 6 “e.g., Please solve the following math problems” (examiner interprets natural language as “e.g. please solve…”) (See Song Figure 2 “user proxy”) instructing a public language model (see Song Table 2 “Comparison results on different real-world scenarios. We record each scenario’s accuracy for each baseline and Captain Agent, and mark the best results in bold..”)( see Song see Appendix F. “hand-crafted ConversableAgent” archived in AutoGen” (examiner interprets public language model as “hand-crafted ConversableAgent” ) to split the task into a plurality of sub-tasks based on a capability of an operation pool which includes a plurality of private services, and to pair the plurality of sub-tasks with respective private services unavailable to the public; (See Song Figure 1 “instruct” “task decomposition”) (see Song Section 2.1 “The overall workflow of Captain Agent is illustrated in Figure 2. Given a task, Captain Agent is prompted to derive a plan before task execution. According to the plan, Captain Agent will repeat the following two steps until it thinks the task is done and output the results: (Step 1) Captain Agent will first identify a subtask instructed by our prompt, list several roles needed for this subtask, and then create a team of agents accordingly by retrieval, selection, and generation. Each of these will be equipped with predefined tools retrieved from the tool library (Section 2.2); (Step 2) this team of agents will attempt to solve the subtask via conversation with the free-form tool using. Once it’s done, a reflector LLM will provide Captain Agent with a reflection report for it to decide whether to adjust the team or subtask instruction or to terminate and output the results (Section 2.3).”) (see Song Figure 3, Agent and Tool Retrieval ““Model”: [gpt-4…]”) Examiner further notes Song uses AutoGen (see appendex F) to call any of "gpt -4", " 0125 ", " 1106 ", " claude3 ", " sonnet ", " haiku ", 5 6 "gemini -1.5 ", " llama3 ", "8b", "70b", " mixtral ", "8 x22b ", "8x7b"] ) (examiner notes Song explicitly teaches a plurality of subtasks in section 4. Paragraph 2, Lines 8 and 9) “involves a meta-model decomposing tasks and assigning subtasks to different LLMs for completion and aggregation.”) receiving the plurality of sub-tasks paired with respective private services: (See Song Figure 1 Task Decomposition (red, blue, green, orange) subtasks, Retrieval and Selection, Agent Team.”) and instructing the respective private services to perform the plurality of sub-tasks so as to complete the task. (see song Figure 3 (bottom right) “General Task and Coding Instructions”) (see Song Figure 3, Agent and Tool Retrieval ““Model”: [gpt-4…]”…]” ) (examiner interprets private services as ““Model”: [gpt-4…]” …]”) examiner further notes Song uses AutoGen (see appendex F) to call any of "gpt -4", " 0125 ", " 1106 ", " claude3 ", " sonnet ", " haiku ", 5 6 "gemini -1.5 ", " llama3 ", "8b", "70b", " mixtral ", "8 x22b ", "8x7b"] )
As to Claim 2, Song teaches 2. The computer-implemented method of claim 1,
Furthermore, Song teaches wherein the task is further described with multimodal information, and wherein the public language model includes a Multimodal Large Language Model (MLLM). (see Song Figure 3, Agent and Tool Retrieval ““Model”: [gpt-4…]” see Song appendex F AutoGen to call any of "gpt -4", " 0125 ", " 1106 ", " claude3 ", " sonnet ", " haiku ", 5 6 "gemini -1.5 ", " llama3 ", "8b", "70b", " mixtral ", "8 x22b ", "8x7b"] (examiner notes gpt 4, gemini 1.5 are Multimodal Large Language Model (MLLM)).
As to Claim 3, Song teaches 3. The computer-implemented method of claim 1,
Furthermore, Song teaches wherein the capability of the operation pool is composed of capabilities of the plurality of private services, and the capabilities of the plurality of private services are not overlapped from each other. (see Song Figure 3 Agent Generation “skill” “expert” (examiner interprets capabilities as “skill”))(see Song Section 3, Paragraph 2 “For mathematics problems, programming, data analysis, and scientific scenarios,… Vanilla LLM (prompt an LLM once for an answer), AutoAgents [1], Meta-prompting [34], and a two-agent system (a system involving an Assistant agent with an Executor agent) realized with AutoGen [33]… For meta-prompting, we improve the code execution ability of meta-prompting by reproducing it with the AutoGen framework… For world information retrieval scenarios, we compare Captain Agent with the top-5 baselines (with reference) reported to the GAIA validation leaderboard, which includes AutoGen: GAIA _Orchestrator (a specific three-agent setting organized by an Orchestrator agent designed for GAIA ) [59], FRIDAY [60], Warm-up Act4 , and HuggingFace Agent [61]. All these baselines have a gpt-4-1106-preview backbone, except the HuggingFace Agent equipped with an LLaMA-3-70B as the backbone.”) (see Song Table 2)
As to Claim 4, Song teaches 4. The computer-implemented method of claim 3,
Furthermore, Song teaches wherein the method further comprises: indicating the capability of the operation pool to the public language model (see Song Table 2 “Comparison results on different real-world scenarios. We record each scenario’s accuracy for each baseline and Captain Agent, and mark the best results in bold..”)( see Song see Appendix F. “hand-crafted ConversableAgent” archived in AutoGen” (examiner interprets public language model as “hand-crafted ConversableAgent” ) by describing functions of each of the plurality of private services with natural language. (see Song Table 1: Scenario, Dataset, and Samples: 1-6. Examiner notes natural language is used in Samples 1-6 as word problems and questions.)
As to Claim 5, Song teaches 5. The computer-implemented method of claim 1,
Furthermore, Song teaches wherein the method further comprises: instructing the public language model(see Song Table 2 “Comparison results on different real-world scenarios. We record each scenario’s accuracy for each baseline and Captain Agent, and mark the best results in bold..”)( see Song see Appendix F. “hand-crafted ConversableAgent” archived in AutoGen” (examiner interprets public language model as “hand-crafted ConversableAgent” ) to generate an execution order of the respective private services based on dependency relations of the private services; (see Song Figures 1-3) (See Song Figure 3 “It will generate the agent’s name and task-specific instructions, (examiner interprets order as “task-specific instructions”) combined with general task and coding skills and group chat instructions as the final system message. “) and wherein the plurality of sub-tasks are performed by the respective private services based on the execution order. (see Song Section 2.1 “The overall workflow of Captain Agent is illustrated in Figure 2. Given a task, Captain Agent is prompted to derive a plan before task execution. According to the plan, Captain Agent will repeat the following two steps until it thinks the task is done and output the results: (Step 1) Captain Agent will first identify a subtask instructed by our prompt, list several roles needed for this subtask, and then create a team of agents accordingly by retrieval, selection, and generation. Each of these will be equipped with predefined tools retrieved from the tool library (Section 2.2); (Step 2) this team of agents will attempt to solve the subtask via conversation with the free-form tool using. Once it’s done, a reflector LLM will provide Captain Agent with a reflection report for it to decide whether to adjust the team or subtask instruction or to terminate and output the results (Section 2.3).”)
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 6-20 are rejected under 35 U.S.C. 103 as being unpatentable over Song (NPL Adaptive In-conversation Team Building for Language Model Agents) in view of Stergioudis (U.S. Patent Number US 20220230095 A1).
Regarding Independent Claim 8, Song teaches receiving a task described at least in part with natural language; (see Song Section 2.1 “The overall workflow of Captain Agent is illustrated in Figure 2. Given a task, Captain Agent is prompted to derive a plan before task execution. According to the plan, Captain Agent will repeat the following two steps until it thinks the task is done and output the results: (Step 1) Captain Agent will first identify a subtask instructed by our prompt, list several roles needed for this subtask, and then create a team of agents accordingly by retrieval, selection, and generation. Each of these will be equipped with predefined tools retrieved from the tool library (Section 2.2); (Step 2) this team of agents will attempt to solve the subtask via conversation with the free-form tool using. Once it’s done, a reflector LLM will provide Captain Agent with a reflection report for it to decide whether to adjust the team or subtask instruction or to terminate and output the results (Section 2.3).”) (see Song Introduction Paragraph 4, Line 6 “e.g., Please solve the following math problems” (examiner interprets natural language as “e.g. please solve…”) (See Song Figure 2 “user proxy”) instructing a public language model (see Song Table 2 “Comparison results on different real-world scenarios. We record each scenario’s accuracy for each baseline and Captain Agent, and mark the best results in bold..”)( see Song see Appendix F. “hand-crafted ConversableAgent” archived in AutoGen” (examiner interprets public language model as “hand-crafted ConversableAgent” ) to split the task into a plurality of sub-tasks based on a capability of an operation pool which includes a plurality of private services, and to pair the plurality of sub-tasks with respective private services unavailable to the public; (See Song Figure 1 “instruct” “task decomposition”) (see Song Section 2.1 “The overall workflow of Captain Agent is illustrated in Figure 2. Given a task, Captain Agent is prompted to derive a plan before task execution. According to the plan, Captain Agent will repeat the following two steps until it thinks the task is done and output the results: (Step 1) Captain Agent will first identify a subtask instructed by our prompt, list several roles needed for this subtask, and then create a team of agents accordingly by retrieval, selection, and generation. Each of these will be equipped with predefined tools retrieved from the tool library (Section 2.2); (Step 2) this team of agents will attempt to solve the subtask via conversation with the free-form tool using. Once it’s done, a reflector LLM will provide Captain Agent with a reflection report for it to decide whether to adjust the team or subtask instruction or to terminate and output the results (Section 2.3).”) (see Song Figure 3, Agent and Tool Retrieval ““Model”: [gpt-4…]” Examiner further notes Song uses AutoGen (see appendex F) to call any of "gpt -4", " 0125 ", " 1106 ", " claude3 ", " sonnet ", " haiku ", 5 6 "gemini -1.5 ", " llama3 ", "8b", "70b", " mixtral ", "8 x22b ", "8x7b"] ) (examiner notes Song explicitly teaches a plurality of subtasks in section 4. Paragraph 2, Lines 8 and 9) “involves a meta-model decomposing tasks and assigning subtasks to different LLMs for completion and aggregation.”) instructions to receive the plurality of sub-tasks paired with respective private services; (See Song Figure 1 Task Decomposition (red, blue, green, orange) subtasks, Retrieval and Selection, Agent Team.”) and instructing the respective private services to perform the plurality of sub-tasks so as to complete the task. (see song Figure 3 (bottom right) “General Task and Coding Instructions”) (see Song Figure 3, Agent and Tool Retrieval ““Model”: [gpt-4…]”…]”() (examiner interprets private services as ““Model”: [gpt-4…]” …]” ) examiner further notes Song uses AutoGen (see appendex F) to call any of "gpt -4", " 0125 ", " 1106 ", " claude3 ", " sonnet ", " haiku ", 5 6 "gemini -1.5 ", " llama3 ", "8b", "70b", " mixtral ", "8 x22b ", "8x7b"] )
Song does not specifically teach 8. A system comprising: one or more processors; a memory coupled to at least one of the one or more processors; a set of computer program instructions stored in the memory and executed by at least one of the one or more processors in order to perform actions of: However, Stergioudis does teach this limitation (see Stergioudis [0035] “As shown in FIG. 1A, computer 100 may include one or more processors 105, volatile memory 110 (e.g., random access memory (RAM)), non-volatile memory 120 (e.g., one or more hard disk drives (HDDs) or other magnetic or optical storage media,…”)
Song and Stergioudis are in the same field of endeavor of signal processing, therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system of Song to incorporate A system comprising: one or more processors; a memory coupled to at least one of the one or more processors; a set of computer program instructions stored in the memory and executed by at least one of the one or more processors in order to perform actions of Stergioudis. This allows for system elements to be deployed in environments as recognized by Stergioudis [0034].
Regarding Independent Claim 15, claim 15 is a storage medium claim with limitations similar to that of Claim 8 and is rejected under the same rational. Furthermore, Stergioudis teaches 15. A computer program product comprising a computer readable storage medium having program instructions embodied therewith, wherein the program instructions being executable by a device to perform a method comprising: (see Stergioudis [0017] At least one aspect is directed to a non-transitory computer-readable medium including instructions that, when executed by one or more processors, train a model for a virtual assistant that interfaces with one or more virtual applications hosted on one or more servers
As to Claim 6, Song teaches 6. The computer-implemented method of claim 5,
Song does not specifically teach wherein the execution order is represented by a Directed Acyclic Graph (DAG) generated by the public language model, However, Stergioudis does teach this limitation. (see Stergioudis [0069] “… Bayesian network can refer to a probabilistic graphical model that can represent a set of variables and their conditional dependencies via a direct acyclic graph...”) each node of the DAG corresponding to a private service with a state variable indicating an execution state of the private service, and an edge between two nodes of the DAG corresponding to a dependency relation between two respective private services. (see Stergioudis [0069] The 3P system 228 can be provided, managed or otherwise maintained by any third-party entity, organization or company. The 3P system 228 can provide one or more task or function based on a machine learning technique. The 3P system 228 can interface with the data processing system 202 via an interface, such as an application programming interface. The 3P system 228 can include a 3P model generator 230 designed, constructed and operational to generate a 3P model 238. The 3P model generator 230 can receive training data from a data processing system 202 or other source, and train the 3P model 238 using one or more machine learning technique. For example, the 3P model generator 230 can train the 3P model 238 using one or more of neural networks, decision trees, support vector machines, regression analysis, or Bayesian networks. A neural network can refer to or include a model based on a collection of connected units or nodes that can be based on the neurons in a biological brain. Example neural networks can include convolution neural network, long short-term memory; or generative adversarial networks. Decision tree learning can utilize a predictive model to go from observations about an item to conclusions about the item's target value. The decision tree model can include classification trees or regression trees. A support vector machine can refer to a supervised learning model configured to analyze data for classification or regression analysis. For example, given a set of training examples that are marked as belonging to one of two categories, a support-vector machine can build a model that assigns new examples to one category or the other. Regression analysis can include a set of statistical processes for estimating relationships between dependent variables (e.g., outcome variable) and one or more independent variables (e.g., predictors, covariates, or features). Bayesian network can refer to a probabilistic graphical model that can represent a set of variables and their conditional dependencies via a direct acyclic graph. A genetic function can refer to a metaheuristic based on the process of natural selection. Thus, the 3P model generator 230 can be configured with any type of machine learning technique or artificial intelligence technique.”) (see Stergioudis [0070-0072] The 3P model generator 230 can train a 3P model 238 based on received training data. The 3P model generator 230 can maintain the 3P model 238 by updating or re-training the 3P model 238 based on subsequently received data or feedback data. The 3P model generator 230 can store the 3P model 238 in a database or other data repository accessible to the 3P system 228. [0071] In some cases, the 3P system 228 can include a 3P ML-based service 232. The 3P ML-based service 232 can perform a function, task or provide a service based on or using the 3P model 238 trained by the 3P model generator 230. The 3P ML-based service 232 can include, for example, a virtual assistant, autonomous driving functionality, natural language understanding, conversational dialogue generator, computer vision, object detection, etc. In some cases, the 3P ML-based service 232 can interface with an intermediary ML-based agent 212 of the data processing system 202 to perform a task or function. For example, the client computing device 234 can receive voice input from a user of the client computing device 234. The voice input can include a hotword, wakeup word or other trigger keyword. The client computing device 234 can transmit the voice input to the data processing system 202 or intermediary ml-based agent 212 for processing. The intermediary ML-based agent 212 can forward the voice input to the 3P ML-based service 232 for natural language processing. The ML-based service 232 can transmit the output or a command back to the intermediary ML-based agent 212 to perform a function. The function can include, for example, performing a task via a virtual application accessible via the client application 236. In some cases, the client application 236 can be configured to forward the voice input to the 3P ML-based service 232.”)
Song in view of Stergioudis are in the same field of endeavor of signal processing, therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified method of Song to incorporate wherein the execution order is represented by a Directed Acyclic Graph (DAG) generated by the public language model, each node of the DAG corresponding to a private service with a state variable indicating an execution state of the private service, and an edge between two nodes of the DAG corresponding to a dependency relation between two respective private services of Stergioudis. This allows for knowledge distillation and promotes active learning and improves the machine learning model as recognized by Stergioudis [0003].
As to Claim 7, Song in view of Stergioudis teaches 7. The computer-implemented method of claim 6,
Furthermore, Stergioudis teaches wherein the DAG is generated based on a Bayesian network model. (see Stergioudis [0069] “… Bayesian network can refer to a probabilistic graphical model that can represent a set of variables and their conditional dependencies via a direct acyclic graph...”)
Song in view of Stergioudis are in the same field of endeavor of signal processing, therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified method of Song and Stergioudis to incorporate wherein the DAG is generated based on a Bayesian network model of Stergioudis. This allows for knowledge distillation and promotes active learning and improves the machine learning model as recognized by Stergioudis [0003].
As to Claim 9, Song in view of Stergioudis teaches 9. The system of claim 8,
Furthermore, Song teaches wherein the task is further described with multimodal information, and wherein the public language model includes a Multimodal Large Language Model (MLLM). (see Song Figure 3, Agent and Tool Retrieval ““Model”: [gpt-4…]” see Song appendex F AutoGen to call any of "gpt -4", " 0125 ", " 1106 ", " claude3 ", " sonnet ", " haiku ", 5 6 "gemini -1.5 ", " llama3 ", "8b", "70b", " mixtral ", "8 x22b ", "8x7b"] (examiner notes gpt 4, gemini 1.5 are Multimodal Large Language Model (MLLM)).
As to Claim 10, Song in view of Stergioudis teaches 10. The system of claim 8,
Furthermore, Song teaches wherein the capability of the operation pool is composed of capabilities of the plurality of private services, and the capabilities of the plurality of private services are not overlapped from each other. (see Song Figure 3 Agent Generation “skill” “expert” (examiner interprets capabilities as “skill”))(see Song Section 3, Paragraph 2 “For mathematics problems, programming, data analysis, and scientific scenarios,… Vanilla LLM (prompt an LLM once for an answer), AutoAgents [1], Meta-prompting [34], and a two-agent system (a system involving an Assistant agent with an Executor agent) realized with AutoGen [33]… For meta-prompting, we improve the code execution ability of meta-prompting by reproducing it with the AutoGen framework… For world information retrieval scenarios, we compare Captain Agent with the top-5 baselines (with reference) reported to the GAIA validation leaderboard, which includes AutoGen: GAIA _Orchestrator (a specific three-agent setting organized by an Orchestrator agent designed for GAIA ) [59], FRIDAY [60], Warm-up Act4 , and HuggingFace Agent [61]. All these baselines have a gpt-4-1106-preview backbone, except the HuggingFace Agent equipped with an LLaMA-3-70B as the backbone.”) (see Song Table 2)
As to Claim 11, Song in view of Stergioudis teaches 11. The system of claim 10,
Furthermore, Song teaches wherein the actions further comprise: indicating the capability of the operation pool to the public language model by describing functions of each of the plurality of private services with natural language. (see Song Table 1: Scenario, Dataset, and Samples: 1-6. Examiner notes natural language is used in Samples 1-6 as word problems and questions.) (see Song Table 2 “Comparison results on different real-world scenarios. We record each scenario’s accuracy for each baseline and Captain Agent, and mark the best results in bold..”)( see Song see Appendix F. “hand-crafted ConversableAgent” archived in AutoGen” (examiner interprets public language model as “hand-crafted ConversableAgent” )
As to Claim 12, Song in view of Stergioudis teaches The system of claim 8,
Furthermore, Song teaches wherein the actions further comprise: instructing the public language model(see Song Table 2 “Comparison results on different real-world scenarios. We record each scenario’s accuracy for each baseline and Captain Agent, and mark the best results in bold..”)( see Song see Appendix F. “hand-crafted ConversableAgent” archived in AutoGen” (examiner interprets public language model as “hand-crafted ConversableAgent” ) to generate an execution order of the respective private services based on dependency relations of the private services; and wherein the plurality of sub-tasks are performed by the respective private services based on the execution order. (see Song Figures 1-3) (See Song Figure 3 “It will generate the agent’s name and task-specific instructions,(examiner interprets order as “task-specific instructions”) combined with general task and coding skills and group chat instructions as the final system message. “) (see Song Section 2.1 “The overall workflow of Captain Agent is illustrated in Figure 2. Given a task, Captain Agent is prompted to derive a plan before task execution. According to the plan, Captain Agent will repeat the following two steps until it thinks the task is done and output the results: (Step 1) Captain Agent will first identify a subtask instructed by our prompt, list several roles needed for this subtask, and then create a team of agents accordingly by retrieval, selection, and generation. Each of these will be equipped with predefined tools retrieved from the tool library (Section 2.2); (Step 2) this team of agents will attempt to solve the subtask via conversation with the free-form tool using. Once it’s done, a reflector LLM will provide Captain Agent with a reflection report for it to decide whether to adjust the team or subtask instruction or to terminate and output the results (Section 2.3).”)
As to Claim 13, Claim 13 is a system claim with limitations similar to that of claim 6 and is rejected under the same rationale.
As to Claim 14, Claim 14 is a system claim with limitations similar to that of claim 7 and is rejected under the same rationale.
As to Claim 16, Claim 16 is a computer program product claim with limitations similar to that of claim 9 and is rejected under the same rationale.
As to Claim 17, Claim 17 is a computer program product claim with limitations similar to that of claim 10 and is rejected under the same rationale.
As to Claim 18, Claim 18 is a computer program product claim with limitations similar to that of claim 11 and is rejected under the same rationale.
As to Claim 19, Claim 19 is a computer program product claim with limitations similar to that of claim 12 and is rejected under the same rationale.
As to Claim 20, Claim 20 is a computer program product claim with limitations similar to that of claim 13 and is rejected under the same rationale.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KRISTEN MICHELLE MASTERS whose telephone number is (703)756-1274. The examiner can normally be reached M-F 8:30 AM - 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pierre Louis Desir can be reached at 571-272-7799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KRISTEN MICHELLE MASTERS/Examiner, Art Unit 2659
/PIERRE LOUIS DESIR/Supervisory Patent Examiner, Art Unit 2659
Prosecution Insights