DETAILED ACTION
Notice of AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Regarding U.S. Provisional Patent Application No. 63/552,164 (filed 2/11/2024), Applicant’s claim for the benefit of a prior-filed application under 35 U.S.C. 119(e) is acknowledged.
Information Disclosure Statement
The listing of references in the specification is not a proper information disclosure statement. 37 CFR 1.98(b) requires a list of all patents, publications, or other information submitted for consideration by the Office, and MPEP § 609.04(a) states, "the list may not be incorporated into the specification but must be submitted in a separate paper." Therefore, unless the references have been cited by the examiner on form PTO-892, they have not been considered. In particular, the references cited by the parenthetical citations in at least paras. 0022, 0027, and 0087 have not been considered.
Drawings
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because they include the following reference character(s) not mentioned in the description: Fig. 5, Training Data Generator Submodule 534. Fig. 8, Step 813.
The examiner further notes that the descriptions for 532 and 533 in para. 0048 do not match the language used in Fig. 5.
The examiner further objects to Figs. 3A, 3B, 9A, and 9B because they should be drafted using India Ink. MPEP 608.01(f) and 37 C.F.R. 1.84 (a)(1).
Corrected drawing sheets in compliance with 37 CFR 1.121(d), or amendment to the specification to add the reference character(s) in the description in compliance with 37 CFR 1.121(b) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specification
The disclosure is objected to because of the following informalities:
In para. 0050, the reference to “submodules 631-435” in line 4 does not match the submodules referenced in Fig. 5. The examiner believes this should read “submodules 531-534”.
In para. 0055, the reference to “submodules 631-335” in line 2 does not match the submodules referenced in Fig. 5. The examiner believes this should read “submodules 531-534”.
In para. 0056, lines 1 and 3, “LLM training pipeline module 330” does not exist. This should be 530.
In para. 0056, line 3, “submodules 331-333” should be “submodules 531-534”.
In para. 0072, lines 1 and 3, “LLM training pipeline module 330” does not exist. This should be 530.
Appropriate correction is required.
Claim Objections
Claims 1, 10, and 19 are objected to because of the following informalities:
In claim 1, line 13, “raining dataset” should read “training dataset”.
In claim 10, line 16, “raining dataset” should read “training dataset”.
In claim 19, line 15, “raining dataset” should read “training dataset”.
Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding Step 1 of the Alice/Mayo framework, Claims 1-9 are directed to a method (a process), Claims 10-18 are directed to a system (a machine), and Claims 19-20 are directed to a non-transitory processor-readable storage medium (an article of manufacture), which each fall within one of the four statutory categories of inventions.
Regarding Claim 1
Step 2A, prong 1 (Is the claim directed to a law of nature, a natural phenomenon or an abstract idea).
Claim 1 recites the following mental processes, that in each case under the broadest reasonable interpretation, covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components (e.g., “server”, “hardware platform”, “neural network”).
converting the plurality of multi-turn user-agent interaction records into a plurality of agent trajectories having a homogeneous data format; (under the broadest reasonable interpretation, a human can review multi-turn user-agent records and mentally create (or using pencil and paper) agent trajectories having a homogenous data format, such as a specific tuple for (input, output, observation))
generating, … a plurality of rating scores corresponding to the plurality of agent trajectories; (under the broadest reasonable interpretation, a human can mentally generate a score for each agent trajectory created)
aggregating a training dataset based on the plurality of agent trajectories and the plurality of rating scores; (under the broadest reasonable interpretation, a human can mentally combine an agent trajectory and its corresponding score to create a training dataset, such as by writing the agent trajectory and its score on a piece of paper)
Step 2A, prong 2 (Does the claim recite additional elements that integrate the judicial exception into a practical application?).
The judicial exception is not integrated into a practical application.
Regarding the “A method of training an intelligent agent for carrying out actions in response to user interactions, the method comprising” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception. In particular, the claim only recites the additional element of generically training an intelligent agent, such as a neural network. This additional element is recited at a high-level of generality and amounts to no more than mere instructions to apply the exception using a generic computer technique (generic training). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f)).
Regarding the “receiving, via a communication interface at a server and from a plurality of data source servers, a plurality of multi-turn user-agent interaction records corresponding to a plurality of different formats” limitation, such additional element of a data gathering step is recited at a high level of generality and amounts to extra-solution activity of receiving data, i.e. pre-solution activity of gathering data for use in the claimed process (see MPEP 2106.05(g)).
Regarding the “by a neural network rating model” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception. In particular, the claim only recites the additional element of a generic neural network. This additional element is recited at a high-level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component (generic neural network). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f)).
Regarding the “training a neural network based language models to generate a predicted agent action in response to a user query using the raining dataset” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception. In particular, the claim only recites the additional element of generically training a neural network. This additional element is recited at a high-level of generality and amounts to no more than mere instructions to apply the exception using a generic computer technique (generic training). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f)).
Regarding the “deploying the trained neural network based model on a hardware platform as an agent application for handling different user queries in different user environments” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, because the limitation attempts to cover a solution to an identified problem with no restriction on how the result is accomplished, or provides no description of the mechanism for accomplishing the result. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f)).
Accordingly, at Step 2A, prong two, after considering all claim elements individually and as an ordered combination, it is determined that the claims do not integrate the judicial exception into a practical application.
Step 2B (Does the claim recite additional elements that amount to significantly more than the judicial exception?)
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception.
Regarding the “A method of training an intelligent agent for carrying out actions in response to user interactions, the method comprising” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, because the limitation merely provides instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not add significantly more than the judicial exception. (See MPEP 2106.05(f)).
Regarding the “receiving, via a communication interface at a server and from a plurality of data source servers, a plurality of multi-turn user-agent interaction records corresponding to a plurality of different formats” limitation, as discussed above, the additional element of a data gathering step is recited at a high level of generality and amounts to extra-solution activity of receiving data, i.e. pre-solution activity of gathering data for use in the claimed process. The courts have found limitations directed to obtaining information electronically, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), “receiving or transmitting data over a network”, "electronic record keeping," and "storing and retrieving information in memory").
Regarding the “by a neural network rating model” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, because the limitation merely provides instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not add significantly more than the judicial exception. (See MPEP 2106.05(f)).
Regarding the “training a neural network based language models to generate a predicted agent action in response to a user query using the raining dataset” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, because the limitation merely provides instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not add significantly more than the judicial exception. (See MPEP 2106.05(f)).
Regarding the “deploying the trained neural network based model on a hardware platform as an agent application for handling different user queries in different user environments” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, because the limitation attempts to cover a solution to an identified problem with no restriction on how the result is accomplished, or provides no description of the mechanism for accomplishing the result. Accordingly, this additional element does not add significantly more than the judicial exception. (See MPEP 2106.05(f)).
Accordingly, at Step 2B after considering all claim elements individually and as an ordered combination, it is determined that the claims do not integrate the judicial exception into a practical application.
Regarding Claim 2
Step 2A, Prong 2
Regarding the “wherein the received plurality of multi-turn user-agent interaction records comprise a first data format indicating a first user-agent interaction to complete a first type of task in a first environment and a second data format indicating a second user-agent interaction to complete a second type of task in a second environment” limitation, such limitation merely describes aspects of the multi-turn user-agent interaction records, and therefore such limitation amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not integrate a judicial exception into a practical application.
Step 2B
Regarding the “wherein the received plurality of multi-turn user-agent interaction records comprise a first data format indicating a first user-agent interaction to complete a first type of task in a first environment and a second data format indicating a second user-agent interaction to complete a second type of task in a second environment” limitation, such limitation amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use as explained above, which does not amount to significantly more than the judicial exception. MPEP 2106.05(h).
Regarding Claim 3
Step 2A, Prong 1
parsing a multi-turn user-agent interaction record into a plurality of turns of interactions; (under the broadest reasonable interpretation, a human can mentally review and parse the multi-turn user-agent interaction record into a plurality of turns, such as by writing down the individual turns on a piece of paper)
extracting, for each turn of interactions, an interaction input indicating a context for the respective turn of interactions, an interaction output indicating an agent action executed at the respective turn of interactions and an observation indicating a response from an environment at which the agent action is executed, from the multi-turn user-agent interaction record; (under the broadest reasonable interpretation, a human can mentally extract the input, output, and observation information recited herein, such as by writing such information down in a tuple on a piece of paper)
aggregating turns of interaction inputs, interaction outputs and observations in the homogeneous data format. (under the broadest reasonable interpretation, a human can mentally or using pencil and paper, write down the inputs/outputs/observations in a particular homogeneous data format)
Regarding Step 2A, Prong 2, the claim does not include any additional elements that integrate the judicial exception into a practical application and regarding Step 2B, there are no additional elements recited that amount to significantly more than the judicial exception.
Regarding Claim 4
Step 2A, Prong 2
Regarding the “generating, by a neural network based language model, the plurality of agent trajectories using a prompt describing the homogeneous data format” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception. In particular, the claim only recites the additional element of a neural network utilizing a prompt. This additional element is recited at a high-level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component (a neural network utilizing a prompt). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f)).
Step 2B
Regarding the “generating, by a neural network based language model, the plurality of agent trajectories using a prompt describing the homogeneous data format” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, because the limitation merely provides instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not add significantly more than the judicial exception. (See MPEP 2106.05(f)).
Regarding Claim 5
Step 2A, Prong 2
Regarding the “wherein the neural network rating model is trained with a dataset of agent trajectories and annotated ratings” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception. In particular, the claim only recites the additional element of generic neural network training. This additional element is recited at a high-level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component (generic neural network training). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f)).
Step 2B
Regarding the “wherein the neural network rating model is trained with a dataset of agent trajectories and annotated ratings” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, because the limitation merely provides instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not add significantly more than the judicial exception. (See MPEP 2106.05(f)).
Regarding Claim 6
Step 2A, Prong 2
Regarding the “wherein the neural network rating model is a large language model external to the server and accessible via an application programming interface (API)” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception. In particular, the claim only recites the additional element of a generic neural network model being accessed via an API. This additional element is recited at a high-level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component (a generic neural network model being accessed via an API). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f)).
Regarding the “wherein the generating, by the neural network rating model, the plurality of rating scores comprises: generating a prompt input for the large language model, comprising at least one agent trajectory to be rated, and an instruction on evaluating the at least one agent trajectory” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception. In particular, the claim only recites the additional element of a neural network utilizing a prompt. This additional element is recited at a high-level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component (a neural network utilizing a prompt). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does
Step 2B
Regarding the “wherein the neural network rating model is a large language model external to the server and accessible via an application programming interface (API)” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, because the limitation merely provides instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not add significantly more than the judicial exception. (See MPEP 2106.05(f)).
Regarding the “wherein the generating, by the neural network rating model, the plurality of rating scores comprises: generating a prompt input for the large language model, comprising at least one agent trajectory to be rated, and an instruction on evaluating the at least one agent trajectory” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, because the limitation merely provides instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not add significantly more than the judicial exception. (See MPEP 2106.05(f)).
Regarding Claim 7
Step 2A, Prong 1
wherein the training dataset is aggregated by filtering agent trajectories having rating scores lower than a threshold and shuffling remaining agent trajectories randomly. (under the broadest reasonable interpretation, a human can mentally remove agent trajectories having a score below a threshold from consideration, and then using a random number generator such as a coin, which is a physical aid, determining the order of a training dataset by shuffling the agent trajectories having a sufficiently high score)
Regarding Step 2A, Prong 2, the claim does not include any additional elements that integrate the judicial exception into a practical application and regarding Step 2B, there are no additional elements recited that amount to significantly more than the judicial exception.
Regarding Claim 8
Step 2A, Prong 2
Regarding the “wherein the training dataset comprises agent trajectories originated in different environments from the different data sources” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception. In particular, the claim only recites the additional element of generic neural network training. This additional element is recited at a high-level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component (generic neural network training). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f)).
Step 2B
Regarding the “wherein the training dataset comprises agent trajectories originated in different environments from the different data sources” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, because the limitation merely provides instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not add significantly more than the judicial exception. (See MPEP 2106.05(f)).
Regarding Claim 9
Step 2A, Prong 1
generating, …respective agent actions in response to different user queries from the different user environments. (under the broadest reasonable interpretation, a human can mentally generate different actions that an agent should respond to different user queries from different user environments, such as mentally determining that an agent should provide an informational response due to a user query for information from a mobile phone, or that the agent should provide driving directions due to the environment of the user’s device being in an automobile)
Step 2A, Prong 2
Regarding the “by the trained neural network based model” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception. In particular, the claim only recites the additional element of generic neural network training. This additional element is recited at a high-level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component (generic neural network training). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f)).
Step 2B
Regarding the “by the trained neural network based model” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, because the limitation merely provides instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not add significantly more than the judicial exception. (See MPEP 2106.05(f)).
Regarding Claim 10
Step 2A, Prong 1
Claim 10 recites a system that corresponds to the method of claim 1, and therefore the analysis under Step 2A, Prong 1 with respect to claim 1 also applies to this claim 10.
Step 2A, Prong 2
Claim 10 recites a system that corresponds to the method of claim 1, and therefore the analysis under Step 2A, Prong 2 with respect to claim 1 also applies to this claim 10. While claim 10 recites additional generic computing components (“memory”, “processors”, “instructions”), such additional generic computing components do not change the analysis under Step 2A, Prong 2. These additional elements are recited at a high-level of generality and amount to no more than mere instructions to apply the exception using generic computer components. Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f)).
Step 2B
Claim 10 recites a system that corresponds to the method of claim 1, and therefore the analysis under Step 2B with respect to claim 1 also applies to this claim 10. While claim 10 recites additional generic computing components (“memory”, “processors”, “instructions”), such additional generic computing components, such additional generic computing components do not change the analysis under Step 2B because the limitations merely provide instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not add significantly more than the judicial exception. (See MPEP 2106.05(f)).
Claims 11-18 depend from claim 10 and correspond to the methods of claims 2-9, respectively, and therefore are each rejected for the same reasons explained above with respect to claim 10 and claims 2-9, respectively.
Regarding Claim 19
Step 2A, Prong 1
Claim 19 recites a non-transitory processor-readable storage medium that corresponds to the method of claim 1, and therefore the analysis under Step 2A, Prong 1 with respect to claim 1 also applies to this claim 19.
Step 2A, Prong 2
Claim 19 recites a non-transitory processor-readable storage medium that corresponds to the method of claim 1, and therefore the analysis under Step 2A, Prong 2 with respect to claim 1 also applies to this claim 19. While claim 19 recites additional generic computing components (“non-transitory processor-readable storage medium”, “processors”, “instructions”), such additional generic computing components do not change the analysis under Step 2A, Prong 2. These additional elements are recited at a high-level of generality and amount to no more than mere instructions to apply the exception using generic computer components. Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f)).
Step 2B
Claim 19 recites a non-transitory processor-readable storage medium that corresponds to the method of claim 1, and therefore the analysis under Step 2B with respect to claim 1 also applies to this claim 19. While claim 19 recites additional generic computing components (“non-transitory processor-readable storage medium”, “processors”, “instructions”), such additional generic computing components, such additional generic computing components do not change the analysis under Step 2B because the limitations merely provide instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not add significantly more than the judicial exception. (See MPEP 2106.05(f)).
Claim 20 depends from claim 19 and corresponds to the method of claim 3 and is therefore rejected for the same reasons explained above with respect to claims 3 and 19.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-2, 6, 8-11, 15, and 17-19 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang, Jianguo, et al. "DialogStudio: Towards richest and most diverse unified dataset collection for conversational AI." arXiv:2307.10172v3 (Feb. 5, 2024), in view of Chen, Lichang, et al. "Alpagasus: Training a better alpaca with fewer data." arXiv:2307.08701v4 (Nov. 4, 2023), hereinafter referenced as CHEN, and further in view of US 20250181846 A1, hereinafter referenced as HAN.
Regarding Claim 1
ZHANG teaches:
A method of training an intelligent agent for carrying out actions in response to user interactions, the method comprising: (ZHANG, p. 2, section 1: “We train conversational AI models based on DialogStudio, and these models have demonstrated superior performance over strong baselines in both zero-shot and few-shot learning scenarios”;
Examiner’s Note: providing a response to a query is included in the broadest reasonable interpretation of “carrying out actions in response to user interactions” because determining and providing a written response can be considered an action that is carried out)
receiving, … a plurality of multi-turn user-agent interaction records corresponding to a plurality of different formats; (ZHANG, p. 3, section 3: “We collect and process a wide range of datasets, involving different domains, types, and tasks”;
ZHANG, p. 4, section 3.1: “Before unifying the format of those datasets, web fixed several issues as follows: 1) we remove those dialogues labeled as multi-turn dialogues, but actually
with only one turn and miss either user utterance or system utterance. 2) We manually check the individual dialogues.”;
ZHANG, p. 6, section 4.1: “Here, a dialogue turn refers to a pair consisting of a dialogue context and its corresponding system response. The rest datasets in DialogStudio are preserved for future evaluations and downstream fine-tuning. … We use the format Instruction \n <USER> user utterance <SYSTEM> system response <USER> ... <USER> user utterance \n <EXTERNAL KNOWLEDGE> supported knowledge to train the model, where <USER>, <SYSTEM> and <EXTERNAL KNOWLEDGE> are special tokens.;
Examiner’s Note: ZHANG discloses unifying several different multi-turn dialogue datasets, where the dialogue can refer to a user interaction’s and a system’s responses (corresponding to recited “user-agent interaction records”))
converting the plurality of multi-turn user-agent interaction records into a plurality of agent trajectories having a homogeneous data format; (ZHANG, p. 4, section 3.1: “Before unifying the format of those datasets, web fixed several issues as follows: 1) we remove those dialogues labeled as multi-turn dialogues, but actually with only one turn and miss either user utterance or system utterance. 2) We manually check the individual dialogues. … Next, we construct a uniform JSON dictionary
format to store all relevant information of each dialogue as illustrated in Figure 3.”;
Examiner’s Note: the broadest reasonable interpretation of “agent trajectories” includes a multi-turn data format representing a user-agent interaction record as explained in para. 0017 to the instant specification; ZHANG, Fig. 3, shows a multi-turn data format (in JSON) representing an interaction between a user and a system’s responses)
aggregating a training dataset based on the plurality of agent trajectories …; (ZHANG, p. 2, section 1: “We train conversational AI models based on DialogStudio, and these models have demonstrated superior performance over strong baselines in both zero-shot and few-shot learning scenarios”;
ZHANG, p. 8, section 5: “In this study, we have introduced DialogStudio, a comprehensive collection that aggregates more than 80 diverse dialogue datasets while preserving their original information. This aggregation not only represents a significant leap towards consolidating dialogues from varied sources but also offers a rich tapestry of conversational patterns, intents, and structures, capturing the nuances and richness of human interaction.”)
training a neural network based language models to generate a predicted agent action in response to a user query using the raining dataset (ZHANG, p. 2, section 1: “We train conversational AI models based on DialogStudio, and these models have demonstrated superior performance over strong baselines in both zero-shot and few-shot learning scenarios”;
ZHANG, p. 7, section 4.2: “We follow the prompt from (Bang et al., 2023) to instruct models, i.e., Continue the dialogue as a task-oriented dialogue system called
SYSTEM. The answer of SYSTEM should follow the ACTION provided next while answering the USER’s last utterance.”)
However, ZHANG fails to explicitly teach:
via a communication interface at a server and from a plurality of data source servers,
generating, by a neural network rating model, a plurality of rating scores corresponding to the plurality of agent trajectories;
… and the plurality of rating scores
deploying the trained neural network based model on a hardware platform as an agent application for handling different user queries in different user environments.
However, in a related field of endeavor, using large language models (see p. 1, section 1), CHEN teaches and makes obvious:
generating, by a neural network rating model, a plurality of rating scores corresponding to the plurality of agent trajectories; (CHEN, p. 2, section 1: “Specifically, we design a prompt applied to a powerful LLM (e.g., ChatGPT) for evaluating the quality of each (instruction, input, response) tuple and then filter out the ones with scores lower than a threshold. By applying this filter to the 52k data used to train ALPACA, we find that a majority of the data suffer from low-quality issues. Using the LLM filter, IFT on a much smaller but carefully filtered subset of 9k data produces a much better model, i.e., ALPAGASUS, than the original ALPACA, as shown in Fig. 2, following exactly the same training configuration of ALPACA.”;
Examiner’s Note: CHEN teaches using a LLM to score each of a plurality of standardized tuples; the ZHANG-CHEN combination now applies the LLM scoring to the DialogStudio set of ZHANG, where now an LLM is used to evaluate and score each of the unified JSON-formatted multi-turn user-system dialogues of ZHANG)
aggregating a training dataset based on the plurality of agent trajectories and the plurality of rating scores (CHEN, p. 2, section 1: “Specifically, we design a prompt applied to a powerful LLM (e.g., ChatGPT) for evaluating the quality of each (instruction, input, response) tuple and then filter out the ones with scores lower than a threshold. By applying this filter to the 52k data used to train ALPACA, we find that a majority of the data suffer from low-quality issues. Using the LLM filter, IFT on a much smaller but carefully filtered subset of 9k data produces a much better model, i.e., ALPAGASUS, than the original ALPACA, as shown in Fig. 2, following exactly the same training configuration of ALPACA.”;
Examiner’s Note: CHEN teaches using a LLM to score each of a plurality of standardized tuples; the ZHANG-CHEN combination now applies the LLM scoring to the DialogStudio set of ZHANG, where now an LLM is used to evaluate and score each of the unified JSON-formatted multi-turn user-system dialogues of ZHANG and to store such scores along with such JSON-formatted structures)
Before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the combine the teachings of ZHANG with CHEN as explained above. As disclosed by CHEN, one of ordinary skill would have been motivated to do so because removing lower-quality training samples can provide a “much better model” band also “reduce[] the training time.” (p. 2, section 1).
However, ZHANG and CHEN fail to explicitly teach:
via a communication interface at a server and from a plurality of data source servers,
deploying the trained neural network based model on a hardware platform as an agent application for handling different user queries in different user environments.
However, in a related field of endeavor (communications systems with respect to LLMs, see para. 0009), HAN teaches and makes obvious:
via a communication interface at a server and from a plurality of data source servers, (HAN, para. 0015: “The computing system 105 may include one or more servers 125 and may provide (e.g., to the one or more computing devices 115) local or remote access to applications, databases, or files stored within the computing system 105. The computing system 105 may further include one or more data storage devices 130. Though one server 125 and one data storage device 130 are shown in FIG. 1, it is to be understood that the computing system 105 may include any quantity of servers 125 and any quantity of data storage devices 130, which may be in communication with one another and collectively perform one or more functions ascribed herein to the server 125 and data storage device 130.”;
HAN, para. 0018: “A server 125 may include a network interface 140, processor 145, memory 150, disk 155, and computing system manager 160. The network interface 140 may enable the server 125 to connect to and exchange information via the network 120 (e.g., using one or more network protocols).”;
Examiner’s Note: the ZHANG-CHEN-HAN combination now uses the server 125 and network interface 140 of HAN run the LLM of ZHANG, and to collect training sets from disparate sources (as taught by ZHANG) using the server and communication network of HAN)
deploying the trained neural network based model on a hardware platform as an agent application for handling different user queries in different user environments. (HAN, para. 0014: “A computing device 115 may be a stationary device (e.g., a desktop computer or access point) or a mobile device (e.g., a laptop computer, tablet computer, or cellular phone). In some examples, a computing device 115 may be a commercial computing device, such as a server or collection of servers. And in some examples, a computing device 115 may be a virtual device (e.g., a virtual machine). Though shown as a separate device in the example computing environment of FIG. 1, it is to be understood that in some cases a computing device 115 may be included in (e.g., may be a component of) the computing system 105 or the DMS 110.”;
HAN, para. 0040: “In the computing environment 200, a communication service 205 of the DMS 110-a may establish a communication session 210 with a user of the computing device 115-a (e.g., via a user interface of the computing device 115-a) and may use an LLM 235 to handle/process queries 215 received from the user. For example, the communication service 205 may generate a prompt 230 based on a query 215. The communication service 205 may transmit the prompt 230 to the LLM 235, which may return a response 240 to the prompt 230. The communication service 205 may provide a message 245 to the user (e.g., displayed on a user interface of the computing device 115-a as part of the communication session 210) based on the response 240. The communication service 205 may also be referred to as a communication manager or a chat manager.”
Examiner’s Note: the ZHANG-CHEN-HAN combination now deploys the conversational AI models of ZHANG in the computing environment of HAN, which includes computer hardware, so that the AI models can handle different user queries from different user environments (e.g., mobile or desktop) as taught by HAN)
Before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the combine the teachings of ZHANG with CHEN and HAN as explained above. As disclosed by HAN, one of ordinary skill would have been motivated to do so in order to “support techniques for session handlers for AI communications, which may provide one or more benefits such as, for example, improved reliability, improved user experience, more efficient utilization of computing resources, network resources or both, improved scalability, and/or improved security, among other possibilities.” (para. 0099).
Regarding Claim 2
ZHANG, CHEN, and HAN teach the method of claim 1 as explained above. ZHANG further teaches:
wherein the received plurality of multi-turn user-agent interaction records comprise a first data format indicating a first user-agent interaction to complete a first type of task in a first environment and a second data format indicating a second user-agent interaction to complete a second type of task in a second environment. (ZHANG, p. 3, section 2.1: “Figure 1a presents an overview of DialogStudio’s dataset categories. Note that the category boundaries are fuzzy as some datasets span multiple categories.”
ZHANG, p. 3, section 3: “We collect and process a wide range of datasets, involving different domains, types, and tasks”;
ZHANG, p. 4, section 3.1: “Before unifying the format of those datasets, web fixed several issues as follows: 1) we remove those dialogues labeled as multi-turn dialogues, but actually
with only one turn and miss either user utterance or system utterance. 2) We manually check the individual dialogues.”;
Examiner’s Note: As shown in Fig. 1a, for example, a first data format can be in the MulDoGo task-ordered dialog format and the second data format can be for the TOP natural language understanding unit format).
Regarding Claim 6
ZHANG, CHEN, and HAN teach the method of claim 1 as explained above. However, ZHANG fails to explicitly teach:
wherein the neural network rating model is a large language model external to the server and accessible via an application programming interface (API), and
wherein the generating, by the neural network rating model, the plurality of rating scores comprises: generating a prompt input for the large language model, comprising at least one agent trajectory to be rated, and an instruction on evaluating the at least one agent trajectory.
However, in a related field of endeavor, using large language models (see p. 1, section 1), CHEN teaches and makes obvious:
wherein the generating, by the neural network rating model, the plurality of rating scores comprises: generating a prompt input for the large language model, comprising at least one agent trajectory to be rated, and an instruction on evaluating the at least one agent trajectory. (CHEN, p. 2, section 1: “Specifically, we design a prompt applied to a powerful LLM (e.g., ChatGPT) for evaluating the quality of each (instruction, input, response) tuple and then filter out the ones with scores lower than a threshold. By applying this filter to the 52k data used to train ALPACA, we find that a majority of the data suffer from low-quality issues. Using the LLM filter, IFT on a much smaller but carefully filtered subset of 9k data produces a much better model, i.e., ALPAGASUS, than the original ALPACA, as shown in Fig. 2, following exactly the same training configuration of ALPACA.”;
Examiner’s Note: CHEN teaches using a prompt to an LLM to score each of a plurality of standardized tuples; the ZHANG-CHEN-HAN combination now applies the LLM scoring to the DialogStudio set of ZHANG, where now a prompt to an LLM is used to evaluate and score each of the unified JSON-formatted multi-turn user-system dialogues of ZHANG, where as shown in Fig. 4 of CHEN, the system prompt includes the particular formatted-information to be scored)
Before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the combine the teachings of ZHANG with CHEN and HAN as explained above. As disclosed by CHEN, one of ordinary skill would have been motivated to do so because removing lower-quality training samples can provide a “much better model” band also “reduce[] the training time.” (p. 2, section 1).
However, ZHANG and CHEN fail to explicitly teach:
wherein the neural network rating model is a large language model external to the server and accessible via an application programming interface (API),
However, in a related field of endeavor (communications systems with respect to LLMs, see para. 0009), HAN teaches and makes obvious:
wherein the neural network rating model is a large language model external to the server and accessible via an application programming interface (API), (HAN, para. 0015: “Though one server 125 and one data storage device 130 are shown in FIG. 1, it is to be understood that the computing system 105 may include any quantity of servers 125 and any quantity of data storage devices 130, which may be in communication with one another and collectively perform one or more functions ascribed herein to the server 125 and data storage device 130.”
HAN, para. 0047: “Accordingly, the communication service 205 may manage communication with the LLM 235 (e.g., API calls to transmit the prompts 230 and receive the responses 240), communication with the computing device 115-a (e.g., formatting the messages 245, receiving the queries 215, and associated API calls), and storage and retrieval of the chat history in the database 250. As the communication service 205 may handle the chat initiation and the chat continuation functions, which may call the APIs to communicate with the computing device 115-a and the LLM 235”;
Examiner’s Note: the ZHANG-CHEN-HAN combination now has the LLM performing ratings (as in CHEN) to be a separate server that is external to server 125 of HAN, and accesses such LLM through an API as disclosed by HAN)
Before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the combine the teachings of ZHANG with CHEN and HAN as explained above. As disclosed by HAN, one of ordinary skill would have been motivated to do so in order to “support techniques for session handlers for AI communications, which may provide one or more benefits such as, for example, improved reliability, improved user experience, more efficient utilization of computing resources, network resources or both, improved scalability, and/or improved security, among other possibilities.” (para. 0099).
Regarding Claim 8
ZHANG, CHEN, and HAN teach the method of claim 1 as explained above. ZHANG further teaches:
wherein the training dataset comprises agent trajectories originated in different environments from the different data sources. (ZHANG, p. 3, section 2.1: “Figure 1a presents an overview of DialogStudio’s dataset categories. Note that the category boundaries are fuzzy as some datasets span multiple categories.”
ZHANG, p. 3, section 3: “We collect and process a wide range of datasets, involving different domains, types, and tasks”;
ZHANG, p. 4, section 3.1: “Before unifying the format of those datasets, web fixed several issues as follows: 1) we remove those dialogues labeled as multi-turn dialogues, but actually
with only one turn and miss either user utterance or system utterance. 2) We manually check the individual dialogues.”;
ZHANG, p. 8, section 5: “In this study, we have introduced DialogStudio, a comprehensive collection that aggregates more than 80 diverse dialogue datasets while preserving their original information. This aggregation not only represents a significant leap towards consolidating dialogues from varied sources but also offers a rich tapestry of conversational patterns, intents, and structures, capturing the nuances and richness of human interaction.”;
Examiner’s Note: DialogStudio aggregates more than 80 diverse dialog sets, where a common format is used to unify the different data sources (e.g., each of the 80 diverse dialog datasets) relating to different environments as shown in Fig. 1b)
Regarding Claim 9
ZHANG, CHEN, and HAN teach the method of claim 1 as explained above. ZHANG further teaches:
generating, by the trained neural network based model, respective agent actions in response to different user queries from the different user environments. (ZHANG, p. 2, section 1: “We train conversational AI models based on DialogStudio, and these models have demonstrated superior performance over strong baselines in both zero-shot and few-shot learning scenarios”;
ZHANG, p. 7, section 4.2: “We follow the prompt from (Bang et al., 2023) to instruct models, i.e., Continue the dialogue as a task-oriented dialogue system called
SYSTEM. The answer of SYSTEM should follow the ACTION provided next while answering the USER’s last utterance.”)
Regarding Claim 10
ZHANG teaches:
A system … comprising a memory storing a plurality of processor-executable instructions; and one or more processors executing the plurality of processor-executable instructions to perform operations comprising (ZHANG, p. 6, section 4.1: “Experiments are conducted using 16 A100 GPUs, each with 40GB of GPU memory.”)
The remaining limitations correspond to the method of claim 1, and are therefore rejected for the same reasons explained below with respect to claim 1.
Claim 11 depends from claim 10 and recites a system that corresponds to the method of claim 2, and is therefore rejected for the same reasons explained above with respect to claims 2 and 10.
Claim 15 depends from claim 10 and recites a system that corresponds to the method of claim 6, and is therefore rejected for the same reasons explained above with respect to claims 6 and 10.
Claim 17 depends from claim 10 and recites a system that corresponds to the method of claim 8, and is therefore rejected for the same reasons explained above with respect to claims 8 and 10.
Claim 18 depends from claim 10 and recites a system that corresponds to the method of claim 9, and is therefore rejected for the same reasons explained above with respect to claims 9 and 10.
Regarding Claim 19
ZHANG teaches:
A non-transitory processor-readable storage medium storing a plurality of processor-executable instructions for (ZHANG, p. 6, section 4.1: “Experiments are conducted using 16 A100 GPUs, each with 40GB of GPU memory.”)
The remaining limitations correspond to the method of claim 1, and are therefore rejected for the same reasons explained below with respect to claim 1.
Claims 3, 12, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over ZHANG in view of CHEN and HAN and further in view of US 20250013915 A1, hereinafter referenced as PARK.
Regarding Claim 3
ZHANG, CHEN, and HAN teach the method of claim 1 as explained above. ZHANG further teaches:
parsing a multi-turn user-agent interaction record into a plurality of turns of interactions; (ZHANG, p. 5, see Figure 3;
ZHANG, p. 6, section 4.1: “For each dialogue dataset, we sample at most 11k dialogues. Additionally, we extracted around 11k dialogue turns from question-answering dialogues featured in RACE”;
Examiner’s Note: As shown in Fig. 3, the records from the datasets are converted to a particular DialogStudio data format, where there is a “turn id” flag, such that different turn IDs are labeled differently)
extracting, for each turn of interactions, an interaction input indicating a context for the respective turn of interactions, an interaction output indicating an agent action executed at the respective turn of interactions …, from the multi-turn user-agent interaction record; (ZHANG, p. 5, see Figure 3;
ZHANG, p. 6, section 4.1: “For each dialogue dataset, we sample at most 11k dialogues. Additionally, we extracted around 11k dialogue turns from question-answering dialogues featured in RACE”;
Examiner’s Note: As shown in Fig. 3, the records from the datasets are converted to a particular DialogStudio data format, and there is a “user utterance” field showing the user’s utterance (corresponding to recited “context for the respective turn of interactions”) and a “system response” field showing the systems’ response (corresponding to recited “indicating an agent action executed at the respective turn of interactions))
aggregating turns of interaction inputs, interaction outputs … in the homogeneous data format. (ZHANG, p. 5, see Figure 3;
ZHANG, p. 6, section 4.1: “For each dialogue dataset, we sample at most 11k dialogues. Additionally, we extracted around 11k dialogue turns from question-answering dialogues featured in RACE”;
Examiner’s Note: As shown in Fig. 3, a single DialogStudio data record can include multiple inputs and system outputs and turns)
However, ZHANG, CHEN, and HAN fail to explicitly teach:
and an observation indicating a response from an environment at which the agent action is executed (
… and observations …
However, in a related field of endeavor (machine learning reward models, see paras. 0004-0005), PARK teaches and makes obvious:
and an observation indicating a response from an environment at which the agent action is executed (PARK, para. 0087: “FIG. 4 illustrates an example implementation of reward model 130 in an iterative processing flow. Input 102 can include a sequence of state features 404-1, 404-2, . . . , etc. Agent model 110 can process input 102 to output an action feature 412. Action feature 412 can specify an action to be performed by a tool 420 to generate a result feature 422.”;
PARK, para. 0091: “Result feature 422 can include substantially any output of a tool 420. This can be termed an “observation,” such as an “observation” of the external world external to agent model 110.”
Examiner’s Note: the ZHANG-CHEN-HAN-PARK combination now stores the result feature 44 of PARK, corresponding to an “observation of the external world” in the JSON-formatted unifying data structure of ZHANG)
… and observations …(PARK, para. 0087: “FIG. 4 illustrates an example implementation of reward model 130 in an iterative processing flow. Input 102 can include a sequence of state features 404-1, 404-2, . . . , etc. Agent model 110 can process input 102 to output an action feature 412. Action feature 412 can specify an action to be performed by a tool 420 to generate a result feature 422.”;
PARK, para. 0091: “Result feature 422 can include substantially any output of a tool 420. This can be termed an “observation,” such as an “observation” of the external world external to agent model 110.”
Examiner’s Note: the ZHANG-CHEN-HAN-PARK combination now stores the result feature 44 of PARK, corresponding to an “observation of the external world” in the JSON-formatted unifying data structure of ZHANG)
Before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the combine the teachings of ZHANG with CHEN, HAN, and PARK as explained above. As disclosed by PARK one of ordinary skill would have been motivated to do so because PARK teaches techniques to provide “lower-cost training as compared to surfacing and presenting inputs and outputs for human review. (para. 0029).
Claim 12 depends from claim 10 and recites a system that corresponds to the method of claim 3, and is therefore rejected for the same reasons explained above with respect to claims 3 and 10.
Claim 20 depends from claim 19 and recites a non-transitory processor-readable storage medium that corresponds to the method of claim 3, and is therefore rejected for the same reasons explained above with respect to claims 3 and 10.
Claims 4 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over ZHANG in view of CHEN and HAN and further in view of US 20250252246 A1, hereinafter referenced as MATURANA.
Regarding Claim 4
ZHANG, CHEN, and HAN teach the method of claim 1 as explained above. However, ZHANG, CHEN, and HAN fail to explicitly teach:
generating, by a neural network based language model, the plurality of agent trajectories using a prompt describing the homogeneous data format.
However, in a related field of endeavor (prompts for generative AI, see paras. 0004-0005), MATURANA teaches and makes obvious:
generating, by a neural network based language model, the plurality of agent trajectories using a prompt describing the homogeneous data format. (MATURANA, para. 0076: “The conversion services 206 translate these discovered document elements to a hierarchical model 208 having a tree-like structure that conforms to a relevant industrial standard such as ISA-88. As will be described in more detail herein, the conversion services 206 can leverage generative AI in connection with extracting, digitizing, and organizing content of the document 204 into the structured format of the hierarchical model 208. This can involve the use of a specialized prompt engineering layer and associated custom models—trained using knowledge of various types industrial control applications, knowledge of specific industrial verticals, vertical-specific industrial standards and best practices, global and customer-specific semantic rules, and other such training data—that can generate prompts or meta-prompts for submission to generative AI models such as large language models (LLMs).”;
Examiner’s Note: the ZHANG-CHEN-HAN-MATURANA combination now uses the prompting + generative AI of MATURANA to create the JSON-formatted DigitalStudio data samples of ZHANG)
Before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the combine the teachings of ZHANG with CHEN, HAN, and MATURANA as explained above. As disclosed by MATURANA one of ordinary skill would have been motivated to do so because MATURANA teaches using generative AI in order to reduce or eliminate the need for custom-built software, such as parsers. (para. 0095). One of ordinary skill would understand that known generative AI technologies are becoming more accepted with respect to parsing data and data conversions.
Claim 13 depends from claim 10 and recites a system that corresponds to the method of claim 4, and is therefore rejected for the same reasons explained above with respect to claims 4 and 10.
Claims 5 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over ZHANG in view of CHEN and HAN and further in view of US 20250061323 A1, hereinafter referenced as NATH.
Regarding Claim 5
ZHANG, CHEN, and HAN teach the method of claim 1 as explained above. However, ZHANG, CHEN, and HAN fail to explicitly teach:
wherein the neural network rating model is trained with a dataset of agent trajectories and annotated ratings.
However, in a related field of endeavor (neural network training, see para. 0003), NATH teaches and makes obvious:
wherein the neural network rating model is trained with a dataset of agent trajectories and annotated ratings. (NATH, para. 0069: “In at least one embodiment, a data sample from among data with labels of unknown quality 206 upon which inferencing was performed by neural network 202 to yield prediction 208 is ranked 214 to determine whether this data sample will be added to training data 204, based on a corresponding annotation score 210 and/or confidence score 212.”;
Examiner’s Note: the ZHANG-CHEN-HAN-NATH combination now uses the annotation score 210 when training the model for providing ratings (as taught by CHEN), in addition to the DigitalStudio teachings of ZHANG)
Before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the combine the teachings of ZHANG with CHEN, HAN, and NATH as explained above. As disclosed by NATH one of ordinary skill would have been motivated to do so because NATH teaches annotated scores to promote correct data labeling, which one of ordinary skill would understand would lead to a better trained model. (para. 0067).
Claim 14 depends from claim 10 and recites a system that corresponds to the method of claim 5, and is therefore rejected for the same reasons explained above with respect to claims 5 and 10.
Claim 7 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over ZHANG in view of CHEN and HAN and further in view of Lowe, Ryan, et al. "The ubuntu dialogue corpus: A large dataset for research in unstructured multi-turn dialogue systems." Proceedings of the 16th annual meeting of the special interest group on discourse and dialogue. 2015, hereinafter referenced as LOWE.
Regarding Claim 7
ZHANG, CHEN, and HAN teach the method of claim 1 as explained above. However, ZHANG fails to explicitly teach:
wherein the training dataset is aggregated by filtering agent trajectories having rating scores lower than a threshold and shuffling remaining agent trajectories randomly.
However, in a related field of endeavor, using large language models (see p. 1, section 1), CHEN teaches and makes obvious:
wherein the training dataset is aggregated by filtering agent trajectories having rating scores lower than a threshold … (CHEN, p. 2, section 1: “Specifically, we design a prompt applied to a powerful LLM (e.g., ChatGPT) for evaluating the quality of each (instruction, input, response) tuple and then filter out the ones with scores lower than a threshold.”
Examiner’s Note: CHEN teaches using a LLM to score each of a plurality of standardized tuples and filtering out samples lower than a threshold; the ZHANG-CHEN-HAN combination now applies the LLM scoring to the DialogStudio set of ZHANG, where now an LLM is used to evaluate and score each of the unified JSON-formatted multi-turn user-system dialogues of ZHANG, and now scores below a threshold are associated with samples that will not be included in the DialogStudio set of ZHANG)
Before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the combine the teachings of ZHANG with CHEN and HAN as explained above. As disclosed by CHEN, one of ordinary skill would have been motivated to do so because removing lower-quality training samples can provide a “much better model” band also “reduce[] the training time.” (p. 2, section 1).
However, ZHANG, CHEN, and HAN fail to explicitly teach:
shuffling remaining agent trajectories randomly.
However, in a related field of endeavor (multi-turn dialogue systems), LOWE teaches and makes obvious:
shuffling remaining agent trajectories randomly. (LOWE, p. 290, section 4: “So a dialogue of length 10 yields 8 training examples. Since these are overlapping, they are clearly not independent, but we consider this a minor issue given the size of the dataset (we further alleviate the issue by shuffling the training examples). Negative responses are selected at random from the rest of the training data.”;
Examiner’s Note: the ZHANG-CHEN-HAN-LOWE combination randomly shuffles training examples in DigitalStudio as in LOWE)
Before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the combine the teachings of ZHANG with CHEN, HAN, and LOWE as explained above. As disclosed by LOWE, one of ordinary skill would have been motivated to do so order to alleviate issues with sample interdependence. (p. 290, section 4).
Claim 16 depends from claim 10 and recites a system that corresponds to the method of claim 7, and is therefore rejected for the same reasons explained above with respect to claims 7 and 10.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Zhang, Chen, et al. "DynaEval: Unifying turn and dialogue level evaluation." Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021. Discloses techniques for mapping individual utterances in a multi-turn dialog into a particular vector space. (p. 5678, section 3).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL C LEE whose telephone number is (571)272-4933. The examiner can normally be reached M-F 12:00 pm - 8:00 pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Omar Fernandez Rivas can be reached at 571-272-2589. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL C. LEE/Examiner, Art Unit 2128
Prosecution Insights