Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
2. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
3. Claims 1-4, 7-8 and 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over Guo et al “CONNECTING LARGE LANGUAGE MODELS WITH EVOLUTIONARY ALGORITHMS YIELDS POWERFUL PROMPT OPTIMIZERS,” herein Guo, in view of Massiceti et al. (US 2024/0104161) herein Massiceti, and further in view of Sample et al (US 2023/0138020), herein Sample.
Regarding Claim 1:
Guo discloses a method comprising:
obtaining a current instruction (Guo: Section 3.1 discloses evolutionary algorithms (EAs) start off with an initial population of n solutions. These initial prompts are analogous to the claimed current instruction);
and for at least two iterations: in a first phase, iteratively:
applying an evolutionary algorithm to the current instruction to generate a revised instruction (Guo: Section 3.1, Algorithm 1 line 4 explicitly applies EA operators (mutations and crossover) to generate revised prompts, therefore teaching applying an evolutionary algorithm to a current prompt (instruction) to generate a revised instruction),
testing, by applying a large language model (LLM), a first prompt comprising the current instruction with a first set of training examples selected (Guo: Algorithm 1 discloses evaluating each prompt on a development set, D and computing a score:
“D, fD(·) denotes the score of a prompt on the desired LLM evaluated on D”
“Initial evaluation scores: S0 ← {si = fD(pi)|i ∈ [1,N]}”
Further, Gao Experimental Settings, shows that each evaluated prompt is used together with one or more demonstration examples (shots) that are prepended to the input for the task, e.g., for classification tasks on example per class is prepended before the test case, therefore Guo teaches testing a first prompt comprising an instruction and training examples by applying an LLM to obtain a first test result),
testing, by applying the LLM, a second prompt comprising the revised instruction (Gao: Algorithm 1, line 5 “Evaluation: s′ i ← f(p′ i,D): teaches generating a revised prompt and evaluating it on the same development set with the LLM to obtain Si. This is analogous to testing a second prompt comprising the revised instruction using the LLM to obtain a second test result.),
comparing the first test result to a second test result to obtain a comparison result (Gao: Section 3.2 and 3.3 discloses comparing scores and retaining the better prompt, i.e., comparing a first and second test result to decide which prompt is better corresponding to the claimed comparison result),
setting the revised instruction as the current instruction (Gao: Section 3.3 discloses that if the revised prompt has the higher score it is kept in the population and becomes part of the current set of prompts for the next iteration),
Guo does not explicitly disclose:
an example selector;
a first set of training examples selected by the example selector;
a second set of training examples selected by the example selector
and in a second phase: selecting, by the example selector, a third set of training examples, testing, using the current instruction, the third set of training examples to obtain a third test result, and modifying, after executing the first phase, the example selector based on the third test result.
However, Massiceti discloses:
an example selector (Massiceti: ¶[0022] and ¶[0042] discloses an example selector that selects training examples from a pool);
a first set of training examples selected by the example selector (Massiceti: ¶[0022] and ¶[0042] discloses and example selector has learned weights the examples are selected by these learned weights);
a second set of training examples selected by the example selector (Massiceti: ¶[0022] and ¶[0042] discloses an example selector has learned weights the examples are selected by learned weights. Reapplying that selector to select examples for evaluation of the revised instruction teaches selecting a second set of training examples by the example selector);
and in a second phase: selecting, by the example selector, a third set of training examples, testing, using the current instruction, the third set of training examples to obtain a third test result, and modifying, after executing the first phase, the example selector based on the third test result (Massiceti: ¶[0022] and ¶[0042] discloses learned weights in an example selector which is then used to select examples from a pool, these correspond to selecting a third set of training examples. ¶[0057] and ¶[0078] disclose using the currently selected examples to adapt and test the model on a query set (a third test result) and modifying the example selector based on that test result).
Guo and Massiceti are combinable because both are directed to improving model performance by iterative optimization using evaluation feedback. Guo optimizes the instruction text for large language models using an evolution algorithm. Massiceti optimizes a set of training examples using constrained optimization algorithms and gradient descent. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate Massiceti’s teaching that learned selection weights over example pools significantly affect model robustness into Guo’s evolutionary algorithm based instruction optimization. The suggestion for doing so is “The inventors have found that by using a projected gradient ascent it is possible to achieve a particularly efficient process which is therefore scalable to large pools of examples” as disclosed in ¶[0025] of Massiceti.
The combination of Guo and Massiceti do not disclose:
exiting the first phase when the comparison result satisfies a first phase stop condition.
However, Sample discloses exiting the first phase when the comparison result satisfies a first phase stop condition (Sample: ¶[0025] and ¶[0028] disclose a comparison result based stop condition, the loop goes until all candidates that do not exceed a fitness threshold are processed).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to substitute Guo’s fixed iteration (present in Algorithm 1, stop condition T) stop condition with Sample’s comparison result based fitness threshold stop condition. Both references are directed to iterative evolutionary algorithm based optimization loops that evaluate and compare candidate solutions. Sample’s threshold based exit is a well-known and standard way to create iterative loops in this field. The suggestion/motivation for doing so is disclosed in Sample, ¶[0010]: “Embodiments of the present invention improve ANN creation and training by leveraging modified selection mechanisms and mutations to simulate ANN evolution, allowing advanced network and data analysis.”
Regarding Claim 2
The combination of Guo, Massiceti and Sample further discloses the method of claim 1, wherein applying the evolutionary algorithm comprises mutating the current instruction to generate the revised instruction (Guo: Section 3.3 Fig. 2 description discloses in step 2, the LLMs perform mutation on the prompt; Section 3.2 discloses the newly generated prompt from the first step undergoes mutation; Algorithm 1 line 4 discloses evolution generate a new prompt based on the selected parent prompts by leveraging an LLM to perform evolutionary operators).
Regarding Claim 3:
The combination of Guo, Massiceti and Sample further discloses the method of claim 1,
wherein the current instruction is in a set of current instructions (Guo: Section 3.1 Algorithm 1 3.1 explicitly discloses the current instruction is one member of a population of N current instructions),
and wherein applying the evolutionary algorithm comprises:
selecting a subset of the set of current instructions (Guo: Section 3.1 Algorithm 1 line 3 discloses selection of a certain number of prompts from the current population as parent prompts, this teaches selecting a subset from the current population of instructions),
and performing a crossover mutation of the subset of the set of current instructions to obtain a set of revised instructions comprising the revised instruction (Guo: Section 3.2 discloses and Fig. 1 teach performing crossover mutation on the selected subset to generate a set of revised instructions),
wherein the set of revised instructions is tested to obtain a set of test results comprising the second test result (Guo: Section 3.1 Algorithm 1 line 5, discloses an evaluation applied to each newly generated prompt in the revised set; Section 3.2 discloses iteratively generating new candidate prompts and assesses each prompt using a development set to obtain a score that quantifies the quality of the prompt. Altogether this discloses each revised instruction in the set is tested to obtain a corresponding test result, yielding a set of test results),
wherein the comparison of the first test result with the second test result is performed using the set of test results (Guo: Section 3.2 discloses the updated population is then selected by retaining the N prompts with the highest scores, and Section 3.3 discloses the prompt with a higher score is retained, this is a pairwise comparison using the set of test results).
Regarding Claim 4:
The combination of Guo, Massiceti and Sample further discloses the method of claim 1, wherein testing the first prompt with the current instruction comprises:
selecting an evaluation input output pair comprising an evaluation input and an evaluation output (Guo: Algorithm 1, the dev set D contains labeled input/output pairs. D is explicitly a set of evaluation input output (text inputs with ground truth labels);
selecting the first set of training examples using the example selector (Guo: Section 4.2, Table 8 explicitly discloses prepending demonstration examples alongside the instruction. Massiceti ¶[0022] and ¶[0042] supplies the learned example selector component discussed in Claim 1 above.);
transmitting, to the LLM, the first prompt with the current instruction, the first set of training examples, and the evaluation input to obtain a first LLM output (Guo: Section 1 discloses a current instruction: “simply adding an instruction to the input text, also known as a type of discrete prompt, steers LLMs to carry out the desired task” the evolved prompt pi is the instruction component of the input to the LLM. The first set of training examples is disclosed in Section 4.2 and Table 8: “we prepend the demonstration consisting of one example per class before the test case.” One labeled demonstration example per class is prepended, these are the training examples fed into the LLM alongside the instruction. Section 3.1 Algorithm 1 and 4.2 discloses “fD(·) denotes the score of a prompt on the desired LLM evaluated on D” and “we prepend the demonstration consisting of one example per class before the test case.” The test case from D is labeled input/output pair from the development set whose label is withheld during the LLM inference, this is the evaluation input transmitted to the LLM. Guo explicitly places the test case after the demonstration examples. Establishing that the instruction, demonstration examples and test case input are all present in the input submitted for evaluation with the ground truth label from D reserved for comparison against the LLM’s predicted output to produce the evaluation score);
and comparing the evaluation output with the first LLM output to obtain the first test result (Guo: Section 4.1 discloses computing a score by comparing the LLM’s output against ground truth).
Guo and Massiceti are combinable because both are directed to improving model performance by optimizing prompt components using evaluation feedback. Guo optimizes the instruction text for large language models using an evolution algorithm. Massiceti optimizes a set of training examples using constrained optimization algorithms and gradient descent. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate Massiceti’s teaching that learned selection weights over example pools significantly affect model robustness into Guo’s evolutionary algorithm based instruction optimization. The suggestion for doing so is: “The inventors have found that by using a projected gradient ascent it is possible to achieve a particularly efficient process which is therefore scalable to large pools of examples” as disclosed in ¶[0025] of Massiceti.
Regarding Claim 7:
The combination of Guo, Massiceti and Sample further discloses the method of claim 1, wherein the first set of training examples and the second set of training examples are a same set of training examples (Guo: Algorithm 1 discloses that the prompts are training examples evaluated on the same development set D, including evaluation of the current prompt and the revised prompt. Further, Guo Experimental Settings discloses that for text classification one example per class is prepended as demonstration examples during evaluation).
Regarding Claim 8:
The combination of Guo, Massiceti and Sample further discloses the method of claim 1, further comprising:
prior to the at least two iterations, randomly selecting an initial set of examples, each example in the initial set of examples comprising an input and a corresponding output (Guo: Section 3.1 discloses an initial population with manual prompts and prompts generated by an LLM),
sending a request to the LLM requesting that the LLM define an instruction that produces the corresponding output from the input for each example in the initial set of examples (Guo: Section 4.1 cites instruction induction for initial prompts. Instruction Induction is the act of providing labels/requests to a model and prompting it to generate a corresponding instruction);
receiving the instruction from the LLM (Guo: Sections 3.1 and 4.1, the inevitable result of the process taught above results in receiving the instruction from the LLM);
and using the instruction as the current instruction (Guo: Sections 3.1 explicitly discloses these instructions are used as current instructions).
Regarding Claim 17:
The combination of Guo, Massiceti, and Sample further disclose the method of claim 1, wherein the first phase is performed over a plurality of iterations before transitioning to the second phase (Guo: Algorithm 1 expressly discloses an iterative optimization loop, “for t = 1 to T do” therefore teaching that the first phase is performed over a plurality of iterations).
Regarding Claim 18:
Claim 18 is rejected for the same reasons set forth with claim 1 because Guo, Massiceti and Sample teach the corresponding system limitation of claim 18 including memory storing instructions and processor executing those instructions to perform the claimed operations at least at Section 4.1 where Guo’s system runs on GPU based computer infrastructure using GPT 3.5/Alpaca APIs, processors and memory are inherent.
Regarding Claim 19:
Claim 19 is rejected for the same reasons set forth with claim 1 because Guo, Massiceti and Sample teach the corresponding system limitation of claim 18. It is noted that Massiceti discloses a production environment at least at ¶[0055]-[0056] where the classifier is stored and deployed.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to deploy the optimized instruction and example selector framework of the combined references in a production environment because Massiceti expressly contemplates real world deployment of the trained system after selection and adaption. The motivation is disclosed in Massiceti’s ¶[0007]: “providing metadata of a stored example to a user or an automated process, providing feedback comprising a selected example”
Regarding Claim 20:
The combination of Guo, Massiceti and Sample further discloses the method of claim 19, further comprising: receiving a user prompt segment from a user device, selecting, by the example selector, a set of examples according to the user prompt segment (Massiceti: ¶[0032], ¶[0034], ¶[0036] and [0039] discloses a client device sends and captures inputs and examples to the example extractor over the network. The example extractor selects examples from a pool using learned weights, the user device provided input is the context in which the selection is performed),
generating an LLM prompt with the user prompt segment, the set of examples, and the current instruction, transmitting the LLM prompt to the LLM, receiving, responsive to the LLM prompt, a LLM result (Guo: Section 3.1 Algorithm 1, Section 4.2 and Appendix teaches that the prompt evaluation input consists of instruction together with examples and the test input, it further teaches applying the prompt to the desired LLM for evaluation),
generating a user result from the LLM result, and transmitting the user result to the user device responsive to the user prompt segment (Massiceti: ¶[0032], ¶[0034, ¶[0036] and ¶[0051] disclose deployed client device and network operation and sending feedback to the user).
Guo and Massiceti are combinable because both are directed to improving model performance by optimizing prompt components using evaluation feedback. Guo optimizes the instruction text for large language models using an evolution algorithm. Massiceti optimizes a set of training examples using constrained optimization algorithms and gradient descent. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use Guo’s instruction and example LLM prompting framework in Massiceti’s deployed client device environment so that user device inputs could be used to selects examples generate an LLM prompt, obtain an LLM result and return a user result to the device because Massiceti expressly contemplates inputs being sent form the user device to the example extractor and feedback results being returned to users.. The suggestion for doing so is: “the example extractor 104 for a variety of useful purposes including but not limited to: providing metadata of the stored example to a user or an automated process” as disclosed in ¶[0029] of Massiceti.
4. Claims 5-6 are rejected under 35 U.S.C. 103 as being unpatentable over Guo, in view of Massiceti, further in view of Sample and further in view of Zheng et al. “Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena,” herein Zheng.
Regarding Claim 5:
The combination of Guo, Massiceti and Sample further discloses the method of claim 4, wherein testing the second prompt with the revised instruction comprises:
selecting the second set of training examples using the example selector (Massiceti: ¶[0022] discloses an example selector that selects training examples from a pool using learned weights. Reapplying that selector to select examples for evaluation of the revised instruction teaches selecting a second set of training examples using the example selector);
transmitting, to the LLM, the second prompt with the revised instruction, the second set of training examples, and the evaluation input to obtain a second (Guo: Algorithm 1 line 5 evaluation s′ i ← f(p′ i,D) which teaches generating a revised prompt and evaluating that revised prompt on the development set with the LLM to obtain a score. Further Guo experimental settings shows that evaluated prompts are used to together with prepended demonstration examples before the test case. Therefore, Guo teaches the second prompt with revised instruction through the evolved candidate prompt, while the second set of training examples is supplied by the example selector of Massiceti, and evaluation input is the test case input from the development set on D which the revised prompt is evaluated);
and comparing the evaluation output with the second (Guo: Section 3.2 teaches that the LLMs predicted output for the revised output is compared against the ground truth label).
The combination of Guo, Massiceti and Sample do not explicitly disclose:
transmitting, to the LLM, the prompt and the evaluation input to obtain an LLM output (Zheng: Section 3.1 and Fig. 1 discloses that an LLM judge is given a question (evaluation input) and one or more candidate answers (LLM outputs));
comparing the evaluation input and candidate LLM output to obtain a test result(Zheng: Section 3.1 teaches grading in which the LLM judge is given a reference answer alongside the candidate LLM output and compares them to produce a score).
Guo, Massiceti, Sample and Zheng are combinable because they are from pertinent fields of endeavor. Guo discloses a score function that evaluates each candidate prompt, both the current instruction and the revised instruction using a development set yielding evaluation scores. Guo treats this evaluation as a black box and does not explicitly describe the mechanics by which the LLM receives the prompt, training examples and evaluation input. Massiceti teaches an example selector component that selects the first and second sets of training examples as set forth in the rejection of claim 1. Zheng teaches the explicit evaluation mechanics that implement Guo’s score function specifically that evaluating prompt with an LLM comprises transmitting the evaluation input and candidate LLM output to an LLM judge and comparing the LLM’s output against a reference answer to obtain a numerical score (test result). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to disclose Guo’s score function using an LLM judge evaluation mechanism as done in Zheng because both are directed to automated LLM evaluation outputs. The suggestion/motivation for doing so is “LLM-as-a-judge is a scalable and explainable way to approximate human preferences, which are otherwise very expensive to obtain” as disclosed by Zheng’s abstract.
Regarding Claim 6:
The combination of Guo, Massiceti, Sample and Zheng further disclose the method of claim 5, wherein the first set of training examples and the second set of training examples are selected by the example selector using the evaluation input (Massiceti: ¶[0019]-[0020] discloses that a support set is a set of training examples and a query set is a held out set of examples used for assessing performance. Massiceti further discloses that the goal of the extraction algorithm is to find a support set S such that the loss on the query set Q is optimized after the model has been adapted on S, therefore teaching selection of training examples by the example selector using the evaluation input).
Guo and Massiceti are combinable because both are directed to improving model performance by optimizing prompt components using evaluation feedback. Guo optimizes the instruction text for large language models using an evolution algorithm. Massiceti optimizes a set of training examples using constrained optimization algorithms and gradient descent. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate Massiceti’s teaching that learned selection weights over example pools significantly affect model robustness into Guo’s evolutionary algorithm based instruction optimization. The suggestion for doing so is: “The inventors have found that by using a projected gradient ascent it is possible to achieve a particularly efficient process which is therefore scalable to large pools of examples” as disclosed in ¶[0025] of Massiceti.
5. Claims 9-12 are rejected under 35 U.S.C. 103 as being unpatentable over Guo in view of Massiceti, further in view of Sample and further in view of Perera (US 2020/0394461).
Regarding Claim 9:
The combination of Guo, Massiceti and Sample further disclose the method of claim 1, further comprising:
selecting an evaluation input output pair comprising an evaluation input and an evaluation output (Massiceti: ¶[0019] discloses a query set comprising held-out examples used for assessing performance and the examples have corresponding class labels, this corresponds to the claimed evaluation input and evaluation output);
(Massiceti: ¶[0057] teaches that support training examples are selected with respect to performance on the query set, i.e., the held out evaluation input),
and wherein modifying the example selector comprises modifying at least one of a plurality of parameters of the example selector based on the third test result (Massiceti: teaches updating selection weights/selection behavior based on query set performance at ¶[0044] “the initialized weights are updated…” and ¶[0078] “gradient ascent issued for updating the weights”),
The combination of Guo, Massiceti and Sample does not explicitly disclose:
and clustering, by the example selector using a clustering parameter, a plurality of training examples in an example store to generate a plurality of clusters;
wherein, the example selector selects, using a selection parameter;
a training example from each of at least a subset of clusters to obtain the third set of training examples;
the plurality of parameters comprising the clustering parameter and the selection parameter.
However, Perera discloses:
and clustering, by the example selector using a clustering parameter, a plurality of training examples in an example store to generate a plurality of clusters (Perera: ¶[0020] discloses clustering module 160 and selection module 165, i.e., clustering by the example selector. ¶[0026] determines the number of clusters that should be identified, analogous to using a clustering parameter. ¶[0030] discloses database 175 may store a corpora of documents which are training examples in an example store, the selection module selects documents for a training set based on options for selecting representative samples and according to selection criteria (see ¶[0018] and ¶[0027] i.e., using a selection parameter);
wherein, the example selector selects, using a selection parameter (Perera: ¶[0018] and ¶[0027] selects the example selector selects documents from a training set using options for selecting representative sample and according to specific selection criteria);
a training example from each of at least a subset of clusters to obtain the third set of training examples (Perera ¶[0003], ¶[0028] discloses selecting one or more document from each cluster which are assigned to a training set of documents)
the plurality of parameters comprising the clustering parameter and the selection parameter (Perera: ¶[0026]-[0027] discloses determining the goal or number of clusters to be identified and selection criteria, analogous to the plurality of parameters comprising a clustering parameter and a selection parameter).
The combination of Guo, Massiceti and Sample in view of Perera are combinable because each relate to natural language processing, use of large language models and prompt/training data generation. Specifically, it would have been obvious to one of ordinary skill in the art to disclose Perera’s clustering based training example selection into Massiceti’s performance driven example selector, because both references are directed to selecting training examples for machine learning based on structured selection processes and doing so would have enabled the selector to choose examples from clustered groups while still tuning the selector based on test/query performance. The motivation for doing so is expressly stated in Perera ¶[0010]: “ Samples may be selected from clusters in a manner that improves the semantic diversity of the sampled documents.”
Regarding Claim 10:
The combination of Guo, Massiceti, Sample and Perera further disclose the method of claim 9, wherein the clustering parameter is a number of clusters and wherein the selection parameter is a number of training examples in the third set of training parameters (Perera: ¶[0018] discloses that a user may specify options for generating a training set, including the size of the training set and ¶[0026] discloses that the cluster module may determine the number of clusters that should be identified, teaching that the clustering parameter is a number of clusters. Perera ¶[0028] further discloses that the selection module performs iterations until the training set reaches a desired size and gives the example of twenty clusters and a training set of one hundred documents, teaching that the selection parameter is a number of training examples in the selected training set).
The combination of Guo, Massiceti in view of Perera are combinable because each relate to natural language processing, use of large language models and prompt/training data generation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use Perera’s disclosed number of clusters and desired training set size as the clustering parameter and selection parameter within the combined selector framework, because doing so would have been a straightforward use of known configurable clustering and selection settings to control how many groups are formed and how many training examples are selected. The motivation for doing so is expressly stated by Perera in paragraphs ¶[0018] “A user may interact with administration module 135 to specify options for generating a training set, such as the size of the training set, the number of iterations to be performed, and options for selecting representative samples for topics (e.g., whether to select a sample nearest a centroid of a cluster, to randomly select a sample from a cluster, etc.).”
Regarding Claim 11:
The combination of Guo, Massiceti, Sample and Perera further disclose the method of claim 9, further comprising: wherein selecting an example selects a single example from each of the at least the subset of clusters (Perera: ¶[0028] discloses that in each iteration the selection module may select one document per cluster, i.e., selecting a single examples from each of the clusters).
The combination of Guo, Massiceti and Sample in view of Perera are combinable because each relate to natural language processing, use of large language models and prompt/training data generation. It would have been obvious to now of ordinary skill in the art before the effective filing date of the claimed invention to use select a single example from each cluster in the combined example selector framework because this is Perera’s disclosed iterative cluster based sampling technique for building the training set. The motivation is expressly stated by Perera ¶[0028] “ The selection criterion that is applied may be determined randomly from one cluster to another and/or from one iteration to the next. In some embodiments, a user may provide instructions regarding the choice of selection criteria.”
Regarding Claim 12:
The combination of Guo, Massiceti, Sample and Perera further disclose the method of claim 9, wherein selecting the training examples, comprises: identifying a closest training example in a respective cluster of the at least the subset of clusters; and selecting the closest training example as the training example for the respective cluster (Perera: ¶[0027] discloses that selection module 165 may identify a centroid for each cluster and select a document based on the distance of the document’s vector representation to the centroid of the cluster to which it belongs and further discloses that the selection module may select a document that is closest to its clusters centroid, teaching identifying and selecting the closest training example for the respective cluster).
The combination of Guo, Massiceti and Sample in view of Perera are combinable because each relate to natural language processing, use of large language models and prompt/training data generation. It would have been obvious to now of ordinary skill in the art before the effective filing date of the claimed invention to identify and select the closest training example from each cluster in the combined example selector framework because Perera expressly uses centroid distances as a cluster based selection criterion for forming the training set. The motivation is expressly stated by in Perera ¶[0010]: “ Samples may be selected from clusters in a manner that improves the semantic diversity of the sampled documents.”
6. Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Guo in view of Massiceti, further in view of Sample, further in view of Perera and further in view of Zheng.
Regarding Claim 13:
The combination of Guo, Massiceti, Sample and Perera further disclose the method of claim 9, further comprising:
transmitting, to the LLM, a third prompt with the current instruction (Guo: Section 3.1 and Algorithm 1 disclose evaluation prompts by applying them to the desired LLM, where prompts in the population are used as instructions for LLM evaluation. This teaches transmitting a prompt with the current instruction to the LLM), the third set of training examples (Massiceti: ¶[0019]-[0020], ¶[0022] and ¶[0042] discloses that a support set is a set of training examples and that an example selector examples from a pool using learned weights therefore teaching the third set of training examples), and the evaluation input (Massiceti: ¶[0019]-[0020], ¶[0057] discloses that a query set is a held out set of examples used for assessing performance, therefore teaching the evaluation input) to obtain an LLM output (Zheng: Section 3.1 teaches that an LLM judge receives the evaluation material and produces and output score therefore teaching obtaining an LLM output);
Guo and Massiceti are combinable because both are directed to improving model performance by optimizing prompt components using evaluation feedback. Guo optimizes the instruction text for large language models using an evolution algorithm. Massiceti optimizes a set of training examples using constrained optimization algorithms and gradient descent. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate Massiceti’s teaching that learned selection weights over example pools significantly affect model robustness into Guo’s evolutionary algorithm based instruction optimization. The suggestion for doing so is: “The inventors have found that by using a projected gradient ascent it is possible to achieve a particularly efficient process which is therefore scalable to large pools of examples” as disclosed in ¶[0025] of Massiceti.
Guo, Massiceti, Sample and Perera do not explicitly disclose and comparing the evaluation output with the LLM output to obtain the third test result.
However, Zheng discloses and comparing the evaluation output with the LLM output to obtain the third test result (Zheng: Section 3.1 discloses comparing candidate LLM output with a reference/evaluation output, the comparison yields a numerical score corresponding to the claimed third test result).
Guo, Massiceti, Sample and Perera in view of Zheng are combinable because they are from pertinent fields of endeavor. Guo discloses a score function that evaluates each candidate prompt, both the current instruction and the revised instruction using a development set yielding evaluation scores. Guo treats this evaluation as a black box and does not explicitly describe the mechanics by which the LLM receives the prompt, training examples and evaluation input. Massiceti teaches an example selector component that selects the first and second sets of training examples as set forth in the rejection of claim 1. Sample teaches comparison result based fitness threshold stop conditions for training. Zheng teaches the explicit evaluation mechanics that implement Guo’s score function specifically that evaluating prompt with an LLM comprises transmitting the evaluation input and candidate LLM output to an LLM judge and comparing the LLM’s output against a reference answer to obtain a numerical score (test result). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to disclose Guo’s score function using an LLM judge evaluation mechanism as done in Zheng because both are directed to automated LLM evaluation outputs. The suggestion/motivation for doing so is “LLM-as-a-judge is a scalable and explainable way to approximate human preferences, which are otherwise very expensive to obtain” as disclosed by Zheng’s abstract.
7. Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Guo, in view of Massiceti, further in view of Sample, further in view of Perera and further in view of Chari (US 2013/0097103).
Regarding Claim 14:
The combination of Guo, Massiceti, Sample, Zheng and Perera further disclose the method of claim 9, except further comprising:
ordering, by the example selector using an ordering parameter, the third set of examples according to an entropy-based method, wherein the plurality of parameters further comprises the ordering parameter.
However, Chari further discloses ordering, by the example selector using an ordering parameter, the third set of examples according to an entropy-based method, wherein the plurality of parameters further comprises the ordering parameter (Chari: ¶[0033], ¶[0048] and ¶[0053] discloses that once the data has been cluster samples are selected from the clusters using maximum entropy sampling, explaining that maximum entropy sampling is performed to draw samples from each cluster and that this process is used to obtain a diverse sample population for classifier training. Chari further discloses that the iterative sampling process uses configurable quantities and weighting to determine how many samples are drawn during each iteration which corresponds to using and ordering parameter in the example selector).
The combination of Guo, Massiceti, Sample, Zheng and Perera in view of Chari are combinable because each relate to natural language processing, use of large language models and prompt/training data generation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the entropy based technique of Chari because doing so would have improved the diversity of the selected training examples while preserving the clustered selection process. The motivation is stated by Chari explicitly in ¶[0033] “Maximum entropy sampling ensures a diverse sample population for classifier training.”
8. Claims 15-16 are rejected under 35 U.S.C. 103 as being unpatentable over Guo, in view of Massiceti, further in view of Sample and further in view of Chari.
Regarding Claim 15:
The combination of Guo, Massiceti and Sample further disclose the method of claim 1, further comprising: over a plurality of iterations of the second phase: (Massiceti: ¶[0057]-[0058] discloses tuning weights based on performance because it updates selection weights through optimization with respect to a query set and test performance).
Guo and Massiceti are combinable because both are directed to improving model performance by iterative optimization using evaluation feedback. Guo optimizes the instruction text for large language models using an evolution algorithm. Massiceti optimizes a set of training examples using constrained optimization algorithms and gradient descent. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate Massiceti’s teaching of tuning weights into Guo’s evolutionary algorithm based instruction optimization. The suggestion for doing so is “The inventors have found that by using a projected gradient ascent it is possible to achieve a particularly efficient process which is therefore scalable to large pools of examples” as disclosed in ¶[0025] of Massiceti.
Guo, Massiceti and Sample do not explicitly disclose applying, by the example selector, a plurality of example selection strategies, weighting the plurality of example selection strategies according to a set of weights to generate the third set of examples.
However, Chari discloses applying, by the example selector, a plurality of example selection strategies, weighting the plurality of example selection strategies according to a set of weights to generate the third set of examples (Chari: ¶[0037]-[0038], ¶[0048] and ¶[0053] discloses using its iterative hybrid sampling process where different selection strategies are applied over iterations and combined using a weight function).
The combination of Guo, Massiceti and Sample in view of Chari are combinable because each relate to natural language processing, use of large language models and prompt/training data generation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate Chari’s weighted hybrid sampling strategies into Massiceti’s performance driven example selector because doing so would have allowed multiple example selection strategies to be combined over iterations while tuning their weights based on resulting test performance. The motivation for doing so is stated by Chari in ¶[0048] “empirically, it was noted that the best performance is with a hybrid approach where there is a mix between the simplistic method and random sampling from the clusters.”
Regarding Claim 16:
The combination of Guo, Massiceti, Sample and Chari further disclose further disclose the method of claim 15, wherein the plurality of example selection strategies comprises a set of parameters (Chari: ¶[0048] discloses that the iterative sampling strategies are based on parameters such as batch size and number of samples to draw), and wherein the method further comprises: modifying the set of parameters over the plurality of iterations of the second phase (Massiceti: ¶[0058]-[0060] teaches updating the selection weights through optimization over iterations).
The combination of Guo, Massiceti and Sample in view of Chari are combinable because each relate to natural language processing, use of large language models and prompt/training data generation. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate Chari’s weighted hybrid sampling strategies into Massiceti’s performance driven example selector because doing so would have allowed multiple example selection strategies to be combined over iterations while tuning their weights based on resulting test performance. The motivation for doing so is stated by Char in ¶[0048] “empirically, it was noted that the best performance is with a hybrid approach where there is a mix between the simplistic method and random sampling from the clusters.”
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to IAN SCOTT MCLEAN whose telephone number is (703)756-4599. The examiner can normally be reached "Monday - Friday 8:00-5:00 EST, off Every 2nd Friday".
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hai Phan can be reached at (571) 272-6338. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/IAN SCOTT MCLEAN/Examiner, Art Unit 2654
/HAI PHAN/Supervisory Patent Examiner, Art Unit 2654
Prosecution Insights