DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is responsive to the claims dated 6/14/2024.
Claims 1-14 are presented for examination.
Drawings
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(4) because reference character "546(2)" has been used to designate both the second output generated by the large language model in response to the second prompt and the third output generated by the large language model in response to the third prompt. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either "Replacement Sheet" or "New Sheet" pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specifically, FIG. 5B (sheet 6 of 11) depicts three output boxes, labeled "Output 546(1)," "Output 546(2)" and "Output 546(2)." The description at paragraph [0071] states that output 546(1) is generated in response to a first prompt 532(1), that output 546(2) is generated in response to a second prompt 532(2), and that output 546(3) is generated in response to a third prompt 532(3), and paragraph [0072] refers to output 546(3) in describing third question and answer pair 566. The third output box in FIG. 5B therefore appears to require the reference character 546(3) in place of the second occurrence of 546(2).
Specification
The disclosure is objected to because of the following informalities: paragraph [0009] recites "when executed by a processors of a processing system," where "a" should be deleted; paragraph [0018] recites "multiple choice question and their corresponding answer(s)," where "question" should be "questions"; paragraph [0047] ends with the extraneous phrase "and/or" following "is available for prompting"; paragraph [0049] recites "to generate the training conversational training data," which contains a redundant occurrence of "training"; paragraph [0050] recites that the "model_name" parameter 220 specifies a model named "gpt-4-23k," whereas FIG. 2 shows the value of the "model_name" parameter as "gpt-4-32k"; paragraph [0050] further recites "example configuration file includes a model_name parameter 220," omitting the reference numeral 202 applied to the example configuration file throughout the remainder of the disclosure; paragraph [0054] recites "providing a more in-depth, thoughtful, and/or accurate responses," where "a" should be deleted; paragraph [0056] recites "example training data 308 including in a prompt," where "including" should be "included"; paragraph [0095] recites "Processing system 900 is generally be an example," where "be" should be deleted; paragraph [0096] recites "processing system 400 includes one or more processors 902," whereas the processing system of FIG. 9 is designated 900, the same sentence later recites "processing system 900," and reference character 400 does not appear anywhere in the drawings; and paragraph [0097] recites "representative of a one or more central processing units," where "a" should be deleted. Appropriate correction is required.
Claim Rejections - 35 U.S.C. 112(b)
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-14 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claims 1, 3, 8 and 10, each of these claims recites the limitation "the LLM". There is insufficient antecedent basis for this limitation in the claim. Claim 1 recites "prompting a first large language model (LLM) with the first prompt to generate the first training data" and thereafter recites "receiving, from the LLM, the first training data based on the first prompt"; claim 8 recites the corresponding limitations in processing system form. The element previously introduced is "a first large language model (LLM)", and no unqualified recitation of an LLM appears anywhere in the claim. The claim set separately recites "a second LLM" in claims 2, 3, 9 and 10, so the unqualified recitation of "the LLM" does not identify which of the recited large language models is referenced. The defect is compounded in claims 3 and 10, which recite "the first LLM" and "a second LLM" in the same claim and then recite receiving the second training data from "the LLM". This is the lack of clarity described in MPEP 2173.05(e), which explains that "if two different levers are recited earlier in the claim, the recitation of 'said lever' in the same or subsequent claim would be unclear where it is uncertain which of the two levers was intended," and that a claim referring to "said aluminum lever," where the claim recites only "a lever" earlier, is likewise unclear. For purposes of examination, "the LLM" is interpreted as the first large language model recited in the prompting limitation of claim 1 and claim 8, consistent with the specification at paragraphs [0068] and [0071], which describe a single LLM 514 being prompted first to generate the questions and then to generate the answers.
Regarding claims 3 and 10, the claims recite "prompting the first LLM with the second prompt to generate the second training data" and, in the immediately following limitation, "receiving, from the LLM, the second training data based on the first prompt" (claim 10 recites the corresponding limitations in processing system form). These two limitations are irreconcilable. The second training data is defined by the claim as the data the first LLM generates when it is prompted with the second prompt, yet the receiving limitation recites that this same second training data is received "based on the first prompt", the first prompt being defined by claim 1 and claim 8 as the prompt carrying "a request to generate first training data". It is therefore unclear whether the received second training data must be responsive to the first prompt, to the second prompt, or to both, and the claim does not inform a person having ordinary skill in the art with reasonable certainty about the scope of the receiving limitation. See Nautilus, Inc. v. Biosig Instruments, Inc., 572 U.S. 898, 901, 910 (2014); In re Packard, 751 F.3d 1307, 1314 (Fed. Cir. 2014); MPEP 2173.05. The specification does not resolve the ambiguity. Paragraph [0087] repeats the same language, while the embodiment described at paragraphs [0069] through [0071] states that each output 546 is generated by LLM 514 when prompted with the corresponding prompt 532, that is, with the second prompt, and not with the first prompt 502. For purposes of examination, the limitation is interpreted as receiving, from the first large language model, the second training data generated in response to the second prompt.
Regarding claims 3 and 10, the claims recite "generating third training data based on the first training data and the second training data comprising a second plurality of question and answer pairs" (claim 10 recites the corresponding limitation in processing system form). It cannot be determined which antecedent the trailing modifier "comprising a second plurality of question and answer pairs" limits. The modifier may be read as limiting the third training data, as limiting the second training data, or as limiting the combination of the first training data and the second training data on which the third training data is based. These readings are not coextensive, and the second of them contradicts an earlier limitation of the same claim, which recites "wherein the second training data comprises an answer corresponding to the respective question", a single answer to a single question rather than a plurality of question and answer pairs. A claim carrying a modifier whose antecedent cannot be ascertained does not particularly point out and distinctly claim the subject matter which the inventor regards as the invention. For purposes of examination, the limitation is interpreted as requiring that the third training data comprise the second plurality of question and answer pairs, consistent with the specification at paragraph [0072], which describes training data 560 as including multiple question and answer pairs formed from output 516 and outputs 546.
Regarding claims 5 and 12, the term "different nuances" in the limitation "a second suggestion to consider different nuances associated with the first domain when generating the first training data" is a relative term which renders the claim indefinite. The term "different nuances" is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. The specification at paragraph [0054] recites only that the guidelines "may include a second suggestion to consider different nuances associated with the real estate domain when generating the training data." It nowhere states what constitutes a nuance of a domain, how a nuance is distinguished from any other attribute of a domain, or by what measure two nuances are different from one another. Whether a given guideline carried in a prompt is or is not a suggestion to consider different nuances therefore rests on the subjective judgment of the person evaluating it. The companion recitation of "different granularities associated with the first domain" is not held indefinite, because paragraph [0054] identifies granularities of the exemplary domain, such as locations and developing areas, and granularity carries an established meaning in the data processing arts. For purposes of examination, the limitation is interpreted as requiring that the one or more first guidelines include text directing the large language model to take account of distinctions particular to the first domain when generating the first training data.
Claims 2 through 7 depend from claim 1, and claims 9 through 14 depend from claim 8. Each of these claims incorporates the indefinite limitation "the LLM" of the claim from which it depends and does not cure that defect, and each is therefore rejected for the same reason. Each dependent claim has additionally been screened for indefiniteness of its own, and the further defects found in claims 3, 5, 10 and 12 are set out separately above.
Claim Rejections - 35 U.S.C. 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 6, 8-9, and 13 are rejected under 35 U.S.C. 103 as being unpatentable Nagaraju et al. (hereinafter Nagaraju), US 2024/0185001 A1, in view of Sudalairaj et al. (hereinafter Sudalairaj) “LAB: Large-Scale Alignment for ChatBots” March 2, 2024.
Regarding independent claim 1, Nagaraju teaches a method, comprising (Nagaraju: [0053], "FIG. 5 is a flow diagram of an example method 500 of using task-specific data, template queries, and a trained large language model for generating training data"; method 500 is an ordered series of processing operations carried out by processing units, and the operations recited below are the blocks of that method):
obtaining domain data associated with a first domain (Nagaraju: [0036], "if training data is being generated to train a task-oriented dialogue system that may be used in a restaurant environment, task data 112 may include menu items that are available for purchase"; the menu items gathered for the restaurant deployment are task data 112 (domain data), and the restaurant environment for which they are gathered is the environment (first domain) the generated data serves; page 19, [0054], "generating the plurality of NL queries may include sampling one or more tokens"; those domain-specific tokens are drawn from the data store before the prompt is assembled, so they are obtained as an input to the generation operation);
generating a first prompt based on one or more configuration parameters included in a configuration file, the first prompt comprising (Nagaraju: [0024], "Task data 112 and template query 114 may be in any suitable format, e.g., plain text, XML, CSV, JSON, YAML, or any other compressed or uncompressed format"; template query 114 (configuration file) is held as a stored file in a structured file format such as YAML or JSON and is read by the system rather than composed at the moment of generation, [0048], "Query prompt generator 132 may replace the placeholder tokens of template query 114 with a sampling"; the input and output labels, the placeholder tokens and the associated selection weights carried by that stored file (based on one or more configuration parameters) are the values on which query prompt generator 132 operates to assemble prompt 310 (generating a first prompt), [0048], "task data 112 may be associated with selection weights and the sampling of task data 112 may be performed based on a probability determined by the selection weights"; the stored weights govern what is placed into the prompt, so the prompt that results is determined by parameter values held in the stored file):
a request to generate first training data for the first domain based on the domain data, wherein the first training data comprises: a first plurality of question and answer pairs; a conversation comprising a first plurality of questions and a first plurality of answers corresponding to the first plurality of questions; a second plurality of questions; or a question and a second plurality of answers corresponding to the question (Nagaraju: [0046], "The output value of the dynamic input/output pairing of dynamic portion 420 of template query 114 may be left empty for trained large language model 134 to generate according to the examples provided in static portion 410 of template query 114"; the empty output value standing beside the domain-specific input value is the standing call (a request) that the model produce the missing output from the supplied task data, and what it is called on to produce is the kind of item shown in the examples (to generate first training data for the first domain on the domain data), [0054], "At block 510, processing units executing method 500 may generate a plurality of natural language (NL) queries"; the items produced are a plurality of natural language queries (second plurality of questions), [0039], "Output value: How do I get to the Convention Center?"; the generated natural language queries are themselves interrogatives, as the recited destination query shows, so the generated set is a plurality of questions);
example first training data for the first domain (Nagaraju: [0037], "Static portion 410 of template query 114 may include example input/output pairings that trained large language model 134 is to simulate"; the example input/output pairings (the example first training data) are worked examples of the very output the model is called on to produce and are carried in the prompt for the model to imitate, [0047], "The example pairing may come from a financial services domain, a food services domain, a health services domain, a retail services domain"; the example pairings are drawn from the same deployment domain as the task data, so the examples carried in the prompt are examples for that domain);
and the domain data (Nagaraju: [0046], "The placeholder tokens may be replaced by one or more task-specific or domain-specific tokens from task data 112"; the placeholder positions in the composed prompt are filled with the very task data 112 (and the domain data) that was obtained, so the domain data itself is carried in the prompt, [0049], "one dynamic input/output pairing with a list of menu items populated from"; the assembled prompt accordingly contains the menu items themselves rather than a reference to them);
prompting a first large language model (LLM) with the first prompt to generate the first training data (Nagaraju: [0049], "Prompt 310 may then be provided to trained large language model 134, with the example input/output pairings providing trained large language model 134 with few-shot examples of the kind of output value to be generated"; prompt 310 (the first prompt) is supplied to trained large language model 134 (the first large language model), and it is supplied for the purpose of having that model produce the output value, [0026], "Trained large language model 134 may be any trained machine-learning model capable of performing text completion based on a provided example"; the model so prompted is a text-completion language model of the kind the claim recites);
and receiving, from the first LLM (as interpreted per the 35 U.S.C. 112(b) rejection above), the first training data based on the first prompt (Nagaraju: [0049], "Trained large language model 134 may then complete the dynamic input/output pairing by generating an output value"; the model returns the completed output value, which is the item the prompt called for, [0026], "Trained large language model 134 may process the generated query prompt to create a generated NL query 172"; the returned item is taken back from the model as generated NL query 172 (the first training data) and is held for use as training data, so it is received from that model as the product of the prompt that was sent).
Nagaraju does not expressly teach one or more first guidelines for generating the first training data.
However, Sudalairaj teaches one or more first guidelines for generating the first training data (Sudalairaj: page 5, Figure 3 Instruction Generator prompt template, "Please follow these guiding principles when generating responses"; the generation prompt states, in its own text, a set of guiding principles (one or more first guidelines) that the model is directed to observe while producing the requested items; page 5, Figure 3 Instruction Generator prompt template, "The questions should be diverse and cover a wide range of topics"; the stated principles speak to how the requested questions are to be produced rather than to what the subject matter is; page 5, Section 3.2.1 Skill Generation, "Each template has its own set of principles and instructions that control the role of the teacher model"; each generation template carries its own principles, so the principles travel with the prompt that requests the generation).
Because Nagaraju and Sudalairaj are analogous art and within the same field of endeavor, specifically the generation of synthetic training data by prompting a large language model and the use of that generated data to tune a further language model, they address the same problem solving area of obtaining large volumes of domain-appropriate training data without human annotation, accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the stated guiding principles that Sudalairaj places inside its generation prompt with the template-driven query prompt of Nagaraju, with a reasonable expectation of success, such that the prompt supplied to the generating model carries not only the worked examples and the domain tokens but also the principles the generated items are to satisfy, to teach one or more first guidelines for generating the first training data. This modification would have been motivated by the desire to keep the generated items usable without hand inspection by stating in the prompt itself the criteria the items must meet (Sudalairaj: page 5, Section 3.2.1 Skill Generation, "This ensures that only high-quality, contextually appropriate questions move forward in the process"; Sudalairaj attributes the usability of the retained items to the principles its prompts state; page 5, Section 3.2.1 Skill Generation, "By iterating through each leaf node of a taxonomy, the teacher generates queries that adhere to specific principles"; stating the principles in the generation prompt is what Sudalairaj credits with holding the generated items to the intended coverage).
Regarding dependent claim 2, Nagaraju, in view of Sudalairaj, teach the method of claim 1, further comprising fine-tuning a second LLM for the first domain using the first training data (Sudalairaj: page 3, Figure 1 caption, "examples in the leaf nodes are used by the synthetic data generators to generate orders of magnitude data for the phased-training step for instruct-tuning"; the items produced by the generators are turned around and used to instruct-tune a further model; page 6, Section 3.3 Multi-Phase Training, "LAB training happens in two phases, knowledge tuning, followed by skills tuning"; the model so tuned is an already pre-trained large language model (the second LLM) that is further trained on the generated items of the curated branch for the subject matter of interest (the first domain), which is a further training of an already trained model rather than training from initialization).
Regarding dependent claim 6, Nagaraju, in view of Sudalairaj, teach the method of claim 1, wherein the first prompt further comprises formatting instructions for formatting the first training data (Sudalairaj: page 5, Figure 3 Instruction Generator prompt template, "Return your responses in the format of"; the generation prompt closes with an express statement of the layout the returned items are to take, which is directed to the form in which the generated items are to be rendered rather than to their subject matter; page 5, Figure 3 Instruction Generator prompt template, "Now generate {num samples} such questions, remember to follow the principles mentioned above and use the same format as the examples"; the same prompt directs the model to render the generated items in the format of the examples, so the format directive is carried within the prompt itself).
Regarding independent claim 8, this is a processing system claim that is substantially the same as the method of claim 1. Thus, claim 8 is rejected for the same reason as claim 1. In addition, Nagaraju teaches a processing system, comprising: one or more memories comprising computer-executable instructions; and one or more processors configured to execute the computer-executable instructions and cause the processing system to (Nagaraju [0026], "Query generator system 120 may include a memory 130…communicatively coupled with one or more processing devices, such as one or more graphics processing units"; query generator system 120 (the processing system) carries memory 130 (the one or more memories) coupled to the graphics and central processing units 150 and 140 (the one or more processors) that run its codes, [0053], "processing units performing any of methods 500 and/or 600 may be executing instructions stored on a non-transient computer-readable storage media"; the stored instructions (the computer-executable instructions) are what the processing units execute in order to carry out the operations of method 500).
Regarding dependent claims 9 and 13, these are processing system claims that are substantially the same as the method of claims 2 and 6, respectively. Thus, claims 9 and 13 are rejected for the same reasons as claims 2 and 6.
Claims 3 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Nagaraju in view Sudalairaj, as applied in the rejections of claims 1 and 8 above, and in further view of Zhang et al. (hereinafter Zhang) “Self-QA: Unsupervised Knowledge Guided Language Model Alignment” 2023.
Regarding dependent claim 3, Nagaraju, in view of Sudalairaj, teach the method of claim 1, wherein: the first training data comprises the second plurality of questions (Nagaraju: [0054], "generating the plurality of NL queries may include sampling one or more tokens"; the items the method produces are a plurality of natural language queries, which are the interrogative items recited as the second plurality of questions, so the generated data takes that recited form).
Nagaraju and Sudalairaj do not expressly teach the method further comprises: for each respective question of the second plurality of questions: automatically generating a second prompt based on the one or more configuration parameters included in the configuration file, the second prompt comprising: a second request to generate second training data for the first domain based on the respective question and the domain data, wherein the second training data comprises an answer corresponding to the respective question; one or more second guidelines for generating the second training data; the respective question; and the domain data; prompting the first LLM with the second prompt to generate the second training data; receiving, from the first LLM, the second training data based on the [[first]]second prompt (as interpreted per the 35 U.S.C. 112(b) rejection above); and generating third training data based on the first training data and the second training data, the third training data comprising a second plurality of question and answer pairs (as interpreted per the 35 U.S.C. 112(b) rejection above); and fine-tuning a second LLM for the first domain using the third training data.
However, Zhang teaches the method further comprises: for each respective question of the second plurality of questions: (Zhang: page 3, Section 3 Methodology, "Then we can obtain several related instructions, which can be used in the next stage"; the questions produced by the first prompt are carried forward individually into the next stage, so the operations that follow are performed for each of the generated questions) automatically generating a second prompt based on the one or more configuration parameters included in the configuration file, the second prompt comprising: (Zhang: page 3, Section 3 Methodology, "As in the previous stage, the prompt for the reading comprehension stage is as follows"; a second prompt is composed for the answering stage in the same manner as the first, from a fixed template whose named slots are filled programmatically for each question rather than written by hand, so the second prompt is produced automatically from stored parameter values) a second request to generate second training data for the first domain based on the respective question and the domain data, wherein the second training data comprises an answer corresponding to the respective question; (Zhang: page 3, Section 3 Methodology, "In this stage, the language model needs to generate answers to the generated instruction questions according to the corresponding unsupervised knowledge"; the output the second prompt calls for is an answer to the identified question, produced from the same unsupervised text of the subject matter that grounded the first prompt) one or more second guidelines for generating the second training data; (Zhang: page 3, Reading Comprehension Prompt, "Please answer this question as thoroughly as possible, but do not change the key information in the original text"; the answering prompt states its own constraints on how the answer is to be produced, which are different from the constraints stated in the question-generating prompt) the respective question; and (Zhang: page 3, Reading Comprehension Prompt, "Please answer the following question based on the content of the article above: {the generated question}"; the named slot carries the individual question produced in the first stage into the text of the second prompt) the domain data; (Zhang: page 3, Section 3 Methodology, "{unsupervised knowledge data} in the prompt represents sequential text"; the same unsupervised text of the subject matter that grounded the first prompt is carried into the second prompt through its own named slot) prompting the first LLM with the second prompt to generate the second training data; (Zhang: page 3, Section 3 Methodology, "Our proposed SELF-QA consists of three different stages: knowledge-guided instruction generation, machine reading comprehension, and filtering and pruning"; the same language model is invoked a second time in the machine reading comprehension stage, this time with the answering prompt, to obtain the answer) receiving, from the first LLM, the second training data based on the [[first]]second prompt; and (Zhang: page 4, Table 2, "Question1: When was DXM founded? Answer1: DXM was founded on April 28, 2018"; the answers returned by the model for the individual generated questions are collected, as the worked example shows) generating third training data based on the first training data and the second training data, the third training data comprising a second plurality of question and answer pairs; and (Zhang: page 4, Table 2 caption, "Examples of unsupervised background knowledge and generated question and answer pairs"; the questions returned by the first prompt and the answers returned by the second are assembled into question and answer pairs, which is the data set the pipeline produces; page 3, Reading Comprehension Prompt, "Please generate the corresponding answer in the following format"; the answering prompt fixes the form in which the second stage returns its output as a question line followed by the corresponding answer line, so the item that stage yields is the generated question joined to its generated answer) fine-tuning a second LLM for the first domain using the third training data (Zhang: page 4, Section 4.1 Performance, "It is precisely because of these domain-specific instruction-tuning data that our model has achieved better performance"; the assembled pairs are used to instruction-tune a separate pre-trained model for the subject matter of interest, as page 4 also records with the tuning of BLOOM-7B).
Because Nagaraju, in view of Sudalairaj, and Zhang are analogous art and within the same field of endeavor, specifically the automated generation of question and answer training data for language models by prompting a language model, they address the same problem solving area of producing accurate, subject-matter-grounded training pairs without human annotation, accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the separate per-question answering prompt of Zhang with the template-driven prompt generation of Nagaraju as modified by the stated guiding principles of Sudalairaj, with a reasonable expectation of success, such that each question returned from the first prompt is fed back into a second automatically composed prompt that carries that question, the same subject-matter text and its own constraints and that requests the corresponding answer, and the returned answers are joined to the questions as pairs on which a further model is tuned, to teach the method further comprises: for each respective question of the second plurality of questions: automatically generating a second prompt based on the one or more configuration parameters included in the configuration file, the second prompt comprising: a second request to generate second training data for the first domain based on the respective question and the domain data, wherein the second training data comprises an answer corresponding to the respective question; one or more second guidelines for generating the second training data; the respective question; and the domain data; prompting the first LLM with the second prompt to generate the second training data; receiving, from the first LLM, the second training data based on the second prompt; and generating third training data based on the first training data and the second training data, the third training data comprising a second plurality of question and answer pairs; and fine-tuning a second LLM for the first domain using the third training data. This modification would have been motivated by the desire to keep the generating model on one task at a time so that the answers it returns are fuller and more accurate (Zhang: page 4, Section 4.2 Different Stages of SELF-QA, "by combining these two tasks, the model may not be able to focus on a single task as effectively, which can result in less detailed and accurate answers"; Zhang weighs asking one prompt to do both jobs against issuing a separate answering prompt for each generated question and identifies the loss of answer quality that follows from the former, which is a reason stated in the art itself for the separate second prompt).
Regarding dependent claim 10, it is a processing system claim that is substantially the same as the method of claim 3. Thus, claim 10 is rejected for the same reason as claim 3.
Claims 4 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over Nagaraju in view of Sudalairaj, as applied in the rejections of claims 1 and 8, respectively, above, and in further view of Attali et al. (hereinafter Attali) , US 2023/0080674 A1.
Regarding dependent claim 4, Nagaraju, in view of Sudalairaj, teach all the elements of claim 1.
Nagaraju and Sudalairaj do not expressly teach wherein: the first training data comprises the question and the second plurality of answers corresponding to the question, and only one answer of the second plurality of answers comprises a correct answer to the question.
However, Attali teaches wherein: the first training data comprises the question and the second plurality of answers corresponding to the question, (Attali: [0036], "generate a set of possible incorrect responses to each of the multiple-choice questions"; for each question the model is prompted to return a set of further answer options for that same question, so the item produced is the question accompanied by a plurality of answers that correspond to it, [0190], "the top N selected as incorrect answers for a multiple-choice test item"; those generated options are carried into the finished item alongside the question) and only one answer of the second plurality of answers comprises a correct answer to the question (Attali: [0182], "The top-ranked candidate may then be selected as the correct answer for an item"; a single candidate is designated the item's correct answer, and [0190] designates the remainder incorrect, so exactly one of the answers presented with the question is the correct one).
Because Nagaraju, in view of Sudalairaj, and Attali are analogous art and within the same field of endeavor, specifically the generation of question-form training and assessment items by prompting a large language model, they are also each reasonably pertinent to the problem of obtaining well-formed question items in quantity without human authoring, accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the answer-option generation of Attali with the template-driven prompt generation of Nagaraju as modified by the stated guiding principles of Sudalairaj, with a reasonable expectation of success, such that the prompt requests, for a question drawn from the subject matter, a set of answer options of which only the designated candidate is correct, to teach wherein: the first training data comprises the question and the second plurality of answers corresponding to the question, and only one answer of the second plurality of answers comprises a correct answer to the question. This modification would have been motivated by the desire to obtain usable answer options at scale without an author writing each one (Attali: [0004], "the use of item models expands the potential number of items that may be generated"; Attali attributes the practical value of automated item generation to the number of items obtainable without authoring each, and [0190], "Distractors can be filtered and ranked"; the generated options are selected mechanically rather than composed by hand).
Regarding dependent claim 11, it is a processing system claim that is substantially the same as the method of claim 4. Thus, claim 11 is rejected for the same reason as claim 4.
Claims 5 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Nagaraju in view of Sudalairaj, as applied in the rejections of claims 1 and 8, respectively, above, and in further view of Xu et al. (hereinafter Xu) “WizardLM: Empowering Large Language Models to Follow Complex Instructions” 2023.
Regarding dependent claim 5, Nagaraju, in view of Sudalairaj, teach all the elements of claim 1.
Nagaraju and Sudalairaj do not expressly teach wherein the one or more first guidelines comprise at least one of: a first suggestion to consider different granularities associated with the first domain when generating the first training data[[; or a second suggestion to consider different nuances associated with the first domain when generating the first training data]] (as interpreted per the 35 U.S.C. 112(b) rejection above).
However, Xu teaches wherein the one or more first guidelines comprise at least one of: a first suggestion to consider different granularities associated with the first domain when generating the first training data (Xu: page 5, Section 3.2 In-Depth Evolving prompt template, "If #Given Prompt# contains inquiries about certain issues, the depth and breadth of the inquiry can be increased"; the generation prompt states to the generating model, as one of the methods it is directed to apply, that it vary the depth and breadth at which the inquiry is pitched, which is a stated suggestion that the model work at differing levels of specificity when producing the instruction data; page 5, Section 3.2 In-Depth Evolving prompt template, "Please replace general concepts with more specific concepts"; the same prompt states the substitution of finer concepts for coarser ones as a further method the model is to consider, which is a suggestion addressed to the level of detail of the generated items rather than to their subject matter; page 6, Section 3.2 In-Breadth Evolving prompt template, "This new prompt should belong to the same domain as the #Given Prompt# but be even more rare"; the levels the model is told to consider are held inside the subject-matter area of the item it started from, so the differing levels of specificity are levels associated with that subject-matter area (the first domain); page 4, Section 3 Approach, "Evol-Instruct can control the difficulty and complexity level of the generated instructions"; states that these prompt-borne directives are what let the level at which the generated data sits be controlled).
Because Nagaraju, in view of Sudalairaj, and Xu are analogous art and within the same field of endeavor, specifically the generation of instruction and question training data for language models by prompting a language model, they address the same problem solving area of producing generated data that covers a subject-matter area at more than one level of detail, accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the specificity-varying directives that Xu places in its generation prompt with the guiding principles that Sudalairaj places in the generation prompt of the Nagaraju template-driven arrangement, with a reasonable expectation of success, such that the principles carried in the prompt include a suggestion that the model consider differing levels of specificity within the subject-matter area when producing the items, to teach wherein the one or more first guidelines comprise at least one of: a first suggestion to consider different granularities associated with the first domain when generating the first training data. This modification would have been motivated by the desire to obtain generated data that is not uniformly pitched at one level and therefore covers the subject-matter area more completely (Xu: page 2, Section 1 Introduction, "a novel method using LLMs instead of humans to automatically mass-produce open-domain instructions of various difficulty levels, to improve the performance of LLMs"; Xu attributes the improvement it reports to producing items across a range of levels rather than at a single level, which is a reason stated in the art itself for directing the generating model to consider differing levels).
Regarding dependent claim 12, it is a processing system claim that is substantially the same as the method of claim 5. Thus, claim 12 is rejected for the same reason as claim 5.
Allowable Subject Matter
Claims 7 and 14 objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims as well as overcoming any outstanding 35 U.S.C. 112 issues.
The following is a statement of reasons for the indication of allowable subject matter:
The closest prior arts found when taken individually or in combination do not expressly teach claims 7 and 14 when taken in the context of the claims as a whole.
At best the closest prior arts uncovered, specifically, Nagaraju discloses systems and techniques that may generate datasets for training task-oriented dialogue systems. The techniques include generating natural language queries by selecting a template query, sampling one or more tokens from a data store of domain-specific tokens, modifying the selected template query using the one or more sampled tokens to generate a query prompt, and using a natural language generative machine-learning model to generate, based on the query prompt, a respective natural language query of the subset of the plurality of natural language queries, and causing the generated plurality of natural language queries to be provided to a machine-learning model training engine configured to train, using the generated plurality of natural language queries, a conversational machine-learning model to perform a domain-specific conversational task (Abstract); Sudalairaj “LAB: Large-Scale Alignment for ChatBots” discloses LAB (Large-scale Alignment for chatBots), a novel methodology designed to overcome the scalability challenges in the instruction-tuning phase of large language model (LLM) training. Leveraging a taxonomy-guided synthetic data generation process and a multi-phase tuning framework, LAB significantly reduces reliance on expensive human annotations and proprietary models like GPT-4. We demonstrate that LAB-trained models can achieve competitive performance across several benchmarks compared to models trained with traditional human-annotated or GPT-4 generated synthetic data. Thus offering a scalable, cost-effective solution for enhancing LLM capabilities and instruction-following behaviors without the drawbacks of catastrophic forgetting, marking a step forward in the efficient training of LLMs for a wide range of applications (Abstract); and Nayak “Learning to Generate Instruction Tuning Datasets for Zero-Shot Task Adaptation” discloses Bonito, an open-source model for conditional task generation that converts unannotated text into task-specific training datasets for instruction tuning. We aim to enable zero-shot task adaptation of large language models on users’ specialized, private data. We train Bonito by fine-tuning a pretrained large language model on a new large-scale dataset with 1.65 M examples created by remixing existing instruction tuning datasets into meta-templates. The meta-templates for a dataset produce training examples where the input is the unannotated text and the task attribute and the output consists of the instruction and the response. We use Bonito to generate synthetic tasks for seven datasets from specialized domains with unannotated text across three task types—yes-no question answering, extractive question answering, and natural language inference—and adapt language models. We show that Bonito significantly improves the average performance of pretrained and instruction tuned models over the de facto self supervised baseline. For example, adapting Mistral-Instruct-v2 and instruction tuned variants of Mistral and Llama2 with Bonito improves the strong zero-shot performance by 22.1 F1 points whereas the next word prediction objective undoes some of the benefits of instruction tuning and reduces the average performance by 0.8 F1 points. We conduct additional experiments with Bonito to understand the effects of the domain, the size of the training set, and the choice of alternative synthetic task generators. Overall, we show that learning with synthetic instruction tuning datasets is an effective way to adapt language models to new domains. The model, dataset, and code are available at https://github. com/BatsResearch/bonito (Abstract).
In addition, references uncovered have not provided a basis of evidence for asserting a motivation for one of ordinary skill in the art before the effective filing date of the invention, knowing the teachings of the prior art of record, would have combined them to arrive at the present invention in the context of claims 7 and 14 as a whole.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Nayak et al. “Learning to Generate Instruction Tuning Datasets for Zero-Shot Task Adaptation” (arXiv:2402.18334v1, published February 28, 2024) (ABSTRACT We introduce Bonito, an open-source model for conditional task generation that converts unannotated text into task-specific training datasets for instruction tuning. We aim to enable zero-shot task adaptation of large language models on users’ specialized, private data. We train Bonito by fine-tuning a pretrained large language model on a new large-scale dataset with 1.65 M examples created by remixing existing instruction tuning datasets into meta-templates. The meta-templates for a dataset produce training examples where the input is the unannotated text and the task attribute and the output consists of the instruction and the response. We use Bonito to generate synthetic tasks for seven datasets from specialized domains with unannotated text across three task types—yes-no question answering, extractive question answering, and natural language inference—and adapt language models. We show that Bonito significantly improves the average performance of pretrained and instruction tuned models over the de facto self supervised baseline. For example, adapting Mistral-Instruct-v2 and instruction tuned variants of Mistral and Llama2 with Bonito improves the strong zero-shot performance by 22.1 F1 points whereas the next word prediction objective undoes some of the benefits of instruction tuning and reduces the average performance by 0.8 F1 points. We conduct additional experiments with Bonito to understand the effects of the domain, the size of the training set, and the choice of alternative synthetic task generators. Overall, we show that learning with synthetic instruction tuning datasets is an effective way to adapt language models to new domains. The model, dataset, and code are available at https://github. com/BatsResearch/bonito).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KUANG FU CHEN whose telephone number is (571)272-1393. The examiner can normally be reached M-F 9:00-5:30pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Welch can be reached on (571) 272-7212. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KC CHEN/Primary Patent Examiner, Art Unit 2143
