Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This action is responsive to the following communication: Non-Provisional Application filed Apr. 8, 2024.
Claims 1-20 are pending in the case. Claims 1, 10 and 12 are independent claims.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-20 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Kotikalapudi et al. (hereinafter Koti) U.S. Patent Publication No. 2024/0394471.
With respect to independent claim 1, Koti teaches a method comprising:
causing a first language model to generate a response based on a prompt (see e.g., Fig. 1 Para [36]-[52]-"The client device 110 can execute one or more software applications, via application engine 115, through which NL based input can be submitted and/or NL based output and/or other output that is responsive to the NL based input can be rendered (e.g., audibly and/or visually).");
identifying a set of instructions in the prompt (see e.g., Para [54] - " a set of instructions 212 included in the NL based input 210 can also be obtained");
for each instruction in the set of instructions, causing a second language model to determine whether the response indicates that the first language model followed said each instruction (see e.g., Para [60]-[61]-"A corresponding critique response, of the critique responses 230, can be generated for each of the candidate responses 220. As such, each of the critique responses 230 can be indicative of an extent to which one or more of the corresponding candidate response 220 complies with the set of instructions 212. For instance, a given critique response 230 can include an indication of whether the corresponding candidate response 220 complies with each one of the instructions of the set of instructions 212.");
in response to determining that the response indicates that the first language model did not follow a particular instruction in the set of instructions (see e.g., Para [82]-[85]-"the response revision engine 161 can be directed (e.g., via the request) to ensure that the refined candidate responses 260 generated comply with the instructions of the set of instructions 212 which the candidate response 222 was determined not to comply with "):
generating a second prompt that prompts the first language model to follow the particular instruction (see e.g., Para [82][194]-" the request can be generated and processed, along with the candidate response 222 and the corresponding critique response 232 (and optionally the set of instructions 212), using the LLM.""The request for the LLM to refine the first candidate LLM response can include an indication of the instructions of the set of instructions that the first candidate LLM response was determined to violate based on the corresponding critique response");
causing the first language model to generate a second response based on the second prompt (see e.g., Para [82][183]-"the refined candidate responses 260 can be generated based on processing a request for the LLM to refine a candidate response 222. For instance, the request can be generated and processed, along with the candidate response 222 and the corresponding critique response 232""the method can further include processing the request using the LLM to generate the plurality of refined LLM responses.");
wherein the method is performed by one or more computing devices (see e.g., Para [128]-"This system of the method 600B includes one or more processors, memory, and/or other component(s) of computing device(s) (e.g., client device 110 of FIG. 1, NL based response system 120 of FIGS. 1 to 5, computing device 910 of FIG. 9, one or more servers, and/or other computing devices). Moreover, while operations of the method 600B are shown in a particular order, this is not meant to be limiting. One or more operations may be reordered, omitted, and/or added.").
With respect to dependent claim 2, Koti teaches in response to determining that the second response indicates that the first language model followed each instruction in the set of instructions, providing the second response to the prompt (see e.g., Para [206]-" determining whether the alternative candidate LLM response satisfies each of the instructions included in the set of instructions for the LLM response; and in response to determining that the alternative candidate LLM response satisfies each of the instructions included in the set of instructions for the LLM response: causing the alternative candidate LLM response to be rendered at the client device.").
With respect to dependent claim 3, Koti teaches storing the second prompt, the second response, and the set of instructions as a training instance in a training dataset; finetuning the first language model based on the training dataset (see e.g., Para [8][51][99]-"The NL based input and the high quality response can then be stored as a training instance to be used for fine-tuning the LLM." “An LLM stored in the LLM(s) database 142A can be fine-tuned using the training engine 132 based on the training instances stored in the training instance(s) database 132A (e.g., as described in FIG. 5). ““Furthermore, the selected response 224, the NL based input 210, and optionally the corresponding critique response can be stored in a training instance(s) database 132A.”).
With respect to dependent claim 4, Koti teaches identifying the set of instructions comprises: identifying a plurality of sentences in the prompt (see e.g., Para [69]-"at least some of the instructions of the set of instructions 212 can relate to statistics of the candidate response (e.g., the instructions can specify a maximum number of characters, words, sentences, lines, paragraphs, etc.). "); for each sentence of one or more sentences in the plurality of sentences, identifying a plurality of phrases in said each sentence (see e.g., Para [96]-"a determination 242 as to whether progressive refinement should be performed. The determination can be based on contextual data and/or the content of the NL based input 210 (e.g., based on the type or number of instructions included in the NL based input 210, based on keywords or phrases included in the NL based input 210, etc.). "); for each sentence or phrase in the plurality of sentences or the plurality of phrases, determine whether said each sentence or phrase is an instruction (see e.g., Para [64]-[66]).
With respect to dependent claim 5, Koti teaches the second prompt includes the response (see e.g., Para [81]-[82]-" the request can be generated and processed, along with the candidate response 222 and the corresponding critique response 232 (and optionally the set of instructions 212), using the LLM. The content of the request for the LLM to generate the refined candidate responses 260 can be predefined, or can be generated through utilization of the NL based response system 120.").
With respect to dependent claim 6, Koti teaches the second prompt includes the prompt (see e.g., Para [81]-[82]-" the request can be generated and processed, along with the candidate response 222 and the corresponding critique response 232 (and optionally the set of instructions 212), using the LLM. The content of the request for the LLM to generate the refined candidate responses 260 can be predefined, or can be generated through utilization of the NL based response system 120.").
With respect to dependent claim 7, Koti teaches causing the second language model to determine whether the second response indicates that the first language model followed the particular instruction (see e.g., Para [84]-[85]-"The refined responses 260 can then be evaluated against the set of instructions 212 to generate critique responses 234.").
With respect to dependent claim 8, Koti teaches the second prompt prompts the first language model to follow each instruction in the set of instructions (see e.g., Para [84]-"the response revision engine 161 can be directed (e.g., via the request) to ensure that the refined candidate responses 260 generated comply with the instructions of the set of instructions 212").
With respect to dependent claim 9, Koti teaches the first language model and the second language model are the same language model (see e.g., Para [60]-"the critique responses 230 can be generated based on processing the candidate responses 220 using an LLM (e.g., the same LLM used to generate the candidate responses 220 and/or the set of instructions 212, or a different LLM)").
With respect to independent claim 10, Koti teaches a method comprising: for each prompt of a plurality of prompts: causing a first language model to generate a response based on said each prompt (see e.g., Para [04][53]-"an LLM can be used to process an NL based input to generate a plurality of responses" ”The NL based response system 120 can generate LLM output based on processing the NL based input 210 (e.g., by the LLM engine 142 using an LLM stored in the LLM(s) database 142A).”); identifying a plurality of instructions based on said each prompt (see e.g., Para [54]-[60]-"the set of instructions 212 can be extracted from the NL based input 210 by the NL based response system 120 "); causing a second language model to generate, based on the plurality of instructions, one or more outputs that indicate that the first language model followed each instruction in the plurality of instructions (see e.g., Para [60]-"the critique responses 230 can be generated based on processing the candidate responses 220 using an LLM (e.g., the same LLM used to generate the candidate responses 220 and/or the set of instructions 212, or a different LLM)");
storing the prompt, the response, and the plurality of instructions in a training instance; (see e.g., Para [08][99]-"The NL based input and the high quality response can then be stored as a training instance to be used for fine-tuning the LLM."” the selected response 224, the NL based input 210, and optionally the corresponding critique response can be stored in a training instance(s) database 132A.”); adding the training instance to training data (see e.g., Para [99]-" the selected response 224, the NL based input 210, and optionally the corresponding critique response can be stored in a training instance(s) database 132A."); finetuning the first language model based on the training data (see e.g., Para [51]-"An LLM stored in the LLM(s) database 142A can be fine-tuned using the training engine 132 based on the training instances stored in the training instance(s) database 132A (e.g., as described in FIG. 5)."); wherein the method is performed by one or more computing devices (see e.g., Para [128]-"This system of the method 600B includes one or more processors, memory, and/or other component(s) of computing device(s) (e.g., client device 110 of FIG. 1, NL based response system 120 of FIGS. 1 to 5, computing device 910 of FIG. 9, one or more servers, and/or other computing devices). Moreover, while operations of the method 600B are shown in a particular order, this is not meant to be limiting. One or more operations may be reordered, omitted, and/or added.").
With respect to dependent claim 11, Koti teaches finetuning the first language model comprises: identifying, in the training data, a first training instance that comprises a first prompt, a first plurality of instructions, and a first response (see e.g., Para [107]-"As shown in FIG. 5, an NL based input 512 and a training instance response 514 can be obtained from a particular training instance 510 (which can be retrieved, e.g., from the training instance(s) database 132A). "); causing the first language model to generate a particular response based on the first prompt in the first training instance (see e.g., Para [107]-" A selected response 520 can be generated based on processing the NL based input 512 using the NL based response system 120 (or using an LLM thereof)"); based on the particular response, determining whether the first language model followed all of the instructions in the first plurality of instructions (see e.g., Para [107]-"The selected response 520 can be compared with the training instance response 514 to generate a training loss 530.");
in response to determining that the first language model did not follow all of the instructions in the first plurality of instructions, backpropagating a loss to the first language model (see e.g., Para [107]-" Moreover, the LLM can be updated based on the training loss 530.").
With respect to independent claim 12, Koti teaches one or more non-transitory storage media storing instructions which, when executed by one or more computing devices (see e.g., Fig. 6A Para [121], cause: causing a first language model to generate a response based on a prompt (see e.g., Para [04][53]-"an LLM can be used to process an NL based input to generate a plurality of responses" ”The NL based response system 120 can generate LLM output based on processing the NL based input 210 (e.g., by the LLM engine 142 using an LLM stored in the LLM(s) database 142A).”); identifying a set of instructions in the prompt (see e.g., Para [54]-[60]-"the set of instructions 212 can be extracted from the NL based input 210 by the NL based response system 120 ");
for each instruction in the set of instructions, causing a second language model to determine whether the response indicates that the first language model followed said each instruction (see e.g., Para [60]-[61]-"A corresponding critique response, of the critique responses 230, can be generated for each of the candidate responses 220. As such, each of the critique responses 230 can be indicative of an extent to which one or more of the corresponding candidate response 220 complies with the set of instructions 212. For instance, a given critique response 230 can include an indication of whether the corresponding candidate response 220 complies with each one of the instructions of the set of instructions 212.").
With respect to dependent claim 13, Koti teaches in response to determining that the response indicates that the first language model followed each instruction in the set of instructions, providing the response to the prompt (see e.g., Para [206]-" determining whether the alternative candidate LLM response satisfies each of the instructions included in the set of instructions for the LLM response; and in response to determining that the alternative candidate LLM response satisfies each of the instructions included in the set of instructions for the LLM response: causing the alternative candidate LLM response to be rendered at the client device.").
With respect to dependent claim 14, Koti teaches storing the prompt, the response, and the set of instructions as a training instance in a training dataset; finetuning the first language model based on the training dataset (see e.g., Para [8][51][99]-"The NL based input and the high quality response can then be stored as a training instance to be used for fine-tuning the LLM." “An LLM stored in the LLM(s) database 142A can be fine-tuned using the training engine 132 based on the training instances stored in the training instance(s) database 132A (e.g., as described in FIG. 5). ““Furthermore, the selected response 224, the NL based input 210, and optionally the corresponding critique response can be stored in a training instance(s) database 132A.”).
With respect to dependent claim 15, Koti teaches identifying the set of instructions comprises: identifying a plurality of sentences in the prompt (see e.g., Para [69]-"at least some of the instructions of the set of instructions 212 can relate to statistics of the candidate response (e.g., the instructions can specify a maximum number of characters, words, sentences, lines, paragraphs, etc.). "); for each sentence of one or more sentences in the plurality of sentences, identifying a plurality of phrases in said each sentence (see e.g., Para [96]-"a determination 242 as to whether progressive refinement should be performed. The determination can be based on contextual data and/or the content of the NL based input 210 (e.g., based on the type or number of instructions included in the NL based input 210, based on keywords or phrases included in the NL based input 210, etc.). "); for each sentence or phrase in the plurality of sentences or the plurality of phrases, determine whether said each sentence or phrase is an instruction (see e.g., Para [64]-[66]).
With respect to dependent claim 16, Koti teaches in response to determining that the response indicates that the first language model did not follow a particular instruction in the set of instructions: generating a second prompt that prompts the first language model to follow the particular instruction (see e.g., Para [82]-[85]-"the response revision engine 161 can be directed (e.g., via the request) to ensure that the refined candidate responses 260 generated comply with the instructions of the set of instructions 212 which the candidate response 222 was determined not to comply with "); causing the first language model to generate a second response based on the second prompt ((see e.g., Para [82][183]-"the refined candidate responses 260 can be generated based on processing a request for the LLM to refine a candidate response 222. For instance, the request can be generated and processed, along with the candidate response 222 and the corresponding critique response 232""the method can further include processing the request using the LLM to generate the plurality of refined LLM responses.").
With respect to dependent claim 17, Koti teaches the second prompt includes the response or the prompt (see e.g., Para [81]-[82]-" the request can be generated and processed, along with the candidate response 222 and the corresponding critique response 232 (and optionally the set of instructions 212), using the LLM. The content of the request for the LLM to generate the refined candidate responses 260 can be predefined, or can be generated through utilization of the NL based response system 120.").
With respect to dependent claim 18, Koti teaches causing the second language model to determine whether the second response indicates that the first language model followed the particular instruction (see e.g., Para [84]-[85]-"The refined responses 260 can then be evaluated against the set of instructions 212 to generate critique responses 234.").
With respect to dependent claim 19, Koti teaches the second prompt prompts the first language model to follow each instruction in the set of instructions (see e.g., Para [84]-"the response revision engine 161 can be directed (e.g., via the request) to ensure that the refined candidate responses 260 generated comply with the instructions of the set of instructions 212").
With respect to dependent claim 20, Koti teaches the first language model and the second language model are the same language model (see e.g., Para [60]-"the critique responses 230 can be generated based on processing the candidate responses 220 using an LLM (e.g., the same LLM used to generate the candidate responses 220 and/or the set of instructions 212, or a different LLM)").
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PEIYONG WENG whose telephone number is (571)270-1660. The examiner can normally be reached on Mon.-Fri. 8 am to 5 pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Matthew Ell, can be reached on (571) 270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://portal.uspto.gov/external/portal. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
/PEI YONG WENG/Primary Examiner, Art Unit 2141
Prosecution Insights