DETAILED ACTION
This action is responsive to communications filed on June 3, 2024. This action is made Non-Final.
Claims 1-20 are pending in the case.
Claims 1, 8, and 15 are independent claims.
Claims 1-20 are rejected.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
Step 1: Independent claims 1, 8, and 15 are directed towards a method, non-transitory medium, and system, respectively. Therefore, these claims, as well as their dependent claims, are directed towards one of the four statutory categories (process, machine (i.e. apparatus), manufacture, or composition of matter.
With respect to claim 1:
2A Prong 1:
Claim 1 recites the following judicial exceptions:
generating a prompt using the initial version of the prompt template at least partially by populating the dynamic input with training data; providing an evaluation at least partially based on the output; and selectively updating the prompt template … at least partially based on the evaluation, the updated version of the prompt template comprising second static input … and that is different from the first static input (mental process –can be performed in the human mind, or by a human using a pen and paper (e.g. a person may craft an input by using training data to fill in a prompt template, evaluate based on the output produced therefrom, and modify the prompt template based on the evaluation.).
2A Prong 2: The additional elements recited in the claim do not integrate the judicial exception into a practical application.
Additional elements:
A computer-implemented method for optimization of prompt templates … the method being executed by one or more processors and comprising: providing an initial version of a prompt template, the prompt template comprising dynamic input and first static input (mere instructions to apply the exception or implement the exception on a computer (e.g. a computer may be used to provide a prompt template; see MPEP §2106.05(f).). The additional elements do not effectively integrate the abstract idea into a practical application. 2B: revisiting the additional elements, the additional elements do not amount to significantly more than the judicial exception – recited high level of generality and corresponds to storing and retrieving information in memory and performing calculations).
for prompting large language models (LLMs); receiving, from a LLM, an output that is responsive to the prompt; by prompting the LLM … that is generated by the LLM (generally linking the use of a judicial exception to a particular technological environment or field of use and further, mere instructions to apply the exception or implement the exception on a computer (e.g. an LLM maybe used to generate an output based on an input, the model being trained on relevant data; see MPEP §2106.05(f) and MPEP §2106.05(h).). The additional elements do not effectively integrate the abstract idea into a practical application. 2B: revisiting the additional elements, the additional elements do not amount to significantly more than the judicial exception – recited high level of generality and corresponds to storing and retrieving information and performing calculations.).
With respect to claim 2:
2A Prong 1:
Claim 2 recites the following judicial exceptions:
wherein the prompt template is updated at least partially in response to a score of the evaluation indicating that the prompt template is to be updated (mental process –can be performed in the human mind, or by a human using a pen and paper (e.g. a person may craft an input by using training data to fill in a prompt template, evaluate based on the output produced therefrom, and modify the prompt template based on the evaluation.).
2A Prong 2: The additional elements recited in the claim do not integrate the judicial exception into a practical application.
Additional elements:
the score being provided by the LLM in response to an evaluation prompt (generally linking the use of a judicial exception to a particular technological environment or field of use and further, mere instructions to apply the exception or implement the exception on a computer (e.g. an LLM maybe used to generate an output based on an input, the model being trained on relevant data; see MPEP §2106.05(f) and MPEP §2106.05(h).). The additional elements do not effectively integrate the abstract idea into a practical application. 2B: revisiting the additional elements, the additional elements do not amount to significantly more than the judicial exception – recited high level of generality and corresponds to storing and retrieving information and performing calculations.).
With respect to claim 3:
2A Prong 1:
Claim 3 recites the following judicial exceptions:
wherein two or more iterations of updating the prompt template are performed until a score exceeds a threshold score, the score representing an evaluation metric associated with the prompt template (mental process –can be performed in the human mind, or by a human using a pen and paper (e.g. a person may craft an input by using training data to fill in a prompt template, evaluate based on the output produced therefrom, and modify the prompt template based on the evaluation.).
With respect to claim 4:
2A Prong 1:
Claim 4 recites the following judicial exceptions:
wherein two or more iterations of updating the prompt template are performed until a value of a score fails to exceed a prior value of the score, the score representing an evaluation metric associated with the prompt template (mental process –can be performed in the human mind, or by a human using a pen and paper (e.g. a person may craft an input by using training data to fill in a prompt template, evaluate based on the output produced therefrom, and modify the prompt template based on the evaluation.).
With respect to claim 5:
2A Prong 1:
Claim 5 recites the following judicial exceptions:
wherein updating the prompt template … is at least partially based on the evaluation and the prompt template … returning the updated version of the prompt template in response to the update prompt (mental process –can be performed in the human mind, or by a human using a pen and paper (e.g. a person may craft an input by using training data to fill in a prompt template, evaluate based on the output produced therefrom, and modify the prompt template based on the evaluation.).
2A Prong 2: The additional elements recited in the claim do not integrate the judicial exception into a practical application.
Additional elements:
comprises prompting the LLM using an update prompt that … the LLM (generally linking the use of a judicial exception to a particular technological environment or field of use and further, mere instructions to apply the exception or implement the exception on a computer (e.g. an LLM maybe used to generate an output based on an input, the model being trained on relevant data; see MPEP §2106.05(f) and MPEP §2106.05(h).). The additional elements do not effectively integrate the abstract idea into a practical application. 2B: revisiting the additional elements, the additional elements do not amount to significantly more than the judicial exception – recited high level of generality and corresponds to storing and retrieving information and performing calculations.).
With respect to claim 6:
2A Prong 1:
Claim 6 recites the following judicial exceptions:
wherein the evaluation … is at least partially based on the output … returning the evaluation in response to the evaluation prompt (mental process –can be performed in the human mind, or by a human using a pen and paper (e.g. a person may craft an input by using training data to fill in a prompt template, evaluate based on the output produced therefrom, and modify the prompt template based on the evaluation.).
2A Prong 2: The additional elements recited in the claim do not integrate the judicial exception into a practical application.
Additional elements:
is provided by prompting the LLM using an evaluation prompt that … the LLM (generally linking the use of a judicial exception to a particular technological environment or field of use and further, mere instructions to apply the exception or implement the exception on a computer (e.g. an LLM maybe used to generate an output based on an input, the model being trained on relevant data; see MPEP §2106.05(f) and MPEP §2106.05(h).). The additional elements do not effectively integrate the abstract idea into a practical application. 2B: revisiting the additional elements, the additional elements do not amount to significantly more than the judicial exception – recited high level of generality and corresponds to storing and retrieving information and performing calculations.).
With respect to claim 7:
2A Prong 1:
Claim 7 recites the following judicial exceptions:
and the evaluation is determined from a batch of evaluations (mental process –can be performed in the human mind, or by a human using a pen and paper (e.g. a person may craft an input by using training data to fill in a prompt template, evaluate based on the output produced therefrom, and modify the prompt template based on the evaluation, where evaluating is based on prior evaluations).
2A Prong 2: The additional elements recited in the claim do not integrate the judicial exception into a practical application.
Additional elements:
wherein the prompt is included in a batch of prompts used to prompt the LLM, the output is included in a batch of outputs returned from the LLM (generally linking the use of a judicial exception to a particular technological environment or field of use and further, mere instructions to apply the exception or implement the exception on a computer (e.g. an LLM maybe used to generate outputs based on inputs during a session, the model being trained on relevant data; see MPEP §2106.05(f) and MPEP §2106.05(h).). The additional elements do not effectively integrate the abstract idea into a practical application. 2B: revisiting the additional elements, the additional elements do not amount to significantly more than the judicial exception – recited high level of generality and corresponds to storing and retrieving information and performing calculations.).
With respect to claim 8:
Claim 8 substantially corresponds to claim 1 and is rejected under the same rationale.
With respect to claim 9:
Claim 9 substantially corresponds to claim 2 and is rejected under the same rationale.
With respect to claim 10:
Claim 10 substantially corresponds to claim 3 and is rejected under the same rationale.
With respect to claim 11:
Claim 11 substantially corresponds to claim 4 and is rejected under the same rationale.
With respect to claim 12:
Claim 12 substantially corresponds to claim 5 and is rejected under the same rationale.
With respect to claim 13:
Claim 13 substantially corresponds to claim 6 and is rejected under the same rationale.
With respect to claim 14:
Claim 14 substantially corresponds to claim 7 and is rejected under the same rationale.
With respect to claim 15:
Claim 15 substantially corresponds to claim 1 and is rejected under the same rationale.
With respect to claim 16:
Claim 16 substantially corresponds to claim 2 and is rejected under the same rationale.
With respect to claim 17:
Claim 17 substantially corresponds to claim 3 and is rejected under the same rationale.
With respect to claim 18:
Claim 18 substantially corresponds to claim 4 and is rejected under the same rationale.
With respect to claim 19:
Claim 19 substantially corresponds to claim 5 and is rejected under the same rationale.
With respect to claim 20:
Claim 20 substantially corresponds to claim 6 and is rejected under the same rationale.
2B continued: After considering all claim elements individually and as an ordered combination, it is determined that the claims do not include any additional elements that are sufficient to amount to significantly more than the judicial exception.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 5-8, 12-15, 19, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cuomo et al., US 2025/0131187 (“Cuomo”), and further in view of Madhwani et al., US Publication 2025/0260741 (“Madhwani”).
Claim 1:
Cuomo teaches or suggests a computer-implemented method for optimization of prompt templates for prompting large language models (LLMs), the method being executed by one or more processors and comprising:
providing an initial version of a prompt template, the prompt template comprising dynamic input and first static input (see Fig. 1-4; para. 0034 – Generative AI system 108 may be implemented as any of a variety of Large Language Models (LLMs); para. 0042 - evaluation system 110 may be implemented as an LLM; para. 0028 - utilize a matching technique that pairs the user provided prompt with a particular prompt template; para. 0029 – prompt template to which the user prompt is matched belongs to a prompt class type of data structure. e.g., as predetermined conditioning instructions, that can be added to the user prompt. Use of the prompt template in combination with the conditioning instructions; para. 0039 - conditioner 104 is capable of matching user prompt 204 to a selected prompt class from a plurality of available prompt classes 106. Conditioner 104 transforms the unstructured user prompt 204 into a conditioned prompt 210 using the selected prompt class. In general, the conditioning operations 208 involved in generating conditioned prompt 210 include translating user prompt 204 into a well-structured prompt based on a prompt template from the selected prompt class. Conditioner 104 adds one or more preconditioning instructions as specified by the selected prompt class to the well structured prompt to form conditioned prompt 210; para. 0046 - Each prompt class includes a prompt template that represents an expected or optimal prompt structure for the application corresponding to the prompt class; para. 0050 - each prompt class 106 may correspond to a particular type of structured output; para. 0074 - input that aligns or corresponds to available prompt classes 106 and/or prompt templates of such prompt classes. Conditioner 104 is capable of offering suggestions or instructions to help user 202 frame the natural language input appropriately to align with an available one of the prompt classes 106; para. 0078 - ensures that the wellstructured prompt ultimately generated from user prompt 204 aligns with the desired format and structure specified by the prompt template; para. 0082 - ensures that the wellstructured prompt ultimately generated from user prompt 204 aligns with the desired format and structure specified by the prompt template; para. 0086 - the conditioning instructions are a predefined set of prompt instructions; para. 0096 - The prompt template includes a placeholder, {process}, which represents the specific business process to be diagrammed; para. 0097 - the selected prompt class, i.e., the prompt class having the matched prompt template, includes conditioning instructions; para. 0104 - well-structured prompt is formed of the prompt template with keywords of user prompt 204 incorporated into the field(s) therein. The well-structured prompt is combined with the conditioning instructions which are particular for creating a BPMN diagram.);
generating a prompt using then initial version of the prompt template at least partially by populating the dynamic input (see Fig. 1-4; para. 0082 – Conditioner 104 is capable of integrating the relevant components of user prompt 204, e.g., keywords, into appropriate fields of the prompt template. That is, conditioner 104 is capable of populating fields of the prompt template with the relevant keywords extracted from user prompt 204 based on the NLP analysis. Conditioner 104 performs the integration also ensuring proper sequencing and formatting. resulting well-structured prompt aligns with the prompt template of the selected prompt class and captures the desired workflow structure. In general, the prompt template may be completed by populating fields with keywords extracted from user prompt 204 to form the well-structured prompt; para. 0083 - transformation of user prompt 204 into a well-structured prompt ensures that conditioned prompt 210 being generated aligns with a desired format and structure corresponding to the selected prompt class; para. 0096 - The prompt template includes a placeholder, {process}, which represents the specific business process to be diagrammed; para. 0097 - the selected prompt class, i.e., the prompt class having the matched prompt template, includes conditioning instructions; para. 0104 - well-structured prompt is formed of the prompt template with keywords of user prompt 204 incorporated into the field(s) therein. The well-structured prompt is combined with the conditioning instructions which are particular for creating a BPMN diagram.);
receiving, from a LLM, an output that is responsive to the prompt (see Fig. 1-4; para. 0034 – Generative AI system 108 may be implemented as any of a variety of Large Language Models (LLMs); para. 0042 - evaluation system 110 may be implemented as an LLM; para. 0082 – prompt template is configured to evoke a response including a structured output from generative AI system 108; para. 0088 - conditioner 104 submits conditioned prompt 210 to generative AI system 108; para. 0089 - response 214 as generated by generative AI system 108 may be provided.);
providing an evaluation at least partially based on the output (see Fig. 1-4; para. 0043 – conditioner 104 is optionally capable of initiating an evaluation of any response generated by generative AI system 108 in response to conditioned prompt 210; para. 0090 - conditioner 104 submits response 214 with one or more evaluation questions to evaluation system 110; para. 0091 - evaluation system 110 generates an evaluation result as feedback 220; para. 0092 - as input, evaluation request 216, which includes response 214. evaluate the quality and effectiveness of the generated prompts. The evaluation questions serve as a guideline for evaluation system 110 to assess response 214 based on specific criteria. The evaluation questions are designed to evaluate factors such as prompt completeness, accuracy, relevance, and adherence of response 214 to the desired format or requirements of downstream computer systems.); and
selectively updating the prompt template to provide an updated version of the prompt template by prompting the LLM at least partially based on the evaluation, the updated version of the prompt template comprising second static input that is generated by the LLM and that is different from the first static input (see Fig. 1-4; para. 0093 - feedback 220 is provided from evaluation system 110 to conditioner 104. Conditioner 104, in response to receiving feedback 220, is capable of updating the set of one or more conditioning instructions of the selected prompt class based on feedback 220. For example, feedback 220 may specify an additional conditioning instruction to be added to the conditioning instructions of the selected prompt class, specify a replacement conditioning instruction for a conditioning instruction of the selected prompt class, and/or specify that a conditioning instruction be deleted from the selected prompt class; para. 0094 - para. 0096 - The prompt template includes a placeholder, {process}, which represents the specific business process to be diagrammed; para. 0097 - the selected prompt class, i.e., the prompt class having the matched prompt template, includes conditioning instructions; para. 0104 - well-structured prompt is formed of the prompt template with keywords of user prompt 204 incorporated into the field(s) therein. The well-structured prompt is combined with the conditioning instructions which are particular for creating a BPMN diagram; para. 0106 - refine the list of conditioning instructions and overall prompt generation methodology; para. 0107 - included as new entries in the conditioning instructions of the selected prompt class; para. 0109 - suggests updating the prompt template to include the specific requirement of indicating the start and end events in the diagram; para. 0113 - may update the conditioning instructions of the selected prompt class by adding the two suggested conditioning instructions thereto; para. 0115 - recommendation is then included as a preconditioning instruction within the corresponding prompt class.).
Cuomo does not explicitly disclose with training data.
Madhwani teaches or suggests with training data (see para. 0115 - prompt template may include a natural language instruction instructing a generative language model to determine information; para. 0173 - set of training communication sessions is identified at 1706. Sets of one or more input characteristics and actual routes for the training communication sessions are identified at 1708; para. 0177 - set of training prompts is determined based on the prompt templates and the sets of one or more input characteristics of the communication sessions. In some embodiments, the a training prompt may be determined by filling one or more fillable portions in the prompt template; para. 0180 - tuning layer for the generative language model is updated at 1716 based on a comparison of the proposed communication routes and the actual communication routes. By updating the tuning layer in this way, the generative language model may be adapted to the particular task.).
Accordingly, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Madhwani with those of Cuomo. One would have been motivated to do so for the purpose of efficiently tuning a generative language model to a particular task based on training prompts using prompt templates filled with training data, improving model performance, as taught by Madhwani (0173, 0177, 0180).
Claim(s) 8 and 15:
Claim(s) 8 and 15 correspond to Claim 1, and thus, Cuomo and Madhwani teach or suggest the limitations of claim(s) 8 and 15 as well.
Claim 5:
Cuomo further teaches or suggests wherein updating the prompt template comprises prompting the LLM using an update prompt that is at least partially based on the evaluation and the prompt templates, the LLM returning the updated version of the prompt template in response to the update prompt (see Fig. 1-4; para. 0034 – Generative AI system 108 may be implemented as any of a variety of Large Language Models (LLMs); para. 0042 - evaluation system 110 may be implemented as an LLM; para. 0043 – conditioner 104 is optionally capable of initiating an evaluation of any response generated by generative AI system 108 in response to conditioned prompt 210; para. 0074 - input that aligns or corresponds to available prompt classes 106 and/or prompt templates of such prompt classes. Conditioner 104 is capable of offering suggestions or instructions to help user 202 frame the natural language input appropriately to align with an available one of the prompt classes 106; para. 0075 - conditioner 104 may perform operations such as keyword extraction, keyword comparison, tokenization, and/or part-of speech tagging. Any of a variety of standard NLP software libraries may be used to implement such functions; para. 0090 - conditioner 104 submits response 214 with one or more evaluation questions to evaluation system 110; para. 0093 - feedback 220 is provided from evaluation system 110 to conditioner 104. Conditioner 104, in response to receiving feedback 220, is capable of updating the set of one or more conditioning instructions of the selected prompt class based on feedback 220. For example, feedback 220 may specify an additional conditioning instruction to be added to the conditioning instructions of the selected prompt class, specify a replacement conditioning instruction for a conditioning instruction of the selected prompt class, and/or specify that a conditioning instruction be deleted from the selected prompt class; para. 0094 - para. 0096 - The prompt template includes a placeholder, {process}, which represents the specific business process to be diagrammed; para. 0097 - the selected prompt class, i.e., the prompt class having the matched prompt template, includes conditioning instructions; para. 0104 - well-structured prompt is formed of the prompt template with keywords of user prompt 204 incorporated into the field(s) therein. The well-structured prompt is combined with the conditioning instructions which are particular for creating a BPMN diagram; para. 0106 - refine the list of conditioning instructions and overall prompt generation methodology; para. 0107 - included as new entries in the conditioning instructions of the selected prompt class; para. 0109 - suggests updating the prompt template to include the specific requirement of indicating the start and end events in the diagram; para. 0113 - may update the conditioning instructions of the selected prompt class by adding the two suggested conditioning instructions thereto; para. 0115 - recommendation is then included as a preconditioning instruction within the corresponding prompt class.).
Claim(s) 12 and 19:
Claim(s) 12 and 19 correspond to Claim 5, and thus, Cuomo and Madhwani teach or suggest the limitations of claim(s) 12 and 19 as well.
Claim 6:
Cuomo further teaches or suggests wherein the evaluation is provided by prompting the LLM using an evaluation prompt that is at least partially based on the output, the LLM returning the evaluation in response to the evaluation prompt (see Fig. 1-4; para. 0034 – Generative AI system 108 may be implemented as any of a variety of Large Language Models (LLMs); para. 0042 - evaluation system 110 may be implemented as an LLM; para. 0043 – conditioner 104 is optionally capable of initiating an evaluation of any response generated by generative AI system 108 in response to conditioned prompt 210; para. 0090 - conditioner 104 submits response 214 with one or more evaluation questions to evaluation system 110; para. 0092 - as input, evaluation request 216, which includes response 214. evaluate the quality and effectiveness of the generated prompts. The evaluation questions serve as a guideline for evaluation system 110 to assess response 214 based on specific criteria. The evaluation questions are designed to evaluate factors such as prompt completeness, accuracy, relevance, and adherence of response 214 to the desired format or requirements of downstream computer systems); para. 0093 - feedback 220 is provided from evaluation system 110 to conditioner 104. Conditioner 104, in response to receiving feedback 220, is capable of updating the set of one or more conditioning instructions of the selected prompt class based on feedback 220. For example, feedback 220 may specify an additional conditioning instruction to be added to the conditioning instructions of the selected prompt class, specify a replacement conditioning instruction for a conditioning instruction of the selected prompt class, and/or specify that a conditioning instruction be deleted from the selected prompt class; para. 0106 - refine the list of conditioning instructions and overall prompt generation methodology; para. 0107 - included as new entries in the conditioning instructions of the selected prompt class; para. 0109 - suggests updating the prompt template to include the specific requirement of indicating the start and end events in the diagram; para. 0113 - may update the conditioning instructions of the selected prompt class by adding the two suggested conditioning instructions thereto; para. 0115 - recommendation is then included as a preconditioning instruction within the corresponding prompt class.).
Claim(s) 13 and 20:
Claim(s) 13 and 20 correspond to Claim 6, and thus, Cuomo and Madhwani teach or suggest the limitations of claim(s) 13 and 20 as well.
Claim 7:
Cuomo further teaches or suggests wherein the prompt is included in a batch or prompts used to prompt the LLM, the output is included in a batch of outputs returned from the LLM, and the evaluation is determined from a batch of evaluations (see Fig. 1-4; para. 0030 - feedback from the evaluation system, the conditioning instruction(s) of the prompt class(es) may be modified. The iterative aspect of incorporating feedback from the evaluation system facilitates ongoing enhancements and continued improvement to the prompt classes by way of modifying the conditioning instructions to achieve improved performance from the generative AI system over time; para. 0092 - evaluate the quality and effectiveness of the generated prompts. The evaluation questions serve as a guideline for evaluation system 110 to assess response 214 based on specific criteria. The evaluation questions are designed to evaluate factors such as prompt completeness, accuracy, relevance, and adherence of response 214 to the desired format or requirements of downstream computer systems; para. 0118 - employing the feedback mechanisms illustrated in connection with evaluation system 110, architecture 100 is capable of iteratively improving the prompt generation process through the fine tuning of conditioning instructions. This ensures that the generated prompts meet the desired criteria, resulting in precise, contextually appropriate, and machine-consumable output for downstream systems or applications.).
Claim(s) 14:
Claim(s) 14 correspond to Claim 7, and thus, Cuomo and Madhwani teach or suggest the limitations of claim(s) 14 as well.
Claim(s) 2-4, 9-11, and 16-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cuomo, in view of Madhwani, and further in view of Muthu et al., US Publication 2025/0209372 (“Muthu”).
Claim 2:
Cuomo further teaches or suggests wherein the prompt template is at least partially updated in response to … the evaluation indicating that the prompt template is to be updated, … being provided by the LLM in response to an evaluation prompt (see Fig. 1-4; para. 0034 – Generative AI system 108 may be implemented as any of a variety of Large Language Models (LLMs); para. 0042 - evaluation system 110 may be implemented as an LLM; para. 0043 – conditioner 104 is optionally capable of initiating an evaluation of any response generated by generative AI system 108 in response to conditioned prompt 210; para. 0090 - conditioner 104 submits response 214 with one or more evaluation questions to evaluation system 110; para. 0093 - feedback 220 is provided from evaluation system 110 to conditioner 104. Conditioner 104, in response to receiving feedback 220, is capable of updating the set of one or more conditioning instructions of the selected prompt class based on feedback 220. For example, feedback 220 may specify an additional conditioning instruction to be added to the conditioning instructions of the selected prompt class, specify a replacement conditioning instruction for a conditioning instruction of the selected prompt class, and/or specify that a conditioning instruction be deleted from the selected prompt class; para. 0094 - para. 0096 - The prompt template includes a placeholder, {process}, which represents the specific business process to be diagrammed; para. 0097 - the selected prompt class, i.e., the prompt class having the matched prompt template, includes conditioning instructions; para. 0104 - well-structured prompt is formed of the prompt template with keywords of user prompt 204 incorporated into the field(s) therein. The well-structured prompt is combined with the conditioning instructions which are particular for creating a BPMN diagram; para. 0106 - refine the list of conditioning instructions and overall prompt generation methodology; para. 0107 - included as new entries in the conditioning instructions of the selected prompt class; para. 0109 - suggests updating the prompt template to include the specific requirement of indicating the start and end events in the diagram; para. 0113 - may update the conditioning instructions of the selected prompt class by adding the two suggested conditioning instructions thereto; para. 0115 - recommendation is then included as a preconditioning instruction within the corresponding prompt class.).
Muthu further teaches or suggests a score of … the score (see para. 0029 - optimization configuration (e.g., specified by a user) may specify that an input prompt should be optimized for a given number of iterations or until some other condition is met. (e.g., a score that meets a threshold condition; para. 0031 - automated prompt optimization described herein can greatly reduce and even eliminate this waste; para. 0032 - continuously improving prompt refinement conserves processor resources that would have been wasted using un-optimized prompts; para. 0036 – that generates a score for the input prompt 105 based on the output of the child machine learning model 100 and one or more scoring criteria para. 0038 - the input prompt 105 may be optimized through a process that repeats five times. Five iterations is included as an example, and other numbers and/or conditions are possible. If the score for the input prompt 105 indicates that the prompt does not have deficiencies that need to be corrected, then the input prompt 105 may be presented to the user 108 through the user interface 120 before the specified number of iterations have been completed and/or before one or more other conditions have been satisfied; para. 0045 - generate one or more scoring criteria and use those scoring criteria to generate a score for the input prompt 205; para. 0046 - input prompt score 213 may include an evaluation of the deficiencies of the prompt 105 and/or directions for modifying the input prompt 105; para. 0048 – repeating the optimization process using the optimized prompt 115 as the input prompt 105 for a specified number of iterations, until some other condition is met, or until there are no more unsatisfied scoring criteria.).
Accordingly, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Muthu with those of Cuomo. One would have been motivated to do so for the purpose of efficiently optimizing input for LLMs using specific criteria, improving model performance and reducing resource consumption, as taught by Muthu (0029, 0031, 0032, 0038, 0045).
Claim(s) 9 and 16:
Claim(s) 9 and 16 correspond to Claim 2, and thus, Cuomo and Madhwani teach or suggest the limitations of claim(s) 9 and 16 as well.
Claim 3:
Cuomo further teaches or suggests wherein two or more iterations of updating the prompt template are performed … representing an evaluation metric associated with the prompt template (see Fig. 1-4; para. 0030 - feedback from the evaluation system, the conditioning instruction(s) of the prompt class(es) may be modified. The iterative aspect of incorporating feedback from the evaluation system facilitates ongoing enhancements and continued improvement to the prompt classes by way of modifying the conditioning instructions to achieve improved performance from the generative AI system over time; para. 0092 - evaluate the quality and effectiveness of the generated prompts. The evaluation questions serve as a guideline for evaluation system 110 to assess response 214 based on specific criteria. The evaluation questions are designed to evaluate factors such as prompt completeness, accuracy, relevance, and adherence of response 214 to the desired format or requirements of downstream computer systems; para. 0118 - employing the feedback mechanisms illustrated in connection with evaluation system 110, architecture 100 is capable of iteratively improving the prompt generation process through the fine tuning of conditioning instructions. This ensures that the generated prompts meet the desired criteria, resulting in precise, contextually appropriate, and machine-consumable output for downstream systems or applications.).
Muthu further teaches or suggests wherein two or more iterations of updating prompt … are performed until a score exceeds a threshold score, the score representing an evaluation metric associated with the prompt … (see para. 0029 - optimization configuration (e.g., specified by a user) may specify that an input prompt should be optimized for a given number of iterations or until some other condition is met. (e.g., a score that meets a threshold condition; para. 0031 - automated prompt optimization described herein can greatly reduce and even eliminate this waste; para. 0032 - continuously improving prompt refinement conserves processor resources that would have been wasted using un-optimized prompts; para. 0036 – that generates a score for the input prompt 105 based on the output of the child machine learning model 100 and one or more scoring criteria para. 0038 - the input prompt 105 may be optimized through a process that repeats five times. Five iterations is included as an example, and other numbers and/or conditions are possible. If the score for the input prompt 105 indicates that the prompt does not have deficiencies that need to be corrected, then the input prompt 105 may be presented to the user 108 through the user interface 120 before the specified number of iterations have been completed and/or before one or more other conditions have been satisfied; para. 0045 - generate one or more scoring criteria and use those scoring criteria to generate a score for the input prompt 205; para. 0046 - input prompt score 213 may include an evaluation of the deficiencies of the prompt 105 and/or directions for modifying the input prompt 105; para. 0048 – repeating the optimization process using the optimized prompt 115 as the input prompt 105 for a specified number of iterations, until some other condition is met, or until there are no more unsatisfied scoring criteria.).
Accordingly, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Muthu with those of Cuomo. One would have been motivated to do so for the purpose of efficiently optimizing input for LLMs using specific criteria, improving model performance and reducing resource consumption, as taught by Muthu (0029, 0031, 0032, 0038, 0045).
Claim(s) 10 and 17:
Claim(s) 10 and 17 correspond to Claim 3, and thus, Cuomo and Madhwani teach or suggest the limitations of claim(s) 10 and 17 as well.
Claim 4:
Cuomo further teaches or suggests wherein two or more iterations of updating the prompt template are performed … representing an evaluation metric associated with the prompt template (see Fig. 1-4; para. 0030 - feedback from the evaluation system, the conditioning instruction(s) of the prompt class(es) may be modified. The iterative aspect of incorporating feedback from the evaluation system facilitates ongoing enhancements and continued improvement to the prompt classes by way of modifying the conditioning instructions to achieve improved performance from the generative AI system over time; para. 0092 - evaluate the quality and effectiveness of the generated prompts. The evaluation questions serve as a guideline for evaluation system 110 to assess response 214 based on specific criteria. The evaluation questions are designed to evaluate factors such as prompt completeness, accuracy, relevance, and adherence of response 214 to the desired format or requirements of downstream computer systems; para. 0118 - employing the feedback mechanisms illustrated in connection with evaluation system 110, architecture 100 is capable of iteratively improving the prompt generation process through the fine tuning of conditioning instructions. This ensures that the generated prompts meet the desired criteria, resulting in precise, contextually appropriate, and machine-consumable output for downstream systems or applications.).
Muthu further teaches or suggests wherein two or more iterations of updating prompt … are performed until a value of a score fails to exceed a prior value of the score, the score representing an evaluation metric associated with the prompt … (see para. 0029 - optimization configuration (e.g., specified by a user) may specify that an input prompt should be optimized for a given number of iterations or until some other condition is met. (e.g., a score that meets a threshold condition; para. 0031 - automated prompt optimization described herein can greatly reduce and even eliminate this waste; para. 0032 - continuously improving prompt refinement conserves processor resources that would have been wasted using un-optimized prompts; para. 0036 – that generates a score for the input prompt 105 based on the output of the child machine learning model 100 and one or more scoring criteria para. 0038 - the input prompt 105 may be optimized through a process that repeats five times. Five iterations is included as an example, and other numbers and/or conditions are possible. If the score for the input prompt 105 indicates that the prompt does not have deficiencies that need to be corrected, then the input prompt 105 may be presented to the user 108 through the user interface 120 before the specified number of iterations have been completed and/or before one or more other conditions have been satisfied; para. 0045 - generate one or more scoring criteria and use those scoring criteria to generate a score for the input prompt 205; para. 0046 - input prompt score 213 may include an evaluation of the deficiencies of the prompt 105 and/or directions for modifying the input prompt 105; para. 0048 – repeating the optimization process using the optimized prompt 115 as the input prompt 105 for a specified number of iterations, until some other condition is met, or until there are no more unsatisfied scoring criteria.).
Accordingly, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Muthu with those of Cuomo. One would have been motivated to do so for the purpose of efficiently optimizing input for LLMs using specific criteria, improving model performance and reducing resource consumption, as taught by Muthu (0029, 0031, 0032, 0038, 0045).
Claim(s) 11 and 18:
Claim(s) 11 and 18 correspond to Claim 4, and thus, Cuomo and Madhwani teach or suggest the limitations of claim(s) 11 and 18 as well.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Andrew T McIntosh whose telephone number is (571)270-7790. The examiner can normally be reached M-Th 8:00am-5:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tamara Kyle can be reached at 571-272-4241. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANDREW T MCINTOSH/Primary Examiner, Art Unit 2144
Prosecution Insights