Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Claims 21-26, 28-30, 32-35 and 37-43 are currently pending and have been examined.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 03/26/2026 and 08/10/2026 has been considered. The submission is in compliance with the provisions of 37 CFR 1.97. Form PTO-1449 is signed and attached hereto.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 21, 28-32, 34 and 39-40 are rejected under 35 U.S.C. 103 as being unpatentable over Cai et al. (U.S. Pub. No. 20230112921 A1) in view of Wu et al. “AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts”.
As per claim 21, Cai teaches the invention as claimed including a computer implemented method, comprising:
exposing an interface to a prompt generation processor in a generative artificial intelligence (AI) development system (par. 0002 transparent and controllable human-AI interaction via chaining of machine-learned language models, including … a graphical user interface for modularly building and/or editing a model chain; par. 0033 FIGS. 2A-C depict an example interactive interface for interacting with model chains according to example embodiments of the present disclosure);
receiving through the exposed interface a prompt generation input (par. 0059 a split points’ step/prompt that extracts each individual presentation problem from the original feedback [prompt generation input]; par. 0082, In FIG. 1B … the input of Feedback);
accessing an editable prompt template with a plurality of prompt placeholders (par. 0083 Table 2, Example implementation for Ideation in Table 1 with (1) a prompt template that involves the task description, datatypes, and placeholders for inputs and outputs; par. 0013 the respective prompt to each model instantiation in the model chain is user-selectable from a number of pre-defined template prompts);
detecting user interaction with the editable prompt template (par. 0093 Users frequently [interact] make local fixes on intermediate data points that flow between model steps, and therefore example interfaces can allow in-place editing, without explicitly switching to editing mode; par. 0086 a user can build concrete model steps (prompts therefor) simply by filling in the templates with data layer definitions);;
populating plurality of prompt placeholders within the editable prompt template with a … prompts generated to correspond to a generative AI request, for a generative AI model, based on the user interaction (par. 0083 Table 2, Example implementation for Ideation in Table 1 with (1) a prompt template that involves the task description, datatypes, and placeholders for inputs and outputs; par. 0060 the separate suggestion Ideation step in FIG. 1B, chaining allows users to customize which suggestions to include in the final paragraph; Fig. 1B, IDEATION; par. 0086 a user can build concrete model steps (prompts therefor) simply by filling in the templates with data layer definitions);
providing the populated editable prompt template to a generative AI model application programming interface (API) … (par. 0086 The Ideation example in Table 1 fulfills the template with two data layers: (Alex's problem, Alex's presentation problem) and (short suggestions for improvement, suggestions) ; par. 0011 the calls … can be made to the models 190 using one or more application programming interfaces (APIs)); and
receiving a response from a generative AI model through the generative AI model API (par. 0059 third, a ‘compose points’ step/prompt that synthesizes all the problems and suggestions into a final friendly paragraph. The result is noticeably improved; Fig. 1B, Friendly Paragraph).
Cai does not expressly disclose: populating … with a plurality of chained prompts; providing the populated editable prompt template to a generative AI model application programming interface (API) causing one or more generative AI models to execute the plurality of chained prompts in an order defined by the editable prompt template.
However, Wu teaches: populating …. with a plurality of chained prompts (page 6, left column, lines 28-30 The template allows us to build LLM steps simply by filling [populating] in the placeholders with definitions on data layers; page 2, lines 20-23 The interface visualizes the chain structure, and allows users to customize a chain [chained prompts] at various levels: they can iterate on the local prompts in each step, edit intermediate data between steps, or modify the entire chain);
providing the populated editable prompt template to a generative AI model application programming interface (API) causing one or more generative AI models to execute the plurality of chained prompts in an order defined by the editable prompt template (page 6, left column, line 34 – right column, line 2 We thus create prompt templates to support a wide range of scenarios, with placeholders for input and output data. The template allows us to build LLM steps simply by filling in the placeholders with definitions on data layers; page 6, right column, lines 31-32 We designed an interface that helps users execute and customize LLM Chains interactively; page 8, right column, lines 13-16 Underlying LLM. All of our experiments … and each step of the Chaining interface rely on exactly the same underlying LLM: LaMDA [63]2, a 137 billion parameter, general-purpose language model; Lines 29-33 It presents a single textbox with a run button, which allows the user to enter the text prompt, run the model on that prompt, and then view the model result in the same textbox, with the ability to edit that result and then continue to iterate; page 11, left column, lines 27-29 each step of a Chain involves a separate run of the model, Chaining allowed users to control certain aspects of each sub-task independent of others).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the technique of providing prompt templates and an chaining interface that allows users to enter edit prompt templates and run on a model of Wu with the system and method of Cai resulting in a system and method which provides for filling prompt templates with chined prompts and entering the filled/edited prompt templates to a model and run the model on the edited prompt templates as in Wu. One of ordinary skill in the art would have been motivated to make this combination for the purpose of helping users more fully capitalize on the model’s latent capabilities (page 10, left column, lines 20-12).
As per claim 28, Cai further teaches: accessing a set of prompts in a prompt library in the generative Al development system (par. 0013 In some implementations, the respective prompt to each model instantiation in the model chain is user-selectable from a number of pre-defined template prompts that correspond to primitive subtasks); generating the set of chained prompts based on at least one prompt in the prompt library; and storing the plurality of chained prompts in the prompt library (par. 0058 methods of the present disclosure can assist in resolving this issue by chaining multiple prompts together, so that the problem is broken down into a number of smaller sub-task; par. 0086 a user can build concrete model steps (prompts therefor) simply by filling in the templates with data layer definitions; par. 0101 users can be enabled to select from a number of pre-defined prompts … Likewise, pre-defined or default chains can be used as a starting place for various tasks as well. It is noted that the predefined chains are implied as necessarily being stored in a storage).
As per claim 29, Cai further teaches: wherein accessing a set of prompts comprises: searching the prompt library for prompts based on the prompt generation input to identify the set of prompts; generating an interface with selectable prompt identifiers corresponding to the identified set of prompts; and detecting a user selection input selecting one of the selectable prompt identifiers (par. 0045 users can be enabled to select from a number of pre-defined prompts. For example, each pre-defined prompt can correspond to a primitive operation that includes default prompting and data structures; par. 0086 a user can build concrete model steps (prompts therefor) simply by filling in the templates with data layer definitions; par. 0089 This section describes interactive interfaces which support users in interacting with model chains, including modifying the prompts and intermediate model outputs for each step, and customizing the Chains).
As per claim 30, Cai further teaches: wherein populating an editable prompt template comprises: retrieving a selected prompt corresponding to the selected prompt identifiers; and populating the editable prompt template with the selected prompt (par. 0083 Table 2, Example implementation for Ideation in Table 1 with (1) a prompt template that involves the task description, datatypes, and placeholders for inputs and outputs; par. 0086 Then, a user can build concrete model steps (prompts therefor) simply by filling in the templates with data layer definitions), Wu further teaches: wherein the selected prompt includes a set of chained prompts (page 6, left column, lines 28-30 The template allows us to build LLM steps simply by filling [populating] in the placeholders with definitions on data layers; page 2, lines 20-23 The interface visualizes the chain structure, and allows users to customize a chain [chained prompts] at various levels) and wherein generating a plurality of chained prompts comprises: detecting user interaction with the set of chained prompts in the selected prompt to generate the plurality of chained prompts (page 7, left column, lines 12-14 We observed [detect] users frequently making local fixes on intermediate data points that flow between LLM steps, and therefore designed the UI to allow in-place editing).
As per claim 32, Cai further teaches: generating an evaluation interface with the generative Al development system, the evaluation interface including the plurality of chained prompts and the response (par. 0059 in FIG. 1B, a LLM chain is used that includes … an ‘ideation’ step/prompt that brainstorms suggestions per problem; and third, a ‘compose points’ step/prompt that synthesizes all the problems and suggestions into a final friendly paragraph).
As per claim 34, Cai further teaches: causing display of the evaluation interface with the plurality of chained prompts and the response for manual evaluation (par. 0059 in FIG. 1B, a LLM chain is used that includes … an ‘ideation’ step/prompt that brainstorms suggestions per problem; and third, a ‘compose points’ step/prompt that synthesizes all the problems and suggestions into a final friendly paragraph; par. 0013 the respective prompt to each model instantiation in the model chain is user-selectable from a number of pre-defined template prompts).
As per claim 39, Cai teaches a computing system (Fig. 6A, computer system 100), comprising:
at least one processor (Fig. 6A, Processor 132); and
memory that stores computer executable instructions which, when executed by the at least one processor (Fig. 6A, Memory 136; par. 00115 The memory 134 can store data 136 and instructions 138 which are executed by the processor 132), cause the at least one processor to perform steps comprising:
exposing an interface to a prompt generator in an artificial intelligence (Al) development system (par. 0002 transparent and controllable human-AI interaction via chaining of machine-learned language models, including … a graphical user interface for modularly building and/or editing a model chain; par. 0033 FIGS. 2A-C depict an example interactive interface for interacting with model chains according to example embodiments of the present disclosure);
receiving through the exposed interface a prompt generation input (par. 0059 a split points’ step/prompt that extracts each individual presentation problem from the original feedback [prompt generation input]; par. 0082, In FIG. 1B … the input of Feedback);
accessing a memory storing a set of prompts (par. 0013 pre-defined template prompts; 0045 users can be enabled to select from a number of pre-defined prompts);
generating a plurality of Al prompts for a generative AI model, based on the prompt generation input and a prompt in the prompt memory (par. 0083 Table 2, Example implementation for Ideation in Table 1 with (1) a prompt template that involves the task description, datatypes, and placeholders for inputs and outputs; par. 0060 the separate suggestion Ideation step in FIG. 1B, chaining allows users to customize which suggestions to include in the final paragraph; Fig. 1B, IDEATION; par. 0086 a user can build concrete model steps (prompts therefor) simply by filling in the templates with data layer definitions).
calling a generative Al model accessing layer … (par. 0113 For example, the calls (e.g., requests for inference) can be made to the models 190 using one or more application programming interfaces (APIs)).
Cai does not expressly disclose calling a generative Al model accessing layer to send the editable prompt template to the generative Al model causing the generative AI model to execute the plurality of AT prompts in an order defined by the editable prompt template; and receiving a response from the generative AI model through the generative AI model accessing layer.
However, Wu teaches: populating an editable prompt template with the plurality of AI prompts (page 6, left column, lines 28-30 The template allows us to build LLM steps simply by filling [populating] in the placeholders with definitions on data layers; page 2, lines 20-23 The interface visualizes the chain structure, and allows users to customize a chain [chained prompts] at various levels: they can iterate on the local prompts in each step, edit intermediate data between steps, or modify the entire chain);
calling a generative Al model accessing layer to send the editable prompt template to the generative Al model causing the generative AI model to execute the plurality of AT prompts in an order defined by the editable prompt template (page 3, left column, lines 28-30 The template allows us to build LLM steps simply by filling [populating] in the placeholders with definitions on data layers; page 2, lines 20-23 The interface visualizes the chain structure, and allows users to customize a chain [chained prompts] at various levels: they can iterate on the local prompts in each step, edit intermediate data between steps, or modify the entire chain).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the technique of providing prompt templates and an chaining interface that allows users to enter edit prompt templates and run on a model of Wu with the system and method of Cai resulting in a system and method which provides for filling prompt templates with chined prompts and entering the filled/edited prompt templates to a model and run the model on the edited prompt templates as in Wu. One of ordinary skill in the art would have been motivated to make this combination for the purpose of helping users more fully capitalize on the model’s latent capabilities (page 10, left column, lines 20-12).
As per claim 40, Cai further teaches: generating, as the AI prompt, a plurality of sequential prompts, corresponding to the generative Al request for the generative Al model, based on the prompt generation input and the prompt in the prompt memory (par. 0041 the provided user interface can enable the user to: construct and/or edit a new or existing model chain and/or view and edit the inputs, outputs, and/or prompts for each instantiation within the chain; par. 0043 Thus, the present disclosure introduces the notion of “chaining” multiple language model instantiations together across a number of different model prompts … In a chain, a problem can be broken down into a number of smaller sub-tasks, each mapped to a distinct step with a corresponding prompt).
Claims 22 and 37 are rejected under 35 U.S.C. 103 as being unpatentable over Cai and Wu, and further in view of Gobran et al. (U.S. Pub. No. 20240054546 A1).
Gobran was cited in previous office action.
As per claim 22, Cai further teaches: receiving, through the exposed interface, a context data extraction input defining … context data to extract from one or more … systems for sending to the Al model API with the chained prompts (par. 0069, Table 1, b. Factual Query; Table 1, c. Info. Extraction: Extraction information from the context, Ex. Given text, extract airport codes per city text: I want to fly from Los Angeles to Miami airport codes: LAX, MIA; par. 0024 providing, for display within the user interface, the data indicative of the respective model output of the one or more of the model instantiations comprises providing the user interface in a chain view mode that depicts a structure of the model chain).
Cai and Wu do not expressly describe: user-related context data.
However, Gobran teaches: user-related context data (par. 0025 The user context can be obtained from one or more sub-systems; par. 0026 the user context can be provided to a machine learning model that has been trained to receive user context information as input).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the technique of obtaining user context data and providing to a ML model of Gobran with the system and method of Cai and Wu resulting in system and method that provides transparent and controllable human-AI interaction that includes obtaining/extracting user-context data for providing to an AI model as in Gobran. A person of ordinary skill would have been motivated to make this combination for the purpose of permitting a model to adapt over time to user preferences within a given context (par. 0010).
As per claim 37, Cai teaches the invention substantially as claimed including a generative artificial intelligence (AI) development system (par. 0002 transparent and controllable human-AI interaction via chaining of machine-learned language models, including … a graphical user interface for modularly building and/or editing a model chain), comprising at least one processor; and memory that stores computer executable instructions (par. 119 one or more processors 152 and a memory 154 … The memory 154 can store data 156 and instructions 158) which, when executed by the at least one processor, cause the at least one processor to perform operations comprising:
exposing an AI development interface to receive generative Al system development user inputs (par. 0033 FIGS. 2A-C depict an example interactive interface for interacting with model chains according to example embodiments of the present disclosure);
receiving an AI prompt generation user input from the AI development interface and generate a plurality of Al prompts based on the AI prompt generation user input (par. 0059 a split points’ step/prompt that extracts each individual presentation problem from the original feedback [prompt generation input]; par. 0082, In FIG. 1B … the input of Feedback; par. 0041 the provided user interface can enable the user to: construct and/or edit a new or existing model chain and/or view and edit the inputs, outputs, and/or prompts for each instantiation within the chain; par. 0043 Thus, the present disclosure introduces the notion of “chaining” multiple language model instantiations together across a number of different model prompts … In a chain, a problem can be broken down into a number of smaller sub-tasks, each mapped to a distinct step with a corresponding prompt);
a data extraction system configured to extract extracting … context data from a … system based on a user data extraction input identifying the … context data (par. 0069, Table 1, b. Factual Query; Table 1, c. Info. Extraction: Extraction information from the context).
calling a generative AI model application programming interface (API) … (par. 0113 For example, the calls … can be made to the models 190 using one or more application programming interfaces (APIs)).
Cai does not expressly disclose: populating an editable prompt template with the plurality of AI prompts and the user-related context data; and calling a generative AI model application programming interface (API) to send the editable prompt template to a generative AI model causing the generative Al model to execute the plurality of AI prompts in an order defined by the editable prompt template and to receive a response from the generative AI model.
However, Wu teaches: populating an editable prompt template with the plurality of AI prompts and the user-related context data (page 6, left column, lines 28-30 The template allows us to build LLM steps simply by filling [populating] in the placeholders with definitions on data layers; page 2, lines 20-23 The interface visualizes the chain structure, and allows users to customize a chain [chained prompts] at various levels: they can iterate on the local prompts in each step, edit intermediate data between steps, or modify the entire chain); and
calling a generative AI model application programming interface (API) to send the editable prompt template to a generative AI model causing the generative Al model to execute the plurality of AI prompts in an order defined by the editable prompt template and to receive a response from the generative AI model (page 6, left column, line 34 – right column, line 2 We thus create prompt templates to support a wide range of scenarios, with placeholders for input and output data. The template allows us to build LLM steps simply by filling in the placeholders with definitions on data layers; page 6, right column, lines 31-32 We designed an interface that helps users execute and customize LLM Chains interactively; page 8, right column, lines 13-16 Underlying LLM. All of our experiments … and each step of the Chaining interface rely on exactly the same underlying LLM: LaMDA [63]2, a 137 billion parameter, general-purpose language model; Lines 29-33 It presents a single textbox with a run button, which allows the user to enter the text prompt, run the model on that prompt, and then view the model result in the same textbox, with the ability to edit that result and then continue to iterate; page 11, left column, lines 27-29 each step of a Chain involves a separate run of the model, Chaining allowed users to control certain aspects of each sub-task independent of others).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the technique of providing prompt templates and an chaining interface that allows users to enter edit prompt templates and run on a model of Wu with the system and method of Cai resulting in a system and method which provides for filling prompt templates with chined prompts and entering the filled/edited prompt templates to a model and run the model on the edited prompt templates as in Wu. One of ordinary skill in the art would have been motivated to make this combination for the purpose of helping users more fully capitalize on the model’s latent capabilities (page 10, left column, lines 20-12).
Cai and Wu do not expressly describe: user-related context data.
However, Gobran teaches user-related context data (par. 0025 The user context can be obtained from one or more sub-systems; par. 0026 the user context can be provided to a machine learning model that has been trained to receive user context information as input).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the technique of obtaining user context data and providing to a ML model of Gobran with the system and method of Cai and Wu resulting in system and method that provides transparent and controllable human-AI interaction that includes obtaining/extracting user-context data for providing to an AI model as in Gobran. A person of ordinary skill would have been motivated to make this combination for the purpose of permitting a model to adapt over time to user preferences within a given context (par. 0010).
Claims 23-25 are rejected under 35 U.S.C. 103 as being unpatentable over Cai in view of Wu and Gobran, and further in view of Saxe et al. (U.S. Pub. No. 20230315722 A1).
Saxe was cited in previews officiation.
As per claim 23, Cai further teaches: receiving, through the exposed interface, an … data extraction input defining … data, that the generative Al model processes in response to the plurality of chained prompts, to extract from the one or more … systems for sending to the Al model API with the chained prompts (par. 0069, Table 1, b. Factual Query; Table 1, c. Info. Extraction: Extraction information from the context, Ex. Given text, extract airport codes per city text: I want to fly from Los Angeles to Miami airport codes: LAX, MIA; par. 0024 providing, for display within the user interface, the data indicative of the respective model output of the one or more of the model instantiations comprises providing the user interface in a chain view mode that depicts a structure of the model chain).
Cai, Wu and Gobran do not expressly describe: user-related augmented data
However, Saxe teaches: data extraction input defining user-related augmented data (par. 0025 The processor is configured to receive, via an interface, natural language data associated with a user request; par. 0033 the NL system is configured to receive, via the NL interface, user provided corrections and provide the corrections to augment the training data used to train the ML model. Using the augmented training data, including the user provided corrections, the ML model can be retrained to improve its performance in predicting intent of the user, and generating the template query to match or be closer to the user's intent when provided with the natural language request).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the technique of receiving via an NL interface NL dada, user provided corrections to augment training data and us it to train an ML model of Saxe with the system and method of Cai, Wu and Gobran resulting in system and method which provides for receiving or extracting user-augmented data in order to train an AI model as in Saxe. A person of ordinary skill would have been motivated to make this combination for the purpose of improving its performance in predicting intent of the user (par. 0033).
As per claim 24, Cai further teaches: wherein exposing an interface comprises: exposing a generative Al development environment creation interface (par. 0002 transparent and controllable human-AI interaction via chaining of machine-learned language models, including … a graphical user interface for modularly building and/or editing a model chain that includes a sequence of instantiations of one or more machine-learned language models; par. 0033 FIGS. 2A-C depict an example interactive interface for interacting with model chains according to example embodiments of the present disclosure); receiving a development environment creation input through the generative AI development environment creation interface (par. 0023 The method includes receiving an initial language input. The method includes providing a user interface that visualizes and enables a user to edit a model chain configured to process the initial language input to generate a language output); and assigning computer processing resources, including memory, to a generative AI development environment based on the development environment creation input (par. 0051 computational resources such as processor time, memory usage, network bandwidth, etc. In particular, in the case of a LLM, even a single re-training can consume a very significant amount of resources).
As per claim 25, Gobran further teaches: extracting the context data … from the one or more user-related systems (par. 0025 The user context can be obtained from one or more sub-systems; par. 0026 the user context can be provided to a machine learning model that has been trained to receive user context information as input; par. 0005 the method can further include obtaining updated user context data); and storing the extracted context data … in the memory assigned to the generative Al development input (par. 0065 the user context data can be stored in the user device only and deleted once the data has been used to make a content suggestion or automatic provision … the user context data can be processed and/or stored). Saxe further teaches: extracting the context data and the augmented data (par. 0025 The processor is configured to receive, via an interface, natural language data associated with a user request; par. 0033 the NL system is configured to receive, via the NL interface, user provided corrections and provide the corrections to augment the training data used to train the ML model. Using the augmented training data, including the user provided corrections, the ML model can be retrained to improve its performance in predicting intent of the user, and generating the template query to match or be closer to the user's intent when provided with the natural language request).
Claim 26 is rejected under 35 U.S.C. 103 as being unpatentable over Cai in view of Wu, Gobran and Saxe, and further in view of Hoffman et al. (U.S. Pub. No. 20110047176 A1).
Hoffman was cited in previous office action.
As per claim 26 Gobran further teaches: wherein receiving the context data extraction input … (par. 0025 The user context can be obtained from one or more sub-systems; par. 0026 the user context can be provided to a machine learning model that has been trained to receive user context information as input; par. 0005 the method can further include obtaining updated user context data).
Saxe further teaches: receiving the augmented data extraction input … receiving the augmented data extraction input … (par. 0025 The processor is configured to receive, via an interface, natural language data associated with a user request; par. 0033 the NL system is configured to receive, via the NL interface, user provided corrections and provide the corrections to augment the training data used to train the ML model. Using the augmented training data, including the user provided corrections, the ML model can be retrained to improve its performance in predicting intent of the user, and generating the template query to match or be closer to the user's intent when provided with the natural language request; par. 0048 The data can include context specific training data).
Cai, Wu, Gobran and Saxe do not expressly describe: receiving context data extraction script … augmented data extraction script, wherein extracting the context data and the augmented data comprises; executing the context data extraction script; and executing the augmented data extraction script.
However, Hoffman teaches: receiving context data extraction script … wherein extracting the … data comprises: executing the context data extraction script; and executing the augmented data extraction script (par. 0039 Initially, extraction scripts are generated, and may be generated by the central system 210 or a third party not associated with the central system 210. These extraction scripts are installed or downloaded onto the site system ... As used herein, extraction scripts are software that, once executed, automatically extracts various types of data; par. 0039 generally, the site system 212 receives and executes extraction scripts that extract summary data [metadata/context data] from site databases).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the technique of using extraction scripts to extract various types of data of Hoffman with the system and method of Cai, Wu, Gobran and Saxe resulting in a transparent and controllable human-AI interaction system and method that provides for receiving/downloading scripts and executing the scripts of Hoffman for extracting context and/or augmented data as in Hoffman. A person of ordinary skill in the art would have been motivated to make this combination for the purpose of automatically extracts various [context, augmented] types of data from the site system (par. 0039).
Claim 33 and 35 are rejected under 35 U.S.C. 103 as being unpatentable over Cai in view of Wu, and further in view of Zha et al. (U.S. Pub. No. 20240202458 A1).
As per claim 33, Cai further teaches: processing the plurality of chained prompts and response (par. 0058 chaining multiple prompts together) with a prompt-valuation generative Al model.
Cai and Wu do not expressly describe: to identify evaluation metrics and metric values for the evaluation metrics, the evaluation metrics and metric values being indicative of a performance of the plurality of chained prompts.
However, Zha teaches: identify evaluation metrics and metric values for the evaluation metrics, the evaluation metrics and metric values being indicative of a performance of the plurality of … prompts (par. 0046 Prompt discovery 224 may implement prompt and NLP ML evaluation 430, in various embodiments, in order to evaluate performance of the candidate prompt(s) and candidate NLP ML model(s) … In some embodiments, test data 435 may be maintained by machine learning service 210 … When the test data 435 is obtained, the candidate prompts 432 may be used to generate inferences using the candidate NLP ML model(s) 433 on the test data 435. Results 434 for candidate prompts may be collected. In some embodiments, prompt and NLP ML evaluation 430 may perform an initial analysis by, for example, comparing candidate prompt results 434 sample output 413. Again, a similarity score with sample output 413 may be generated to determine how well candidate prompts and candidate NLP ML models performed, and used to rank or filter out candidate prompt(s) and models).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the technique of evaluating performance of prompts and ML models of Zha with the system and method of Cai and Wu resulting in a transparent and controllable human-AI interaction system and method that provides evaluating/identifying the performance of chained prompts and models as in Zha. A person of ordinary skill in the art would have been motivated to make this combination for the purpose of providing for discovering prompts for different NLP tasks can help to optimize the performance of an NLP task for integration with a particular application. (par. 0027). Further, the combination would enhance the capabilities of systems, services, or applications to better interact with human users (par. 0020).
As per claim 35, Cai further teaches: processing the plurality of chained prompts and response (par. 0058 chaining multiple prompts together) with a model-evaluation generative AI model; generating an additional response to the plurality of chained prompts with an additional generative AI model and comparing the response from the generative AI model with the additional response from the additional generative Al model to determine whether the generative Al model or the additional generative AI model performed better (par. 0042 As examples, users of the system can leverage sub-tasks to calibrate model expectations; compare and contrast alternative strategies by observing parallel downstream effects; par. 0047 . For example, users can: model expectations using the smaller scope of sub-tasks; explore alternative prompting strategies by comparing parallel downstream effects).
Cai and Wu do not expressly teach: to identify evaluation metrics and metric values for the evaluation metrics, the evaluation metrics and metric values being indicative of a performance of the generative Al model in generating a response to the plurality of chained prompts.
Zha further teaches: to identify evaluation metrics and metric values for the evaluation metrics, the evaluation metrics and metric values being indicative of a performance of the generative Al model in generating a response to the plurality of chained prompts (par. 0046 Prompt discovery 224 may implement prompt and NLP ML evaluation 430, in various embodiments, in order to evaluate performance of the candidate prompt(s) and candidate NLP ML model(s) … In some embodiments, test data 435 may be maintained by machine learning service 210 … When the test data 435 is obtained, the candidate prompts 432 may be used to generate inferences using the candidate NLP ML model(s) 433 on the test data 435. Results 434 for candidate prompts may be collected. In some embodiments, prompt and NLP ML evaluation 430 may perform an initial analysis by, for example, comparing candidate prompt results 434 sample output 413. Again, a similarity score with sample output 413 may be generated to determine how well candidate prompts and candidate NLP ML models performed, and used to rank or filter out candidate prompt(s) and models).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the technique of evaluating performance of prompts and ML models of Zha with the system and method of Cai and Wu resulting in a transparent and controllable human-AI interaction system and method that provides evaluating/identifying the performance of chained prompts and models as in Zha. A person of ordinary skill in the art would have been motivated to make this combination for the purpose of providing for discovering prompts for different NLP tasks can help to optimize the performance of an NLP task for integration with a particular application. (par. 0027). Further, the combination would enhance the capabilities of systems, services, or applications to better interact with human users (par. 0020).
Claim 38 is rejected under 35 U.S.C. 103 as being unpatentable over Cai in view of Wu and Gobran, and further in view of Zha et al. (U.S. Pub. No. 20240202458 A1).
As per claim 38, Cai further teaches: processing the plurality of chained prompts and response (par. 0058 chaining multiple prompts together) with a prompt-valuation generative Al model.
Cai, Wu and Gobran do not expressly disclose: to identify evaluation metrics and metric values for the evaluation metrics, the evaluation metrics and metric values being indicative of a performance of the plurality of chained prompts.
However, Zha teaches: to identify evaluation metrics and metric values for the evaluation metrics, the evaluation metrics and metric values being indicative of a performance of the plurality of … prompts (par. 0046 Prompt discovery 224 may implement prompt and NLP ML evaluation 430, in various embodiments, in order to evaluate performance of the candidate prompt(s) and candidate NLP ML model(s) … In some embodiments, test data 435 may be maintained by machine learning service 210 … When the test data 435 is obtained, the candidate prompts 432 may be used to generate inferences using the candidate NLP ML model(s) 433 on the test data 435. Results 434 for candidate prompts may be collected. In some embodiments, prompt and NLP ML evaluation 430 may perform an initial analysis by, for example, comparing candidate prompt results 434 sample output 413. Again, a similarity score with sample output 413 may be generated to determine how well candidate prompts and candidate NLP ML models performed, and used to rank or filter out candidate prompt(s) and models).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the technique of evaluating performance of prompts and ML models of Zha with the system and method of Cai, Wu and Gobran resulting in a transparent and controllable human-AI interaction system and method that provides evaluating/identifying the performance of chained prompts and models as in Zha. A person of ordinary skill in the art would have been motivated to make this combination for the purpose of providing for discovering prompts for different NLP tasks can help to optimize the performance of an NLP task for integration with a particular application. (par. 0027). Further, the combination would enhance the capabilities of systems, services, or applications to better interact with human users (par. 0020).
Claim 41 is rejected under 35 U.S.C. 103 as being unpatentable over Cai in view of Wu, and further in view of Shriberg et al. (U.S. Patent No. 12505854 B2).
As per claim 41, Cai and Wu do not expressly teach: identifying a generative AI model for executing at least one prompt of the plurality of chained prompts based on an estimated generation length associated with the at least one prompt.
However, Shriberg teaches identifying a generative AI model for executing at least one prompt of the plurality of chained prompts based on an estimated generation length associated with the at least one prompt (col. 45, lines 60-64 selecting a particular model or combination of models from said trained models depending at least on an input, wherein said input comprises a prompt used, a speaker's characteristics or metadata, a length of said input [length of a prompt]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the technique of selecting a particular model of a plurality of trained models at least base on a length of an input of Shriberg with the system and method of Cai and Wu resulting in a system and method which provides for selecting a particular Ai model of a plurality of AI models at least base on a length of an input as in Farshin. One of ordinary skill in the art would have been motivated to make this combination for the purpose of achieving a significant performance gain (col. 13, line 12). Further it would provide for optimizing costs, speed, and accuracy.
Claim 42 is rejected under 35 U.S.C. 103 as being unpatentable over Cai in view of Wu, and further in view of Chan et al. (U.S. Pub. No. 20200285856 A1).
As per claim 42, Cai and Wu do not expressly teach: identifying a generative AI model for executing at least one prompt of the plurality of chained prompts based on a processing load associated with execution of the at least one prompt.
However, Chan teaches identifying a generative AI model for executing at least one prompt of the plurality of chained prompts based on a processing load associated with execution of the at least one prompt (par. 0066 the system includes various AI models, such as DNN models available for processing the selected data context. As shown, the system includes Model 1 to Model Z for selection by the operator to use for the analytics. The different models have different accuracy performance. Depending on the need of the operator, a lower accuracy performance model may be selected to reduce processing load. On the other hand, if high accuracy is required, the high accuracy model, such as model 1 is selected for processing).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the technique of selecting a model for processing a data context based on accuracy performance to reduce processing load of Chan with the system and method of Cai and Wu resulting in system and method which provides for selecting an AI model for executing/processing a request/prompt based on processing load. One of ordinary skill in the art would have been motivated to make this combination for the purpose of reducing reduce computational and hardware cost when running complex workloads (par. 0065).
Claim 43 is rejected under 35 U.S.C. 103 as being unpatentable over Cai in view of Wu, and further in view of Tsushima et al. (U.S. Pub. No. 20240265686 A1).
As per claim 43, Cai and Wu do not expressly teach identifying a generative AI model for executing at least one prompt of the plurality of chained prompts based on available processing capability associated with the generative AI model.
However, Tsushima teaches identifying a generative AI model for executing at least one prompt of the plurality of chained prompts based on available processing capability associated with the generative AI model (par. 0055 In this embodiment, the model with the memory 10b having a larger capacity prepared for processing is selected as the learned model 25 for the processing of estimation).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the technique selecting a model with a memory having larger capacity for processing of Tsushima with the system of Cai and Wu resulting in a system and method which provides for selecting an AI model based on a memory capacity of the AI model as in Tsushima. One of ordinary skill in the art would have been motivated to make this combination for the purpose of preventing that the capacity of the memory of the model becomes insufficient (par. 0060).
Response to Arguments
Applicant's arguments with respect to claims 1, 37 and 39 have been considered but are moot in view of the new ground(s) of rejection.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Willy W. Huaracha whose telephone number is (571) 270-5510. The examiner can normally be reached on M-F 8:30-5:00pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Bradley Teets can be reached on (571) 272-3338. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WH/
Examiner, Art Unit 2195
/BRADLEY A TEETS/Supervisory Patent Examiner, Art Unit 2197
Prosecution Insights