DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This office action is responsive to the above application filed February 29, 2024.
Claims 1-20 are pending, all examined and rejected.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 7-9, 14-15, 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Slack, Dylan, et al. "Explaining machine learning models with interactive natural language conversations using TalkToModel." Nature Machine Intelligence 5.8 (2023): 873-883 in view of Nam, Daye, et al. "Using an llm to help with code understanding." Proceedings of the IEEE/ACM 46th international conference on software engineering. 2024.
With respect to claim 1,
Dylan discloses A computer-implemented method, comprising:
receiving a request for a generative artificial intelligence (AI) explanation of an application output associated with a user and an application; (See page 3, Fig. 1 Overview of TalkToModel: Instead of writing code, users have conversations with Talk ToModel as follows: (1) users supply natural language inputs. (2) the dialogue engine parses the input into an executable representation. (3) the execution engine runs the operations and the dialogue engine uses the results in its response. See page 4, table 1 A conversation about diabetes prediction, demonstrating the breadth of different conversation points the system can discuss. Also see page 2, paragraph 3 “For example, on a disease prediction task, users can ask “how important is BMI for the predictions?” or “so how would decreasing the glucose levels by ten change the likelihood of men older than twenty having the disease?” TalkToModel will respond by describing how, for instance, BMI is the most important feature for predictions, and decreasing glucose will decrease the chance by 20%. From there, users can engage further in the conversation by asking follow up questions like, “what if you instead increased glucose by ten for that group of men?” and TalkToModel use the context to accurately respond. Conversations with TalkToModel make model explainability straightforward because users can simply talk with the system in natural language about the model, and the system will generate useful responses.” – users send questions/send request to the AI model, and the AI model receive the question/request to generate the answer/explanation. The model also receiving requests to generate explanation from the outputs that are generated by the model or generates explanation about the output associated with user and an application.)
receiving user application session information associated with the user and the application output; (See Page 2 paragraph “From there, users can engage further in the conversation by asking follow up questions like, “what if you instead increased glucose by ten for that group of men?” and TalkToModel use the context to accurately respond.” See page 4, table 1. See page 12 Supporting context in dialog “User conversations with TalkToModel naturally include com plex conversational phenomena such as anaphora and ellipsis [26, 74, 27]. Meaning, conversations refer back to events earlier in the conversation (“what do you predict for them?”) or omit information that must be inferred from conversation (“Now show me for people predicted incorrectly.”).” – the application information (questions, inputs from users, and the previous responses from the model) are received by the ML/model and being used as context so the model can generates better answers.)
providing the user application session information to a knowledge engine; (See page 2, paragraph 4 “To support such rich conversations with TalkToModel, we introduce techniques for both language understanding and model explainability. First, we propose a dialogue engine that parses user text inputs (referred to as user utterances) into an SQL-like programming language using a large language model (LLM). The LLM performs the parsing by treating the task of translating user utterances into the programming language as a seq2seq learning problem, where the user utterances are the source and parses in the programming language are the targets [71]. In addition, the TalkToModel language combines operations for explanations, ML error analyses, data manipulation, and descriptive text into a single language capable of representing a wide-variety of potential conversation topics most model explainability needs (an overview of the different operations is provided in Table 2). To support the system adapting to any dataset and model, we introduce lightweight adaption techniques to fine-tune LLMs to perform the parsing, enabling strong generalization to new settings. Second, we introduce an execution engine that runs the operations in each parse. To reduce the burden of users deciding which explanations to run, we introduce methods that automatically select explanations for the user. In particular, this engine runs many explanations, compares their fidelities, and selects the most accurate ones.” – The user inputs/session information are provided to the LLM/knowledge engine to generates information needed for the explanation. The LLM is the “knowledge engine”. )
receiving from the knowledge engine a knowledge explanation; (See page 3, paragraph 1 “. Second, we introduce an execution engine that runs the operations in each parse. To reduce the burden of users deciding which explanations to run, we introduce methods that automatically select explanations for the user. In particular, this engine runs many explanations, compares their fidelities, and selects the most accurate ones.” – the LLM/knowledge engine runs explanations/generates explanations and selects the best explanation as output, used as input for other models.)
and providing the generative AI explanation for display within the application. (See page 4, table 1, A conversation about diabetes prediction, demonstrating the breadth of different conversation points the system can discuss. See page 34, section C, Interface Screenshot. – the explanation from the AI/model is displayed for user through the user interface.)
Dylan does not explicitly disclose generating a prompt for a generative AI model based on the knowledge explanation;
providing the prompt to the generative AI model;
receiving from the generative AI model the generative AI explanation;
Daye discloses generating a prompt for a generative AI model based on the knowledge explanation; (See page 1, column 2, paragraph 4, “GILT is capable of generating on-demand information while considering the user’s local code context, which we incorporate into the prompts provided to the LLM behind the scenes.” See page 3, column 1, “Users can also ask a specific question directly to the LLM via the input box (Figure 1 5 ). If no code is selected, the entire source code is used as context (Prompt); alternatively, the relevant code highlighted by the user is used (Prompt-context). The model will then answer the question with that code as context. GILT also allows users to probe the LLM by supporting conversational interaction (Prompt-followup). When previous LLM-generated responses exist, if a user does not highlight any lines from the code, the LLM generates a response with the previous conversion as context.” – the previous explanation, response/output of the LLM is being used as input to generates the prompt for the LLM/AI model.)
providing the prompt to the generative AI model; (see page 3, column 1, “Users can also ask a specific question directly to the LLM via the input box (Figure 1 5 ). If no code is selected, the entire source code is used as context (Prompt); alternatively, the relevant code highlighted by the user is used (Prompt-context). The model will then answer the question with that code as context. GILT also allows users to probe the LLM by supporting conversational interaction (Prompt-followup). When previous LLM-generated responses exist, if a user does not highlight any lines from the code, the LLM generates a response with the previous conversion as context.” – Prompt is provided to the LLM/AI model.)
receiving from the generative AI model the generative AI explanation; (See page 3, column 1, “Users can also ask a specific question directly to the LLM via the input box (Figure 1 5 ). If no code is selected, the entire source code is used as context (Prompt); alternatively, the relevant code highlighted by the user is used (Prompt-context). The model will then answer the question with that code as context.” – the model answers the question/generates the explanation and the display system will receives the explanation to display to the user.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine LLMs generate explanation as output as disclosed by Dylan with LLM generates prompt using the outputs generated by the LLMs, providing the prompt to the LLM and receiving the output/explanation from the LLMs as disclosed by Daye. One would be motivated to do so to alleviate the cognitive load associated with writing prompts. [Daye, page 1, column 2]
With respect to claim 7,
As discussed with regard to claim 1, Dylan and Daye disclose all the limitations.
Daye further discloses the generative AI explanation comprises a user-specific explanation of the application output. (See page 3, Fig. 1. See page 1, column 2, paragraph 4 “GILT is capable of generating on-demand information while considering the user’s local code context, which we incorporate into the prompts provided to the LLM behind the scenes.” – the LLM/AI model generates explanation with information about user’s local code context/user-specific explanation because each user has different local code.)
It would have been obvious to a person having ordinary skill in the art before the effecting filing date of the claimed invention to include the user’s specific explanation of the output by the generative AI model as disclosed by Daye in the explanation from the generative AI model as disclosed by Daye. One would be motivated to do so because of the potential for generation-based information support directly in the developer’s programming environment. [Daye, page 1, column 2, paragraph 4]
With respect to claim 8,
As discussed with regard to claim 1, Dylan and Daye disclose all the limitations.
Daye further discloses the generative AI model is a large language machine learning model. (See page 3, column 1, paragraph 2 “GILT also allows users to probe the LLM by supporting conversational interaction (Prompt-followup). When previous LLM-generated responses exist, if a user does not highlight any lines from the code, the LLM gener ates a response with the previous conversion as context. Users can also reset the context by triggering the tool with code highlighted, or with the Clear all button”. See page 1 Introduction “In this study, we provide a first investigation of an LLM-based con versational UI built directly in the IDE that is geared towards code understanding” – The AI model is a large language machine (LLM))
It would have been obvious to a person having ordinary skill in the art before the effecting filing date of the claimed invention to combine the AI model is a LLM as disclosed by Daye with the generative AI model as disclosed by Dylan. One would be motivated to do so because LLM prompts can allow developers to provide more context, helping the models to receive more precise information and reduce time spent on searching the web for information. [Daye, page 1, column 2 paragraph 3]
With respect to claim 9,
The claim is similar in scope to claim 1 and is rejected under a similar rationale.
With respect to claim 14, 20,
These claims are similar in scope to claim 7 and are rejected under a similar rationale.
With respect to claim 15, 19,
These claims are similar in scope to claim 8 and are rejected under a similar rationale.
Claim(s) 2, 3, 5, 10, 11, 13, 16, 17 are rejected under 35 U.S.C. 103 as being unpatentable over Dylan and Daye in view of Konam, U.S P.G Pub 20230367973, published November 16, 2023, and Qiu, U.S P.G Pub 12670899, filed September 19, 2023.
With respect to claim 2,
As discussed with regard to claim 1, Dylan and Daye teach all the limitations.
Dylan and Daye do not explicitly disclose the knowledge explanation is in XML format
Konam discloses the knowledge explanation is in XML format. (See [0094] – the explanation generated by the explanation generator/LLM is in extensible markup language (XML) format.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the knowledge explanation generated by the knowledge engine as disclosed by Dylan with the explanation is in XML format as disclosed by Konam. One would be motivated to do so to allow the insertion (or removal) of the explanation into a text file for the transcript. [Konam, 0094].
Dylan, Daye, and Konam do not explicitly disclose the computer-implemented method further comprises parsing the knowledge explanation for information elements used in the prompt for the generative AI model.
Qiu discloses the computer-implemented method further comprises parsing the knowledge explanation for information elements used in the prompt for the generative AI model. (See (26), (95) – The output/knowledge explanation generated by the LLM is parsed into tasks/information elements for the prompt/prompt generation component. Truncating/parsing the input prompt can also generate information element)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the LLM generates output/knowledge explanation as disclosed by Dylan with output generated by the LLM is parsed into tasks/information elements for prompt as disclosed by Qiu. One would be motivated to do so the component will have less tokens to process, to increase the accuracy of the component and to determine task(s) to send to the task selection prompt generation component. [Qiu, (95)]
With respect to claim 3,
As discussed with regard to claim 2, Dylan, Daye, Qiu and Konam disclose all the limitations.
Qiu further discloses identifying one or more instructions based on the information elements; (See (95), (96) – the tasks/information elements being used as element to generate/identify prompt data. The prompt data can be the instructions for selecting the models.)
adding the one or more instructions to the prompt for the generative AI model. (See (95), (96) – the instruction is included in the prompt.)
With respect to claim 5,
As discussed with respect to claim 2, Dylan, Daye, Qiu and Konam disclose all the limitations.
Qiu further discloses the information elements are parsed by processing the explanation with a parser model trained to reduce a number of tokens of the prompt for the generative AI model. (See (26) – the information element got after truncating/parsing the prompt by the system/parser helps reducing the number of tokens of the prompt. The prompt/explanation is provided to the LLM for processing.)
With respect to claim 10, 17,
The claims are similar in scope to claim 2 and are rejected under a similar rationale.
With respect to claim 11,
The claim is similar in scope to claim 3 and is rejected under a similar rationale.
With respect to claim 13,
The claim is similar in scope to claim 5 and is rejected under a similar rationale.
With respect to claim 16,
The claim is similar in scope to claims 1, 3 and 5 and is rejected under similar rationales.
Claim(s) 4, 12, 18 are rejected under 35 U.S.C. 103 as being unpatentable over Dylan and Daye in view of Gan, Wensheng, Shicheng Wan, and S. Yu Philip. "Model-as-a-service (MaaS): A survey." 2023 IEEE International Conference on Big Data (BigData). IEEE, 2023.
With respect to claim 4,
As discussed with regard to claim 1, Dylan and Daye disclose all the limitations.
Daye further discloses the request is received via an API from a user interface (UI) component of the application. (See page 3, column 1, paragraph 1 “They can then explore further by clicking on buttons (Figure 1- 4 ) for API (API), domain-specific concepts (Concept), and usage examples (Usage), which provide more detailed explanations with preset prompts. The API button offers detailed explanations about the API calls used in the code, the Concept button provides domain-specific concepts that might be needed to understand the highlighted code fully, and the Usage button offers a code example involving API calls used in the highlighted code.” – the application includes calling API through the API button/sending request via an API. The API button is displayed in the UI of the application.)
Daye does not explicitly disclose the request is received via a graphql protocol.
Wensheng discloses the request is received via a graphql protocol. (See page 4, column 2, API and development tools: “PI and development tools are key technologies that make sure MaaS services are easy to use. MaaS platforms provide a set of APIs for developers to dumb down the use and integration of models. These APIs provide endpoints for making requests and receiving predictions or insights from the models. Correspondingly, development tools such as software development kits (SDKs), command-line interfaces (CLIs), RESTful APIs, and GraphQL are commonly used in MaaS implementations [53], [54], [55].” – LLMs, AI models receive the request via API such as graphql, so the request must get through graphql protocol.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the LLMs receive requests via an API from a user interface as disclosed by Daye with LLMs received requests from graphql API as disclosed by Wensheng. One would be motivated to do so to make it easier for developers to implement MaaS [Wensheng, Page 4, API and development tools]
With respect to claim 12, 18
The claims are similar in scope to claim 4 and are rejected under similar rationale.
Claim(s) 6 is rejected under 35 U.S.C. 103 as being unpatentable over Dylan and Daye in view of Jiang, Huiqiang, et al. "Llmlingua: Compressing prompts for accelerated inference of large language models." Proceedings of the 2023 conference on empirical methods in natural language processing. 2023.
With respect to claim 6,
As discussed with regard to claim 1, Dylan and Daye disclose all the limitations.
Dylan and Daye do not explicitly disclose generating the prompt for the generative AI model based on the explanation, comprises compressing the prompt.
Huiqiang discloses generating the prompt for the generative AI model based on the explanation, comprises compressing the prompt. (See page 3, figure. 1 Framework of the proposed approach LLMLingua. See page 9, column 2 “The method of compressing prompts has the po tential to enhance downstream task performance by compressing longer prompts and to improve the LLMs’s inference efficiency by compressing the KVcache.” – the small model/LLM acts as knowledge engine compresses the input and generates a prompt with lesser tokens for other AI models.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine LLM acts as knowledge engine that takes user input to generate prompt disclosed by Dylan with LLM as knowledge engine compresses input to generate prompt disclosed by Huiqiang. One would be motivated to do so to reduces the number of tokens of the prompt. [Huiqiang, page 3].
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL HOANG LE whose telephone number is (571)270-7292. The examiner can normally be reached Monday-Friday 8:00 am - 5 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Ell can be reached at 5712703264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/M.H.L./Examiner, Art Unit 2141
/MATTHEW ELL/Supervisory Patent Examiner, Art Unit 2141
