FINAL REJECTION, SECOND DETAILED ACTION
Status of Prosecution
The present application, 18/527,077 filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
The application was filed in the Office on Dec. 1, 2023. Applicant’s preliminary amendment received Feb. 8, 2024 is acknowledged and entered.
The Office mailed a non-final rejection, first action detailed action on May 22, 2026.
Applicant filed amended claims with accompanying remarks and arguments on July 21, 2026.
Claims 1-3 and 5-21 are pending and all are rejected. Claim 4 is cancelled by amendment. Claims 1, 11 and 20 are independent.
Status of Claims
Claim 4 is cancelled by amendment.
Claims 12-19 are objected to.
Claims 1-3 and 5-21 are rejected under 35 U.S.C. § 101 because the claimed invention is directed to an abstract idea without significantly more.
Claims 1-3 and 5-8, 11-17 and 20 are rejected under 35 USC. § 103 as being unpatentable over Miller, United States Patent Application Publication 2024/0654641 published on October 24, 2024, in view of Yang et al. (“Yang”), United States Patent Application Publication US 2024/0412029 published on Dec. 12, 2024.
Claims 9-10, 18-19 and 21 as being unpatentable over Miller in view of Yang in further view of Khemani et al., (“Khemani”), United States Patent Application Publication US 2025/0005292 published on Jan. 2, 2025.
Response to Remarks and Arguments
Examiner thanks Applicant for the submitted amendments and remarks and arguments.
Regarding the objections, Examiner maintains them as they appear to not all have been addressed.
Next, the arguments made regarding §101 subject matter eligibility are considered but not deemed persuasive as noted below.
Finally, regarding the prior art rejections, they have been adjusted accordingly based on Applicant’s amendments.
The claims stand rejected.
Objection
Claims 12-19 remain objected to as depending from claim 10. This leads to several antecedent basis issues and appears to be a typographical error and should instead depend from independent claim 11. Examination will proceed with this construction. Correction is required.
Claim Rejections – § 101 Subject Matter Eligibility
Claims 1-3 and 5-21 are rejected under 35 U.S.C. § 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding representative claim 1, at step 1, the claim recites a computer-implemented method, and therefore is a process, which is a statutory category of invention. See MPEP § 2106.03.
At step 2A, prong one, the claim recites a computer-implemented method for multimodal response generation through a virtual agent.
The following limitations are the abstract idea of mental processes. See MPEP § 2106.04(a)(2)(III)(D):
retrieving information related to an input received by the virtual agent,
generating a response corresponding to the input based on the retrieved information;
analyzing user characteristics and emotional state of the user, wherein the emotional state of the user is identified based on one or more sensor data;
generating a plurality of prompts based on user characteristics and the input; and
selecting a prompt from the plurality of prompts based on the user characteristics, the emotional state, and a context of a conversation;
rephrasing the response based on the plurality of prompts to generate a multimodal response;
determining one or more modalities based on user engagement and comprehension levels; and
generating a multimodal response based on the personalized response and the determined one or more output modalities.
Therefore, the claim recites at least one abstract idea per this part of the analysis.
At step 2A prong 2, the claim language is analyzed to determine whether it recites additional elements that integrate the judicial exception into a practical application. See MPEP § 2106.04(d).
The limitation: wherein the virtual agent employs an Artificial Intelligence (AI) model, is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use, specifically artificial intelligence systems. See MPEP §§ 2106.04(d), 2106.05(h).
Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claim is therefore directed to an abstract idea.
Next, at step 2B of the analysis, the claim is considered if it recites additional elements that amount to significantly more than the judicial exception. See MPEP § 2106.05.
As discussed above with respect to integration of the abstract idea into a practical application, the additional element do nothing more than linking the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(h).
Therefore, claim 1 is ineligible.
As to dependent claims 2-3, 5-10 and 21 the analysis of the parent claim is incorporated. In the step 2A, prong 2 analysis, the additional limitations are additional element that are steps under broadest reasonable interpretations, are additional elements that generally link the use of the judicial exception to a particular technological application. See MPEP § 2106.05(h). The claims are also ineligible.
As to independent claim 11, the analysis of claim 1 is incorporated. Where it differs is in the step one analysis, in which the system includes a processor and memory is a manufacture and thus statutory.
As to dependent claims 12-19, they are similarly rejected as to claims 2-10.
As to independent claim 20, the analysis of the claim 1 is incorporated. Where it differs is in the step 0 analysis, in which the computer program product is a manufacture and thus statutory.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. § 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
A.
Claims 1-3 and 5-8, 11-17 and 20 are rejected under 35 USC. § 103 as being unpatentable over Miller, United States Patent Application Publication 2024/0654641 published on October 24, 2024, in view of Yang et al. (“Yang”), United States Patent Application Publication US 2024/0412029 published on Dec. 12, 2024.
As to Claim 1, Miller teaches: A computer-implemented method for multimodal response generation through a virtual agent comprising:
retrieving information related to an input received by the virtual agent (Miller: par. 0122, a personal AI agent system [400] receives user input [402] and communicates it to a neural network [436]; par. 0177, relevant nodes and edges of a knowledge graph are retrieved based on the query), wherein the virtual agent employs an Artificial Intelligence (AI) model (Miller: par. 0122, the AI agent system is an AI one);
generating a response corresponding to the input based on the retrieved information (Miller: par. 0179, a response is generated);
analyzing user characteristics and emotional state of the user, wherein the emotional state of the user is identified based on one or more sensor data (Miller: par. 0085, user emotional responses are detected using sensors for heart rate, facial expressions, etc.);
determining one or more modalities based on user engagement and comprehension levels(Miller: par. 0085, data such as user cognitive performance with attention and other emotional responses may be captured for consideration); and
Miller further teaches: further comprising determining one or more modalities for generating the multimodal response based on user's engagement and comprehension levels
Miller may not explicitly teach: generating a plurality of prompts based on user characteristics, the emotional state and the input;
selecting a prompt from the plurality of prompts based on the user characteristics, the emotional state, and a context of a conversation; and
rephrasing the response based on the selected prompt to generate a personalized response;
generating a multimodal response based on the personalized response and the determined one or more output modalities.
Yang teaches in general concepts related to using sensors to collect data from the user to consider an original prompt and to infer the emotional state of a user to generate an augmented prompt (Yang: Abstract). Specifically, Yang teaches that a prompt, which may be multimodal is received from a user (Yang: pars. 0028-29, Fig. 2, prompt [214]). The sensors detect aspects of the user including contact and non-contact (Yang: par. 0030, sensors [216]). From the processed data of the sensor data, emotion of the user is determined (Yang: par. 0035, emotion [226]). An augmented prompt is generated and a response is received (Yang: par. 0038). The response may be in appropriate modalities for the user (Yang: par. 0039).
PNG
media_image1.png
558
724
media_image1.png
Greyscale
It would have been obvious to a person having ordinary skill in the art at a time before the effective filing date of the application to have modified the Miller disclosures and teachings by allowing for the generation of an augmented prompt that results in specific rephrased responses with different modalities that would then would be selected based on user characteristics or context as taught by Yang. Such a person would have been motivated to do so with a reasonable expectation of success to cater to the user’s particular needs and preferences in an enhanced manner for better user experience (Yang: par. 0040).
As to Claim 2, Miller and Yang teaches the limitations of claim 1.
Yang further teaches: transmitting the multimodal response to the user, wherein the multimodal response is transmitted to the user in one or more combinations of modalities comprising text, speech, visual elements, and gesture (Yang: par. 0039,” response 232 and presents the response to the user 210 using appropriate output modalities, such as text output on a display screen, audio output via a speaker, etc.”).
As to Claim 3, Miller and Yang teach the limitations of claim 1.
Miller further teaches: wherein the Al model is a Generative Al model (Miller: par. 0171, a generative AI agent system).
As to Claim 5, Miller and Yang teach the limitations of claim 1.
Miller further teaches: wherein the Al model continuously learns from historical interactions to upgrade reasoning and response framing, and dynamically adapt user's accustomed communication style (Miller: par. 0088, “the personal AI agent 302 gains a much deeper understanding of the user and can use historical context to find relevancy in any current user activity.”; par. 0104, a custom prompt based on relevant user data is then generated).
As to Claim 6, Miller and Yang teach the limitations of claim 1.
Miller further teaches: wherein the Al model employs a role-based approach following user-provided instructions and the plurality of prompts to generate the multimodal response (Examiner asserts that the system as taught by the combination of Miller and Dasher is one that ultimately is one of any “role” that provides instructions and prompts to generate the multimodal response).
As to Claim 7, Miller and Yang teach the limitations of claim 1.
Miller further teaches: wherein the AI model is trained to understand user emotions enabling generation of the multimodal response adaptive to user's emotional state (Miller: par. 0085, data such as user cognitive performance with attention and other emotional responses may be captured for consideration).
As to Claim 8, Miller and Yang teach the limitations of claim 1.
Miller further teaches: wherein the plurality of prompts facilitate personalization of the response in real-time (Miller: pars. 0116-17, the intent vector, which represents the intent of the user based on user characteristics, is refreshed and updated continuously for use in the AI system).
As to Claim 11, it is rejected for similar reasons as claim 1. Miller further teaches a processor, computer readable memory and storage to have instructions to perform the steps (Miller: pars. 0240-41).
As to Claim 12, it is rejected for similar reasons as claim 2.
As to Claim 13, it is rejected for similar reasons as claim 3.
As to Claim 14, it is rejected for similar reasons as claim 4.
As to Claim 15, it is rejected for similar reasons as claim 5.
As to Claim 16, it is rejected for similar reasons as claim 6.
As to Claim 17, it is rejected for similar reasons as claim 7.
As to Claim 20, it is rejected for similar reasons as claim 1 and 12.
B.
Claims 9-10, 18-19 and 21 are rejected under 35 USC. § 103 as being unpatentable over Miller, United States Patent Application Publication 2024/0654641 published on October 24, 2024, in view of Yang et al. (“Yang”), United States Patent Application Publication US 2024/0412029 published on Dec. 12, 2024 and in further view of Khemani et al., (“Khemani”), United States Patent Application Publication US 2025/0005292 published on Jan. 2, 2025.
As to Claim 9, Miller and Yang teach the limitations of claim 1.
Miller and Yang may not explicitly teach: storing a record of the input, the plurality of prompts, and the generated multimodal response for future reference and analysis.
Khemani teaches in general concepts related to automatically generating content using a prompt generation service and a large language model instance (Khemani: Abstract). Specifically, Khemani teaches that feedback of a reviewer may be used to further training of the LLM (Khemani: par. 0088); Feedback is solicited and used to supplement or modify different responses from the output engine in the future (Khemani: par. 0283). A transactional and historical log is stored to used to train the predictive model (Khemani::par. 0378).
It would have been obvious to a person having ordinary skill in the art at a time before the effective filing date of the application to have modified the Miller- Yang disclosures and teachings by utilizing a feedback system to store the various elements of the prompt and response as taught by Khemani. Such a person would have been motivated to do so with a reasonable expectation of success to allow for the improvement of the user experience based on user feedback and historical records.
As to Claim 10, Miller and Yang teach the limitations of claim 1.
Miller and Yang may not explicitly teach: monitoring a user feedback on the multimodal response;
adjusting subsequent prompts based on the user feedback; and
modifying a subsequent response based on the subsequent prompts.
Khemani teaches in general concepts related to automatically generating content using a prompt generation service and a large language model instance (Khemani: Abstract). Specifically, Khemani teaches that feedback of a reviewer may be used to further training of the LLM (Khemani: par. 0088); Feedback is solicited and used to supplement or modify different responses from the output engine in the future (Khemani: par. 0283).
It would have been obvious to a person having ordinary skill in the art at a time before the effective filing date of the application to have modified the Miller- Yang disclosures and teachings by utilizing a feedback system to adjust the future prompts based on the feedback as taught by Khemani. Such a person would have been motivated to do so with a reasonable expectation of success to allow for the improvement of the user experience based on user feedback.
As to Claim 18, it is rejected for similar reasons as claim 9.
As to Claim 19, it is rejected for similar reasons as claim 10.
As to Claim 21, Miller. Yang and Khemani teach the limitations of claim 10.
Yang further teaches: wherein the plurality of prompts facilitates personalization of the response in real-time (Yang: par. 0110, real-time feedback from the user on the quality of the response may be adjusted based on the detected additional states that are sensed while the prompts are ongoing).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JAMES T TSAI whose telephone number is (571)270-3916. The examiner can normally be reached M-F 8-5 Eastern.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Cesar Paula can be reached on (571)272-4128. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JAMES T TSAI/ Primary Examiner, Art Unit 2177
Prosecution Insights