Prosecution Insights
Last updated: November 04, 2026
Application No. 19/086,754

METHOD FOR NOTIFICATION AND ESCALATION BY DETECTING ABNORMAL USER DIMENSIONS BASED ON AUDIO ANALYSIS OF AUDIO STREAM

Non-Final OA §103§DOUBLEPATENT
Filed
Mar 21, 2025
Priority
Apr 01, 2024 — provisional 63/572,734
Examiner
KIM, JONATHAN C
Art Unit
2653
Tech Center
2600 — Communications
Assignee
Humach LLC
OA Round
1 (Non-Final)
74%
Grant Probability
Favorable
1-2
OA Rounds
10m
Est. Remaining
99%
With Interview

Examiner Intelligence

Grants 74% — above average
74%
Career Allowance Rate
276 granted / 373 resolved
+12.0% vs TC avg
Strong +38% interview lift
Without
With
+38.4%
Interview Lift
resolved cases with interview
Typical timeline
2y 5m
Avg Prosecution
20 currently pending
Career history
405
Total Applications
across all art units

Statute-Specific Performance

§101
20.3%
-19.7% vs TC avg
§103
49.7%
+9.7% vs TC avg
§102
11.9%
-28.1% vs TC avg
§112
10.3%
-29.7% vs TC avg
Black line = Tech Center average estimate • Based on career data from 373 resolved cases

Office Action

§103 §DOUBLEPATENT
DETAILED ACTION This Office Action is in response to the correspondence filed by the applicant on 3/21/2025. Notice of Pre-AIA or AIA Status The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Double Patenting The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the claims at issue are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969). A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on a nonstatutory double patenting ground provided the reference application or patent either is shown to be commonly owned with this application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b). The USPTO internet Web site contains terminal disclaimer forms which may be used. Please visit http://www.uspto.gov/forms/. The filing date of the application will determine what form should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to http://www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp. Claims 1-20 provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over Claims 1-20 of copending Application No. 19/086,800 in further view of MUKHERJEE (US 2012/0008754 A1). Although the claims, at issue are not identical, they are not patentably distinct from each other because the claims of the instant application are rejected as being unpatentable over the claims of the copending application. Please see below for the mapping in the table, where the bolded limitations indicate the corresponding limitations between the copending application and instant application. Instant application: 19/086,754 Copending application: 19/086,800 1. A computer-implemented method for providing support services, the method comprising: receiving, by a digital agent hosted at a server associated with a contact center over a network, a first audio stream from a user device of a user during an interactive session, the first audio stream spoken by the user to inquire about a product provided by a product provider as a first client of the contact center, wherein the contact center is configured to provide support services for a plurality of products provided by a plurality of clients via a plurality of communication channels, wherein each of the plurality of clients represents one of a product manufacturer, a product distributer, a product retailer, or a service provider of the product; invoking, by the digital agent, a voice analysis service to perform an audio analysis on the first audio stream to determine a plurality of dimensions associated with the user, the dimensions of the user including voice characteristics of the user, the voice characteristics including vocal pitch and speech patterns of the user; invoking, by the digital agent, a custom language model (CLM) on content of the first audio stream as an input to generate a response to the inquiry about the product, wherein the CLM was specifically trained and customized for the product provided by the product provider via machine learning; transmitting, by the digital agent, the response to the inquiry about the product to the user device over the network; storing the dimensions of the user in a user profile of the user in a storage device, wherein the dimensions of the user can be retrieved from the user profile and used to generate subsequent responses in response to subsequent inquiries from the user; examining the dimensions of the user to determine whether a first predetermined condition has been satisfied, including determining whether at least one of the dimensions cannot be ascertained; updating a notification attribute of the user profile, in response to determining the first predetermined condition has been satisfied; periodically examining the notification attribute to determine whether a second predetermined condition has been satisfied; and invoking an escalation process in response to determining the second predetermined condition has been satisfied, including transmitting an escalate message to a predetermined destination to allow the predetermined destination to evaluate potential abnormal dimensions of the user. 1. A computer-implemented method for providing support services, the method comprising: receiving, by a digital agent hosted at a server associated with a contact center over a network, a first audio stream from a user device of a user during an interactive session, the first audio stream spoken by the user to inquire about a product provided by a product provider as a first client of the contact center, wherein the contact center is configured to provide support services for a plurality of products provided by a plurality of clients via a plurality of communication channels, wherein each of the plurality of clients represents one of a product manufacturer, a product distributer, a product retailer, or a service provider of the product; invoking, by the digital agent, a voice analysis service to perform an audio analysis on the first audio stream to determine a plurality of dimensions associated with the user, the dimensions of the user including voice characteristics of the user, the voice characteristics including vocal pitch and speech patterns of the user; invoking, by the digital agent, a custom language model (CLM) on content of the first audio stream as an input to generate a response to the inquiry about the product, wherein the CLM was specifically trained and customized for the product provided by the product provider via machine learning; transmitting, by the digital agent, the response to the inquiry about the product to the user device over the network; storing the dimensions of the user in a user profile of the user in a storage device, wherein the dimensions of the user can be retrieved from the user profile and used to generate subsequent responses in response to subsequent inquiries from the user; examining the dimensions of the user to determine whether the dimensions of the user indicate a health trait concerning the user, wherein the health trait indicates a likelihood of a health issue associated with the user; and MUKHERJEE Par 51 MUKHERJEE Pars 60 and70 invoking an escalation process in response to the health trait, including transmitting an escalate message to a predetermined health facility to allow the health facility to evaluate the health trait, including arranging a medical staff to independently reach out to the user discussing the health trait. Other independent claims 12 and 20 are also similar to the independent claims 13 and 20 of the copending application. With respect to the dependent claims, each claim maps to a corresponding dependent claim of the copending application or are found within the scope of the independent claim. This is a provisional nonstatutory double patenting rejection because the patentably indistinct claims have not in fact been patented. Claims 1-20 provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over Claims 1-20 of copending Application No. 19/086,713 in further view of MUKHERJEE (US 2012/0008754 A1). Although the claims, at issue are not identical, they are not patentably distinct from each other because the claims of the instant application are rejected as being unpatentable over the claims of the copending application. Please see below for the mapping in the table, where the bolded limitations indicate the corresponding limitations between the copending application and instant application. Instant application: 19/086,754 Copending application: 19/086,713 1. A computer-implemented method for providing support services, the method comprising: receiving, by a digital agent hosted at a server associated with a contact center over a network, a first audio stream from a user device of a user during an interactive session, the first audio stream spoken by the user to inquire about a product provided by a product provider as a first client of the contact center, wherein the contact center is configured to provide support services for a plurality of products provided by a plurality of clients via a plurality of communication channels, wherein each of the plurality of clients represents one of a product manufacturer, a product distributer, a product retailer, or a service provider of the product; invoking, by the digital agent, a voice analysis service to perform an audio analysis on the first audio stream to determine a plurality of dimensions associated with the user, the dimensions of the user including voice characteristics of the user, the voice characteristics including vocal pitch and speech patterns of the user; invoking, by the digital agent, a custom language model (CLM) on content of the first audio stream as an input to generate a response to the inquiry about the product, wherein the CLM was specifically trained and customized for the product provided by the product provider via machine learning; transmitting, by the digital agent, the response to the inquiry about the product to the user device over the network; storing the dimensions of the user in a user profile of the user in a storage device, wherein the dimensions of the user can be retrieved from the user profile and used to generate subsequent responses in response to subsequent inquiries from the user; examining the dimensions of the user to determine whether a first predetermined condition has been satisfied, including determining whether at least one of the dimensions cannot be ascertained; updating a notification attribute of the user profile, in response to determining the first predetermined condition has been satisfied; periodically examining the notification attribute to determine whether a second predetermined condition has been satisfied; and invoking an escalation process in response to determining the second predetermined condition has been satisfied, including transmitting an escalate message to a predetermined destination to allow the predetermined destination to evaluate potential abnormal dimensions of the user. 1. A computer-implemented method for providing support services, the method comprising: receiving, by a digital agent hosted at a server associated with a contact center over a network, a first audio stream from a user device of a user during an interactive session, the first audio stream spoken by the user to inquire about a product provided by a product provider as a first client of the contact center, wherein the contact center is configured to provide support services for a plurality of products provided by a plurality of clients via a plurality of communication channels, wherein each of the plurality of clients represents one of a product manufacturer, a product distributer, a product retailer, or a service provider of the product; invoking, by the digital agent, a voice analysis service to perform an audio analysis on the first audio stream to determine a plurality of dimensions associated with the user, the dimensions of the user including voice characteristics of the user, the voice characteristics including vocal pitch and speech patterns of the user; invoking, by the digital agent, a custom language model (CLM) on content of the first audio stream as an input to generate a response to the inquiry about the product, wherein the CLM was specifically trained and customized for the product provided by the product provider via machine learning; transmitting, by the digital agent, the response to the inquiry about the product to the user device over the network; storing the dimensions of the user in a user profile of the user in a storage device, wherein the dimensions of the user can be retrieved from the user profile and used to generate subsequent responses in response to subsequent inquiries from the user MUKHERJEE Pars 12, 26, 59 MUKHERJEE Par 51 MUKHERJEE Pars 60 and70 MUKHERJEE Pars 60 and 62 selecting, by the digital agent, a live agent from a plurality of live agents based on the dimensions of the user, wherein each of the live agents is capable of speaking with different voice characteristics; and transmitting the interactive session to the selected live agent to allow the selected live agent to conduct a live session with the user device regarding the inquiry about the product, wherein the selected live agent is capable of speaking with a voice similar to the voice characteristics of the user. Other independent claims 12 and 20 are also similar to the independent claims 12 and 20 of the copending application. With respect to the dependent claims, each claim maps to a corresponding dependent claim of the copending application or are found within the scope of the independent claim. This is a provisional nonstatutory double patenting rejection because the patentably indistinct claims have not in fact been patented. Claims 1-20 provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over Claims 1-20 of copending Application No. 19/086,668 in further view of MUKHERJEE (US 2012/0008754 A1). Although the claims, at issue are not identical, they are not patentably distinct from each other because the claims of the instant application are rejected as being unpatentable over the claims of the copending application. Please see below for the mapping in the table, where the bolded limitations indicate the corresponding limitations between the copending application and instant application. Instant application: 19/086,754 Copending application: 19/086,668 1. A computer-implemented method for providing support services, the method comprising: receiving, by a digital agent hosted at a server associated with a contact center over a network, a first audio stream from a user device of a user during an interactive session, the first audio stream spoken by the user to inquire about a product provided by a product provider as a first client of the contact center, wherein the contact center is configured to provide support services for a plurality of products provided by a plurality of clients via a plurality of communication channels, wherein each of the plurality of clients represents one of a product manufacturer, a product distributer, a product retailer, or a service provider of the product; invoking, by the digital agent, a voice analysis service to perform an audio analysis on the first audio stream to determine a plurality of dimensions associated with the user, the dimensions of the user including voice characteristics of the user, the voice characteristics including vocal pitch and speech patterns of the user; invoking, by the digital agent, a custom language model (CLM) on content of the first audio stream as an input to generate a response to the inquiry about the product, wherein the CLM was specifically trained and customized for the product provided by the product provider via machine learning; transmitting, by the digital agent, the response to the inquiry about the product to the user device over the network; storing the dimensions of the user in a user profile of the user in a storage device, wherein the dimensions of the user can be retrieved from the user profile and used to generate subsequent responses in response to subsequent inquiries from the user; examining the dimensions of the user to determine whether a first predetermined condition has been satisfied, including determining whether at least one of the dimensions cannot be ascertained; updating a notification attribute of the user profile, in response to determining the first predetermined condition has been satisfied; periodically examining the notification attribute to determine whether a second predetermined condition has been satisfied; and invoking an escalation process in response to determining the second predetermined condition has been satisfied, including transmitting an escalate message to a predetermined destination to allow the predetermined destination to evaluate potential abnormal dimensions of the user. 1. A computer-implemented method for providing support services, the method comprising: receiving, by a digital agent hosted at a server associated with a contact center over a network, a first audio stream from a user device of a user, the first audio stream spoken by the user to inquire about a product provided by a product provider as a first client of the contact center, wherein the contact center is configured to provide support services for a plurality of products provided by a plurality of clients via a plurality of communication channels, wherein each of the plurality of clients represents one of a product manufacturer, a product distributer, a product retailer, or a service provider of the product; performing an audio analysis on the first audio stream to determine a plurality of dimensions associated with the user, the dimensions of the user including voice characteristics of the user, the voice characteristics including vocal pitch and speech patterns of the user; performing a content analysis on content produced by the first audio stream, including: invoking a custom language model (CLM) on the content of the first audio stream as an input, wherein the CLM was specifically trained and customized for the product provided by the product provider via machine learning, and generating a response to the inquiry about the product based on an output of the CLM; generating a second audio stream based on the response and the dimensions of the user, such that at least some of voice characteristics of the second audio stream are similar to the voice characteristics of the user; and transmitting the second audio stream to the user device over the network. MUKHERJEE Pars 12, 26, 59 MUKHERJEE Par 51 MUKHERJEE Pars 60 and70 MUKHERJEE Pars 60 and 62 . Other independent claims 12 and 20 are also similar to the independent claims 12 and 20 of the copending application. With respect to the dependent claims, each claim maps to a corresponding dependent claim of the copending application or are found within the scope of the independent claim. This is a provisional nonstatutory double patenting rejection because the patentably indistinct claims have not in fact been patented. Claim Rejections - 35 USC § 103 The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. Claims 1-3, 5-7, 12-14, 16-18, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over RULE (US 2020/0211532 A1), and in further view of MUKHERJEE (US 2012/0008754 A1) and JADHAV (US 2025/0148209 A1). REGARDING CLAIM 1, RULE discloses a computer-implemented method for providing support services, the method comprising: receiving, by a digital agent hosted at a server associated with a contact center over a network, a first audio stream from a user device of a user during an interactive session, the first audio stream spoken by the user (RULE – “Par 29 – “The term “request” refers to any type of communication data transmitted from the user to the system. For example, the request may include an audio or video communication from a user.”) to inquire about a product provided by a product provider as a first client of the contact center, wherein the contact center is configured to provide support services for a plurality of products (RULE Par 102 – “The first request of opening the checking account may be handled by a first VA and the second request of checking the balance in the savings account may be handled by a second VA. Another example may be related to user communications related to purchasing goods or services. For example, the user may purchase power tools and cosmetics at the department store. The first VA may be configured to answer questions related to power tools and the second VA may be configured to answer questions related to cosmetics. For example, the first VA may have a voice of a thirty-five-year-old male, and the second VA may have a voice of a twenty-year-old female.”) provided by a plurality of clients via a plurality of communication channels (RULE Fig. 1; Par 38 – “Server system 135 may be configured to receive the request from users 102A-102C via network 115 and server 180. Network 115 may include any combination of electronic communication networks enabling communication between user devices and the components of the interaction system.”), wherein each of the plurality of clients represents one of a product manufacturer, a product distributer, a product retailer, or a service provider of the product (RULE Par 102 – “For example, the user may purchase power tools and cosmetics at the department store. The first VA may be configured to answer questions related to power tools and the second VA may be configured to answer questions related to cosmetics. For example, the first VA may have a voice of a thirty-five-year-old male, and the second VA may have a voice of a twenty-year-old female.”); invoking, by the digital agent, a voice analysis service to perform an audio analysis on the first audio stream to determine a plurality of dimensions associated with the user, the dimensions of the user including voice characteristics of the user, the voice characteristics including vocal pitch and speech patterns of the user (RULE Par 66 –"Data analysis module 130 may further include an audio/video/image module 424 for processing the audio video and image data, and for analyzing audio characteristics associated with the user request. In various embodiments, audio/video/image module 424 may analyze a pitch, a tone, a cadence of the user speech, volume and a rate of a user speech to extract various characteristics from the speech. For example, audio/video/image module 424 may identify a gender of the user, an age of the user, a dialect of the user, an accent of the user speech, a tone of voice, an emotional content of the speech, or any other aspects that uniquely identify the user based on the audio video or image characteristics of the receiving data. In some embodiments, when the user is communicating with interaction system 105 using video data, audio/video/image module 424 may identify the characteristics of the user (e.g., age, gender, ethnicity, hand gestures, facial expressions, body movements, and the like) associated with the video streaming data.”); invoking, by the digital agent, a custom language model (CLM) on content of the first audio stream (RULE Par 61 – “In some embodiments, the comprehension system may attempt to determine the type of the user request based on natural language processing using various algorithms developed for natural language processing that may include regular expressions, artificial neural networks, n-gram language models, logic regression, vector semantics, part-of-speech tagging, recurrent neural networks, and/or the like. For example, the comprehension module may use key phrases to attempt to determine the type of the user request.”) as an input to generate a response to the inquiry about the product (RULE Par 63 – “In various embodiments, comprehension module 422 of data analysis module 130 is configured to interact with the user via the VA and via data handling module 110. In various embodiments, comprehension module 422 may receive the transcribed data from data handling module 110, analyze the transcribed data, and select an appropriate response.”), wherein the CLM was specifically trained <and customized for the product provided by the product provider> via machine learning (RULE Par 64 – “Comprehension module 422 may additionally identify a choice of words (e.g., vocabulary or colloquialisms) for the user and select the response based on the choice of words. In some embodiments, the words in the pre-compiled responses may be substituted with synonym words chosen to match the vocabulary of the user.”; Par 65 – “In various embodiments, comprehension module 422 may be configured to process transcribed text data to extract characteristics associated with the user or with the user request. In some embodiments, comprehension module 422 may include neural networks, recurrent neural networks (RNN) or convolutional neural networks (CNN) to process the text data to identify various characteristics of the user. In some embodiments, a specific computer model (CM) (e.g., recurrent neural network) may be trained to identify a specific feature of the text data associated with the user request.”; Par 75 – “FIG. 5 shows that configuration of a VA by user profile module 120 may involve selecting from among a plurality of types of pre-configured VAs (for the same user) depending on the type of request associated with the user communication. For example, FIG. 5 shows that for the type of request “Type A”, user profile module 120 may select either VAA1 or VAA2.”;Par 77 – “In an example embodiment, the main attribute (A2) may be the vocabulary of the VA and the runtime attributes may be vocabulary proficiency level (A21), vocabulary containing specialized terminology (A22), southern dialect and vocabulary (A23) and/or northeastern dialect and vocabulary (A24). In another example, the main attributes may include languages such as British English (A21), American English (A22), Chinese (A23), Greek (A24) and/or the like.”); transmitting, by the digital agent, the response to the inquiry about the product to the user device over the network (RULE Par 36 – “For example, synchronous voice data may include multi-directional live audio exchanges, such as voice-over-IP exchanges, or exchanges using other varieties of telephony.”; Par 62 – “For example, comprehension module 422 may prompt the user to answer “Are you calling to report a loss of the credit card? Please answer yes or no. ” After receiving a reply “yes” from the user, comprehension module 422 may proceed to ask the user “Would you like to suspend the card, or would you like to cancel the card?” In various embodiments, comprehension module 422 may select questions for the user resulting in a set of expected answers. For example, for a question “would you like to suspend the card, or would you like to cancel the card?” the set of expected answers may be “suspend” or “cancel.” In some embodiment, the user may answer “the card was lost yesterday, and I would like to suspend it for now” and the comprehension module may be configured to determine that the card needs to be suspended. In some embodiments, the user may answer “I am reporting a theft of the credit card” and comprehension module 422 may be configured to determine that the card needs to be canceled.”); storing the dimensions of the user in a user profile of the user in a storage device, wherein the dimensions of the user can be retrieved from the user profile (RULE Par 57 – “In step 335 of process 300, user profile module 120 may be configured to create or update the user profile. The data for creating or updating the user profile may be transmitted from data analysis module 130. Various elements of user profile module 120 will be explained further below. In step 340 of process 300, the VA may be configured, that is, selected and/or updated, based on the user profile data as well as the data transmitted from data analysis module 130.”; Par 66 –“Data analysis module 130 may further include an audio/video/image module 424 for processing the audio video and image data, and for analyzing audio characteristics associated with the user request. In various embodiments, audio/video/image module 424 may analyze a pitch, a tone, a cadence of the user speech, volume and a rate of a user speech to extract various characteristics from the speech.”; Par 117 – “For the user profile describing the user as a female from Rhode Island, data analysis module 130 may select a female voice with Rhode Island dialect. Data analysis module 130 may additionally modify the first and second set of attributes of the virtual assistant based on the user characteristics, such as user rate of speech, pitch, a cadence of speech and/or the like.”) and used to generate subsequent responses in response to subsequent inquiries from the user (RULE Par 83 – “FIG. 8 is a flowchart of an illustrative process 800 for configuring a VA for a user based, at least in part, on previous user interactions with interaction system 105. In step 811A, the user may interact with interaction system 105 for example, for the first time. The data associated with interaction history (interaction history data A) may be recorded via user profile module 120 in database 170 of server system 135.”); examining the dimensions of the user to determine whether a first predetermined condition has been satisfied (RULE Par 90 – “For example, module 130 may compute a variance of a user characteristic over a select time window. In some embodiments, module 130 may compute a covariance of a first user characteristic and a second user characteristic. In some embodiments, module 130 may monitor and/or store the running variance of the user characteristic or running covariance of the first and second user characteristic.”; Par 92 – “In various embodiments, the average value of user characteristics (averaged over an appropriate time window selected during user communication session) may be used to configure or update a VA. For example, if the averaged value of a user characteristic is changing significantly at a point in time (or during a time interval) during user communication session, system 105 may be configured to update VA attributes.” Par 66 – “For example, audio/video/image module 424 may identify a gender of the user, an age of the user, a dialect of the user, an accent of the user speech, a tone of voice, an emotional content of the speech, or any other aspects that uniquely identify the user based on the audio video or image characteristics of the receiving data.”; In other words, the system examines whether a value of a user characteristic is changing significantly.), [including determining whether at least one of the dimensions cannot be ascertained]; updating a notification attribute of the user profile, in response to determining the first predetermined condition has been satisfied (RULE Par 92– “In various embodiments, the average value of user characteristics (averaged over an appropriate time window selected during user communication session) may be used to configure or update a VA. For example, if the averaged value of a user characteristic is changing significantly at a point in time (or during a time interval) during user communication session, system 105 may be configured to update VA attributes. For example, system 105 may chose to update VA attributes to result in an increased running average satisfaction score for the user. In some embodiments, system 105 may analyzes changes in average values of user characteristics based on historic data for one or more users, and select an appropriate update to VA attributes according to historic updates to VA attributes that led to improved results, such as, for example, increased running average satisfaction score for a user.”; Par 98 – “For instance, during the first time interval, data analysis module 130 may determine a first rate of speech of the user, and during the second time interval data analysis module 130 may determine a second rate of speech of the user. It should be noted, that the rate-of-speech characteristic is only illustrative and other characteristics, such as volume of the sound, pitch of the speech, inflection of the speech, cadence of the speech, etc. may be used. In some embodiments, data analysis module 130 may be configured to determine a change between the first and the second user characteristics and modify the virtual assistant attributes, based on the determined change. For example, if data analysis module 130 detects that the rate of speech of the user has decreased during the second time interval data analysis module 130 may instruct virtual assistant module 140 to reduce the rate of speech of the VA interacting with the user. In various embodiments, data analysis module 130 may be configured to change other attributes of the VA, such as vocabulary of the VA, type of answers provided by the VA, or various aspects of the VA's speech.”); [periodically] examining the notification attribute to determine whether a second predetermined condition has been satisfied (RULE Par 86 – “The term “overall satisfaction” or “general satisfaction” over a period of communication time, such as communication session, may refer to an averaged satisfaction score calculated over the period of communication time. For example, the averaged satisfaction score may be calculated as 1/Δt∫SS(t)dt, where Δt—is a duration of the communication, SS(t)—is a satisfaction score as a function of time, and the integral is calculated over the period of time Δt. Alternatively, interaction system 105 may select a different VA configuration if the user was generally dissatisfied (i.e., the overall satisfaction is negative) during previous communication sessions. The user satisfaction may be based on the emotional content of the user during user communication with interaction system 105 as well as the duration of the user communication.”; Par 88 – “User 2 may have several more communication sessions for the same “Type A” matter, such as the communication sessions 410B and 410C that also lasted for a long period of time, indicating an overall dissatisfaction 410 for user 2. During another communication session 413 for user 2, interaction system 105 may have analyzed the user communication data associated with communication sessions 410A-410C using data analysis module 130 and configured a new VA (VA2 as shown in FIG. 9) resulting in a much shorter communication time, and consecutively a positive experience for the user 2.”); and invoking an escalation process in response to determining the second predetermined condition has been satisfied, including transmitting an escalate message to a predetermined destination to allow the predetermined destination to evaluate potential abnormal dimensions of the user (RULE Par 81 – “Similarly, if data analysis module 130 indicates that the user's sentiment is becoming more negative, data analysis module 130 may instruct interaction system 105 to transfer the communication to a live agent. The term “live assistant” or “live agent” refers to an actual human being that may assist a user during the communication with the interaction system.”; In other words, the system continuously examines the user’s characteristics and updates the values. The updated values are further used to update a notification/response attribute (e.g., reducing the rate of speech of the VA). The system further monitor if the modification has been successful to increase the user’s sentiment (i.e., to increase the satisfaction score). If it fails to increase the satisfaction score, the method/system invokes an escalation process to transfer the communication to a live agent. ). RULE does not explicitly disclose the <square-bracketed> and the <angle-bracketed> limitations. MUKHERJEE discloses the [square-bracketed] limitations. MUKHERJEE discloses the method/system for interacting with a user comprising: examining the dimensions of the user to determine whether a first predetermined condition has been satisfied (MUKHERJEE Par 59 – “The caller interaction measurement system 108 may record and store a number of positive caller interaction points for misrecognitions associated with each of the plurality of interactive modules 102 b of the automated voice portal system 102 during a call session. The caller interaction measurement system 108 may record and store a number of negative caller interaction point for good recognition associated with each of the plurality of interactive modules 102 b of the automated voice portal system 102 during a call session. In another exemplary embodiment, the caller interaction measurement system 108 may record and store a number of caller interaction points for changes in the volume of the caller during a call session. In other exemplary embodiments, the caller interaction measurement system 108 may record and store a number of caller interaction points for derogatory languages used by a caller during a call session. After monitoring caller interaction during a call session, the method 300 may proceed to block 308.”), [including determining whether at least one of the dimensions cannot be ascertained] (MUKHERJEE Par 59 – “The caller interaction measurement system 108 may record and store a number of positive caller interaction points for misrecognitions associated with each of the plurality of interactive modules 102 b of the automated voice portal system 102 during a call session.”; Par 26 – “The misrecognition module 214 may monitor a rate of misrecognitions by the automated voice portal system 102. In an exemplary embodiment, a misrecognition by the automated voice portal system 102 may require a caller to repeat a response and as the rate of misrecognition increases during a call session the caller may become agitated or frustrated with the call session.”; Par 12 – “In the event that the response provided by the caller does not match the predetermined set of grammars, the interactive module 102 b may perform one or more predetermined actions (e.g., repeat the question, propose an acceptable response, or transfer to a live agent).”; In other words, the measurement system 108 examines the dimensions to determine whether an event (good recognition, misrecognition, change in the volume of the caller, etc.) has occurred, including determining whether a misrecognition (i.e., the recognition cannot be ascertained so the system requires a caller to repeat a response) has occurred.); updating a notification attribute of the user profile, in response to determining the first predetermined condition has been satisfied (MUKHERJEE Par 51 – “In another exemplary embodiment, various modules may provide updates to the processing module 210. For example, the misrecognition module 214 may provide a number of caller interaction points associated with misrecognitions by the interactive modules 102 b during a call session to the processing module 210. In another exemplary embodiment, the caller tone/pitch module 216 may provide a number of caller interaction points associated with changes in the volume of a caller during a call session to the processing module 210. In other exemplary embodiments, the language module 218 may provide a number of caller interaction points associated with language used by a caller during a call session to the processing module 210. The various modules may continuously or intermittently providing updated caller interaction points to the processing module 210.”; In other words, the modules provides updates in response to determining an event has occurred (good recognition / misrecognition / change in volume, etc.).). [periodically] examining the notification attribute to determine whether a second predetermined condition has been satisfied (MUKHERJEE Fig. 3 –“Comparing Total Points with Call Interaction Threshold 308”; Par 70 – “At block 408, the caller interaction measurement system 108 may compute a total number of caller interaction points accumulated during a call session. In an exemplary embodiment, the processor module 210 of the caller interaction measurement system 108 may compute a total number of caller interaction points based at least in part on good recognitions and misrecognitions by interactive modules 102 b, changes in caller's tone/pitch, and language used by a caller during a call session. For example, the processor module 210 may periodically or continuously access the misrecognition module 214, the caller tone/pitch module 216, and the language module 218 to obtain caller interaction points computed by each module. Also, the processor module 210 may receive updates of caller interaction points computed by the misrecognition module 214, the caller tone/pitch module 216, and the language module 218.”; Par 60 – “At block 308, the caller interaction measurement system 108 may compare a total number of caller interaction points accumulated during a call session with a caller interaction threshold. In an exemplary embodiment, the caller interaction measurement system 108 may determine a caller interaction threshold based at least in part on previous call sessions with the automated voice portal system 102.”; In other words, the processor module 210 periodically examines the points from each module to determine whether the total point exceeds a threshold.); and invoking an escalation process in response to determining the second predetermined condition has been satisfied (MUKHERJEE Par 60 – “At block 308, the caller interaction measurement system 108 may compare a total number of caller interaction points accumulated during a call session with a caller interaction threshold. In an exemplary embodiment, the caller interaction measurement system 108 may determine a caller interaction threshold based at least in part on previous call sessions with the automated voice portal system 102. In an exemplary embodiment, the caller interaction measurement system 108 may determined that the caller interaction threshold for a caller is 200 caller interaction points. Also, the caller interaction measurement system 108 may determine caller interactions points (x) for one unit of frustration during a call session. In an exemplary embodiment, the caller interaction measurement system 108 may determine that one unit of frustration is 300 caller interaction points.”), including transmitting an escalate message to a predetermined destination to allow the predetermined destination to evaluate potential abnormal dimensions of the user (MUKHERJEE Par 62 – “At block 310, the caller interaction measurement system 108 may take one or more actions. For example, the caller interaction measurement system 108 may prompt the caller to a live agent, providing an option for the caller to transfer to a live agent, or directly transfer the caller to a live agent, or other desired actions to relieve agitation or frustration of the caller during a call session. After taking one or more actions, the method 300 may proceed to block 312.”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method/system of RULE to include determining a dimension cannot be ascertained and periodically examining a condition for escalation, as taught by MUKHERJEE. One of ordinary skill would have been motivated to include determining a dimension cannot be ascertained and periodically examining a condition for escalation, in order to accurately determine an appropriate timing for escalation to relieve agitation or frustration of a caller. JADHAV discloses the <angle-bracketed> limitation. JADHAV discloses a method/system for receiving a user query and generating a response to the query comprising: invoking, by the digital agent, a custom language model (CLM) on content of the first audio stream as an input to generate a response to the inquiry about the product (JADHAV Par 41 – “Different LLMs may produce responses of different accuracies for different domains. Hence, one or more of the plurality of LLMs may be selected based on a domain label associated with the user query 122. Furthermore, the token length (e.g., number of words) that can be handled in the input and the output by the LLM is another consideration that enables the selection of one or more of the plurality of LLMs 160. For example, a subset of the plurality of LLMs 160 may be marked for long context inputs while the choice of LLMs may be wider for shorter context inputs. In an example, the model tester 152 may include a trained AI model such as a classifier with various LLM selection criteria as features for the preliminary evaluation of the plurality of LLMs 160.”), wherein the CLM was specifically trained <and customized for the product provided by the product provider> via machine learning (JADHAV Par 41 – “The plurality of LLMs 160 may include different LLMs from different providers. In an example, a single provider may provide different LLMs. In an example, different LLMs from a provider may be trained for different domains. LLMs may be evaluated on criteria 404 including but not limited to token length, API wrapper availability, accuracy of responses, bias, false positives, and false negatives.”; Par 32 – “Testing and Selecting the Most Relevant LLM Model: Various state-of-the-art LLM models form the plurality of LLMs 160 (e.g., LLM1, LLM2, . . . LLMn, wherein n is a natural number and n>1) which are evaluated and tested by the model tester 152 to identify the most suitable one for question-answering, summarization, tonality and theme generation tasks in various domains associated with an enterprise such as Enterprise Campaigns, Product Launches, Partnership & Alliances, Press Releases, Investor Relations, Public Relations & Community Engagement. The selected LLM model is then fine-tuned using domain-specific data to optimize performance.”); transmitting, by the digital agent, the response to the inquiry about the product to the user device over the network (JADHAV Par 78 – “Prompt Engineering: The document analysis system 100 can formulate the user's question into an optimized prompt to selected ones of the plurality of LLMs 160 thereby ensuring that the most relevant and contextually accurate response is provided to the user.”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method/system of RULE to include training and customizing a language model for a specific domain, as taught by JADHAV. One of ordinary skill would have been motivated to include training and customizing a language model for a specific domain, in order to ensure that the most relevant and contextually accurate response is provided to a user. REGARDING CLAIM 2, RULE in view of MUKHERJEE and JADHAV discloses the method of claim 1, further comprising: selecting, by the digital agent, a live agent from a plurality of live agents based on the dimensions of the user, wherein each of the live agents is capable of speaking with different voice characteristics (RULE Par 93 – “In some embodiments, interaction system 105, based on data analysis performed by data analysis module 130, may identify a live agent having personality traits that may be suitable for communicating with the user. For example, various live agents may have attributes that are similar to the attributes of VA, such as gender, age, accent, a rate of speech and/or the like. The attributes of available live agents may be compared to the preferred VA attributes for the user stored in the user profile, and a live agent may be selected with attributes that match best the preferred VA attributes for the user stored in the user profile. For example, if the user prefers VA with a female voice with New England dialect, data analysis module 130 may find a live agent that best matches the user preferred attributes.”); and transmitting the interactive session to the selected live agent to allow the selected live agent to conduct a live session with the user device regarding the inquiry about the product (RULE Par 93 – “In some embodiments, interaction system 105 may be configured to transfer a user to a live assistant for further help, for example, if a duration of a communication session for a user with a user configured VA exceeded a specified threshold. The specified threshold may be dependent on the type of the user request.”), wherein the selected live agent is capable of speaking with a voice similar to the voice characteristics of the user (RULE Par 117 – “For the user profile describing the user as a female from Rhode Island, data analysis module 130 may select a female voice with Rhode Island dialect. Data analysis module 130 may additionally modify the first and second set of attributes of the virtual assistant based on the user characteristics, such as user rate of speech, pitch, a cadence of speech and/or the like.”). REGARDING CLAIM 3, RULE in view of MUKHERJEE and JADHAV discloses the method of claim 2, further comprising converting the first audio stream into a text stream using a speech-to-text (STT) module (RULE Par 47 – “Examples of the neural network models are only illustrative and various other models may be used by transcription module 222 to recognize and transcribe audio signals into text. The audio signals that cannot be identified as words may be transcribed as a text string identifier such as “inaudible” or another appropriate string identifier. Transcription module 222 may transmit the transcribed data to data analysis module 130 via back end interface module 228 for data analysis and interpretation of the user request.”), wherein the CLM is invoked on the text stream as the input to generate the response (RULE Par 61 – “In some embodiments, the comprehension system may attempt to determine the type of the user request based on natural language processing using various algorithms developed for natural language processing that may include regular expressions, artificial neural networks, n-gram language models, logic regression, vector semantics, part-of-speech tagging, recurrent neural networks, and/or the like. For example, the comprehension module may use key phrases to attempt to determine the type of the user request.”; Par 63 – “In various embodiments, comprehension module 422 may receive the transcribed data from data handling module 110, analyze the transcribed data, and select an appropriate response.’). REGARDING CLAIM 5, RULE in view of MUKHERJEE and JADHAV discloses the method of claim 1, further comprising: generating a second audio stream based on the response and the dimensions of the user, such that at least some of voice characteristics of the second audio stream are similar to the voice characteristics of the user (RULE Par 117 – “For the user profile describing the user as a female from Rhode Island, data analysis module 130 may select a female voice with Rhode Island dialect. Data analysis module 130 may additionally modify the first and second set of attributes of the virtual assistant based on the user characteristics, such as user rate of speech, pitch, a cadence of speech and/or the like.”; Par 67 – “In some embodiments, audio/video/image module 424 may be configured to transmit information regarding the user characteristics to user profile module 120. In some embodiments, data analysis module 130 may be configured to instruct virtual assistant module 140 to synthesize speech of a VA to match the proficiency level, gender, and age associated with the user characteristics, to improve user experience.”); and transmitting the second audio stream to the user device over the network (RULE Fig. 1; Par 36 – “In some embodiments, communication system 100 may allow users 102A-102C to communicate with interaction system 105 using multimedia content, such as text, slides, audio, and video data. The audio data may include synchronous and/or asynchronous voice data. For example, synchronous voice data may include multi-directional live audio exchanges, such as voice-over-IP exchanges, or exchanges using other varieties of telephony. In various embodiments, communication system 100 may be configured to allow users 102A-102C to transmit audio, video, graphics and text data using devices 103A-103C to and from server system 135. Communication system 100 may also be configured to allow interaction system 105 and server system 135 to interact with one another and exchange audio, video, graphics text data as well as programming instructions.”). REGARDING CLAIM 6, RULE in view of MUKHERJEE and JADHAV discloses the method of claim 5, wherein determining dimensions of the user comprises: determining a gender of the user based on the voice characteristics of user (RULE Par 66 – “For example, audio/video/image module 424 may identify a gender of the user, an age of the user, a dialect of the user, an accent of the user speech, a tone of voice, an emotional content of the speech, or any other aspects that uniquely identify the user based on the audio video or image characteristics of the receiving data. In some embodiments, when the user is communicating with interaction system 105 using video data, audio/video/image module 424 may identify the characteristics of the user (e.g., age, gender, ethnicity, hand gestures, facial expressions, body movements, and the like) associated with the video streaming data.”); and determining an age of the user based on the voice characteristics of the user (RULE Par 66 – “For example, audio/video/image module 424 may identify a gender of the user, an age of the user, a dialect of the user, an accent of the user speech, a tone of voice, an emotional content of the speech, or any other aspects that uniquely identify the user based on the audio video or image characteristics of the receiving data. In some embodiments, when the user is communicating with interaction system 105 using video data, audio/video/image module 424 may identify the characteristics of the user (e.g., age, gender, ethnicity, hand gestures, facial expressions, body movements, and the like) associated with the video streaming data.”), wherein the second audio stream is generated with a voice spoken by a person with the same gender and similar age of the user (RULE Par 117 – “For the user profile describing the user as a female from Rhode Island, data analysis module 130 may select a female voice with Rhode Island dialect. Data analysis module 130 may additionally modify the first and second set of attributes of the virtual assistant based on the user characteristics, such as user rate of speech, pitch, a cadence of speech and/or the like.”). REGARDING CLAIM 7, RULE in view of MUKHERJEE and JADHAV discloses the method of claim 5, wherein determining dimensions of the user comprises determining a native language spoken by the user based on the voice characteristics of the user (RULE Par 34 – “In some embodiments, for example, the system may be configured to determine the time that is needed for the user to answer authentication questions. In various embodiments, the VA may be configured based on user characteristics such as age, nationality, language, accent and/or the like.”; Par 47 – “In some embodiments, the STT engine may be trained to recognized dialects and/or accents of languages.”), wherein the second audio stream is generated with a voice spoken by a person with an accent similar to the user (RULE Par 117 – “For the user profile describing the user as a female from Rhode Island, data analysis module 130 may select a female voice with Rhode Island dialect. Data analysis module 130 may additionally modify the first and second set of attributes of the virtual assistant based on the user characteristics, such as user rate of speech, pitch, a cadence of speech and/or the like.”). REGARDING CLAIM 12, RULE in view of MUKHERJEE and JADHAV discloses a non-transitory machine-readable medium having instructions, which when executed by a processor, cause the processor to perform a method for providing support services, the method (RULE Par 28 – “Program Instructions … by a processor”) comprising: performing the steps of claim 1; thus, it is rejected under the same rationale. Claim 13 is similar to claim 2; thus, it is rejected under the same rationale. Claim 14 is similar to claim 3; thus, it is rejected under the same rationale. Claim 16 is similar to claim 5; thus, it is rejected under the same rationale. Claim 17 is similar to claim 6; thus, it is rejected under the same rationale. Claim 18 is similar to claim 7; thus, it is rejected under the same rationale. REGARDING CLAIM 20, RULE in view of MUKHERJEE and JADHAV discloses a data processing system operating as a server, comprising: a processor; and a memory having instructions stored therein, which when executed by the processor, cause the processor to perform a method for providing support services (RULE Fig. 1; Par 28 – “Program Instructions … by a processor”), the method comprising: performing the steps of claim 1; thus, it is rejected under the same rationale. Claim 4 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over RULE in view of MUKHERJEE and JADHAV, and in further view of KONERU (US 2025/0182751 A1). REGARDING CLAIM 4, RULE in view of MUKHERJEE and JADHAV discloses the method of claim 3. RULE in view of MUKHERJEE and JADHAV does not explicitly teach transmitting the text stream to the selected live agent. KONERU discloses a method/system for generating a response based on caller’s query, wherein transmitting the interactive session to the selected live agent comprises transmitting the text stream to the selected live agent, such that the selected live agent review context of the interactive session during the live session (KONERU Par 58 –"In this example, the virtual assistant server 150 may place or move the conversation of the user into a queue of one of the plurality of human agents. Further, the virtual assistant server 150 may provide to the human agent information such as, for example, a reason for the escalation, a transcript of the conversation, user profile of the user, the use case determined, the user sentiment, the conversation state, history of user interactions with the virtual assistant server 150, or any other information that helps the human agent in understanding and addressing the user utterance. Furthermore, in this example, while the conversation is escalated to the human agent, the virtual assistant server 150 may transmit one or more interim responses to the user device 110(1) comprising: a reason for the escalation, an approximate time in which the human agent would start the conversation with the user, or the like.”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method/system of RULE in view of MUKHERJEE and JADHAV to include providing textual transcript to a live agent, as taught by KONERU. One of ordinary skill would have been motivated to include providing textual transcript to a live agent, in order to assist the live agent in understanding and addressing the user utterance. Claim 15 is similar to claim 4; thus, it is rejected under the same rationale. Claims 8-11, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over RULE in view of MUKHERJEE and JADHAV, and in further view of LOVE (US 2024/0194197 A1). REGARDING CLAIM 8, RULE in view of MUKHERJEE and JADHAV discloses the method of claim 5, wherein performing an audio analysis comprises: determining a dimension [score] value for each of the dimensions of the user, wherein a dimension score represents a state of the corresponding dimension (RULE Par 66 – “In various embodiments, audio/video/image module 424 may analyze a pitch, a tone, a cadence of the user speech, volume and a rate of a user speech to extract various characteristics from the speech. For example, audio/video/image module 424 may identify a gender of the user, an age of the user, a dialect of the user, an accent of the user speech, a tone of voice, an emotional content of the speech, or any other aspects that uniquely identify the user based on the audio video or image characteristics of the receiving data. In some embodiments, when the user is communicating with interaction system 105 using video data, audio/video/image module 424 may identify the characteristics of the user (e.g., age, gender, ethnicity, hand gestures, facial expressions, body movements, and the like) associated with the video streaming data.”); calculating a customer voice index (CVI) based on the dimension [scores] values of the dimensions of the user using a predetermined algorithm (RULE Par 80 – “In some embodiments, data analysis module 130 may assign a satisfaction score, or value, to the user satisfaction based on a function that maps the user speech characteristics pertaining to the user emotional state to numerical values.”; ); and storing the CVI in the user profile of the user (RULE Par 95 – “In an example embodiment, data analysis module 130 may be configured to evaluate user satisfaction or emotional state throughout the user communication with interaction system 105 and to store the timetable of user satisfaction or emotional state in the user profile. In some embodiments, data analysis module 130 may evaluate a base level satisfaction or a target emotional state for a user depending on user characteristics. For example, some users may, in general, be satisfied when communicating with VAs of the system 105, and some users may generally be irritated. In various embodiments, data analysis module 130 may store the base level satisfaction and/or target emotional state in the user profile classified by type of request from the user.”). RULE does not explicitly teach the [square-bracketed] limitation and teaches the underlined feature instead. RULE discloses the [square-bracketed] limitation. RULE discloses a method/system classifying input speech into different emotional states comprising: determining a dimension [score] for each of the dimensions of the user, wherein a dimension score represents a state of the corresponding dimension (LOVE Figs. 8 and 9; Par 90 – “Once these attributes have been identified in the voice sample, and a score is assigned to each attribute (e.g., a score from about 0-10 is assigned to each attribute of the five components of paralanguage attributes—pitch, pace, tone, melody, and volume), the classification of which affective category the voice sample belongs, based on the assigned scores, will become apparent through the aggregation.”); calculating a customer voice index (CVI) based on the dimension [scores] of the dimensions of the user using a predetermined algorithm (LOVE Par 90 – “Once these attributes have been identified in the voice sample, and a score is assigned to each attribute (e.g., a score from about 0-10 is assigned to each attribute of the five components of paralanguage attributes—pitch, pace, tone, melody, and volume), the classification of which affective category the voice sample belongs, based on the assigned scores, will become apparent through the aggregation. It is to be understood that the scoring method is a non-limiting example, and different ranges and levels of precision (i.e. number of significant digits) of scores may be assigned to each attribute, as will become apparent to those skilled in the art. The scoring method, itself, will be discussed in further detail below.”; Par 103 – “Block 824 comprises aggregating each of the values from block 804 to block 820 to provide an affective interpretation value. This may be done through a summation equation, as discussed earlier and reproduced here for clarity: Bc=KPiApi+KPaAPa+KT0AT0+KV0AV0+KMel.AscAMel.Asc+KMel.DescAMel.Desc+KMel.MonAMel.Mon, where Be is the emotion and KPi, KPa. KT0. KV0. KMel.Asc.KMel.Desc.KMel.Mon are the model weighting coefficient sets derived from the training system model using pre-recorded voice clips. Furthermore, the “A” values are the extracted value from the given segment for each of pitch, pace, tone, melody and volume.”; Par 104 – “Block 826 comprises associating the resulting affective interpretation value with an emotion. An example of the values assigned to each emotion from block 804 to block 816 can be seen in FIG. 9 . In this example dataset, the values are displayed before the initial performance of method 700, and where all model weighting coefficients are set to 1. In this example embodiment, happy is assigned a pitch value of 6, a pace value of 6, a tone value of 6, a melody value of 9 (aggregation of a pitch range of 5, 3 notes, and an ascending value of 1).”; Par 61 –“ FIG. 3 shows an illustrative example of how happiness and anger may be differentiated within the voice segments. In mathematical terms, this may be modelled by a support vector machine (SVM) for each of the emotions using: Bc=KPiApi+KPaAPa+KT0AT0+KV0AV0+KMel.AscAMel.Asc+KMel,DescAMel,Desc+KMel.MonAMel.Mon, where Be is the emotion and KPi, KPa, KT0. KV0. KMel.Asc. KMel,Desc. KMel.Mon are the model weighting coefficient sets derived from the training system model using pre-recorded voice clips. ”; Par 63 – “From the evaluation of the emotion values Be, a confidence value may be assigned to each of the emotions. In the example of FIG. 3 , emotion ‘Happy’ may be chosen if the value of the ascending melody feature is greater than the monotonous melody feature, and emotion ‘Angry’ may be chosen otherwise. Similarly, a multiclass SVM model may be trained to decide between any number of different emotions. The computation may also include a numerical confidence index/scale for more accurate pattern detection.”); and It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method/system of RULE in view of MUKHERJEE and JADHAV to include normalizing dimensional values to obtain scores, as taught by LOVE. One of ordinary skill would have been motivated to include normalizing dimensional values to obtain scores, in order to obtain more parsable data to enable the accurate analysis of the data. REGARDING CLAIM 9, RULE in view of MUKHERJEE, JADHAV, and LOVE discloses the method of claim 8, wherein each of the dimension scores is associated with a weight factor when calculating the CVI using the predetermined algorithm (LOVE Par 103 – “Block 824 comprises aggregating each of the values from block 804 to block 820 to provide an affective interpretation value. This may be done through a summation equation, as discussed earlier and reproduced here for clarity: Bc=KPiApi+KPaAPa+KT0AT0+KV0AV0+KMel.AscAMel.Asc+KMel.DescAMel.Desc+KMel.MonAMel.Mon, where Be is the emotion and KPi, KPa. KT0. KV0. KMel.Asc.KMel.Desc.KMel.Mon are the model weighting coefficient sets derived from the training system model using pre-recorded voice clips. Furthermore, the “A” values are the extracted value from the given segment for each of pitch, pace, tone, melody and volume.”). REGARDING CLAIM 10, RULE in view of MUKHERJEE, JADHAV, and LOVE discloses the method of claim 8, further comprising: selecting a text-to-speech (TTS) module from a plurality of TTS modules based on the CVI, wherein each of the TTS modules is configured to generate a voice with different voice characteristics (RULE Fig. 9 – “413 Change VA for User 2, Type A Request, Postiive Overall Satisfiaction VA2”; Par 75 – “FIG. 5 shows that configuration of a VA by user profile module 120 may involve selecting from among a plurality of types of pre-configured VAs (for the same user) depending on the type of request associated with the user communication. For example, FIG. 5 shows that for the type of request “Type A”, user profile module 120 may select either VAA1 or VAA2. The choice of VAA1 or VAA2 may be decided based on other data associated with the user request, such as other variable data 520, or streaming data 530.”; Par 88 – “FIG. 9 further illustrates a communication session 410A, for a user 2 for a “Type A” request that may lasted a long time (40 min as shown in FIG. 9). User 2 may have several more communication sessions for the same “Type A” matter, such as the communication sessions 410B and 410C that also lasted for a long period of time, indicating an overall dissatisfaction 410 for user 2. During another communication session 413 for user 2, interaction system 105 may have analyzed the user communication data associated with communication sessions 410A-410C using data analysis module 130 and configured a new VA (VA2 as shown in FIG. 9) resulting in a much shorter communication time, and consecutively a positive experience for the user 2.”; Par 31 – “In various embodiments, the system may provide a VA configured to interact with a user using audio signals including a synthesized speech using a computer system that is generally known as a speech synthesizer and can be implemented in software or hardware products. In some embodiments, a text-to-speech (TTS) system may be used to convert text language into speech. Synthesized speech may be created by concatenating pieces of recorded speech that are stored in a database. For high-quality output, the TTS may store entire words or sentences.”; Par 96 –“In various embodiments, the base level of satisfaction may be one attribute of the target emotional state and interaction system 105 may be configured to modify VA attributes to either bring the user into a target emotional state or to help the user reach base level satisfaction.”; Par 97 – “The decision to update the VA may be implemented in step 1008, and may be based on information obtained from step 1006. Step 1008 may also be based on other available information, such as, for example, the duration of the conversation with the user, the history of previous conversations and/or the like. If an update of the VA is required (step 1008, Yes), process 1000 may proceed to step 1010 and update the VA. Alternatively, if updates are not required (step 1008, No) process 1000 may proceed to step 1002 of receiving additional input from the user.”; In other words, RULE teaches pre-configured VAs, where each VA is configured with different VA attributes to generate different voice characteristics using a TTS algorithm. Thus, a VA itself / a VA attribute is selected as a TTS module to generate a voice with different characteristics. The selection of a VA and/or VA attribute is based on the user’s emotion/satisfaction score. LOVE also teaches the limitations: Par 37 – “In the example of a call center workstation, the Emotion Responder 106 may receive a classification from the Emotion Classifiers 109 that the sounds that were input may be ‘angry’. In the response to this, the Emotion Responder 106 may coach by providing one or more actionable instructions to the Speech Output module 102 such as to lower the pitch, going to a lower note, as well as the pace, by speaking slower, and decreasing the volume, and thereby demonstrating a calm and empathetic emotional sound response.”; Par 38 – “The Emotion Responder 106 may include a library of emotional personas and use prescriptive Artificial Intelligence (AI) analytics models to determine the optimal emotional sound and pre-scripted speech response enhanced with context and outcome classifiers.”; Par 40 – “In an embodiment, Emotion Responder 106 may access a library of different responses provided by Response Library 108. Method and System for Detecting, Classifying and Responding to Emotion derived from Human Speech.”); and invoking the selected TTS module to convert the response into the second audio stream (RULE Par 98 – “In some embodiments, data analysis module 130 may be configured to determine a change between the first and the second user characteristics and modify the virtual assistant attributes, based on the determined change. For example, if data analysis module 130 detects that the rate of speech of the user has decreased during the second time interval data analysis module 130 may instruct virtual assistant module 140 to reduce the rate of speech of the VA interacting with the user. In various embodiments, data analysis module 130 may be configured to change other attributes of the VA, such as vocabulary of the VA, type of answers provided by the VA, or various aspects of the VA's speech.”; Par 31 – “In various embodiments, the system may provide a VA configured to interact with a user using audio signals including a synthesized speech using a computer system that is generally known as a speech synthesizer and can be implemented in software or hardware products. In some embodiments, a text-to-speech (TTS) system may be used to convert text language into speech. Synthesized speech may be created by concatenating pieces of recorded speech that are stored in a database. For high-quality output, the TTS may store entire words or sentences.”; LOVE also teaches the limitations: Fig. 1 Speech Output 102; Par 33-35 – “Response Library 108 comprising previously derived and stored conversation graphs including optimal responses to speech (dialog); Emotion Responder 106 that transforms the information response, history responses and emotional response into output speech samples selected from the content of the Response Library 108 of optimal responses; Speech Output module 102 that delivers output speech samples via the Emotion Connector API 103.”; Par 37 – “In the response to this, the Emotion Responder 106 may coach by providing one or more actionable instructions to the Speech Output module 102 such as to lower the pitch, going to a lower note, as well as the pace, by speaking slower, and decreasing the volume, and thereby demonstrating a calm and empathetic emotional sound response.”). REGARDING CLAIM 11, RULE in view of MUKHERJEE, JADHAV, and LOVE discloses the method of claim 8, further comprising selecting the CLM from a plurality of CLMs associated with the product (JADHAV Par 41 – “Different LLMs may produce responses of different accuracies for different domains. Hence, one or more of the plurality of LLMs may be selected based on a domain label associated with the user query 122. Furthermore, the token length (e.g., number of words) that can be handled in the input and the output by the LLM is another consideration that enables the selection of one or more of the plurality of LLMs 160. For example, a subset of the plurality of LLMs 160 may be marked for long context inputs while the choice of LLMs may be wider for shorter context inputs. In an example, the model tester 152 may include a trained AI model such as a classifier with various LLM selection criteria as features for the preliminary evaluation of the plurality of LLMs 160.”) based on the CVI (JADHAV Par 19 – “User feedback to the plurality of the responses from the one or more LLMs provided as replies is received and the AI-based model tester is further trained on the user feedback for LLM selection. In an example, further training of the model tester may involve providing inputs that carry differential weights favoring subject matter expert (SME) knowledge and user feedback. The AI-based document analysis system can be configured to store in a cache, one of the plurality of responses receiving the highest user approval as a reply to the received user requirement along with the received user requirement and a subset of the structured and unstructured data sources used for generating the prompt.”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method/system of RULE to include training and customizing a language model for a specific domain, as taught by JADHAV. One of ordinary skill would have been motivated to include training and customizing a language model for a specific domain, in order to ensure that the most relevant and contextually accurate response is provided to a user. Claim 19 is similar to claim 8; thus, it is rejected under the same rationale. Any inquiry concerning this communication or earlier communications from the examiner should be directed to JONATHAN C. KIM whose telephone number is (571)272-3327. The examiner can normally be reached Monday to Friday 8:00 AM thru 4:00 PM EST. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, PARAS D. SHAH can be reached at 571-270-1650. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /JONATHAN C KIM/Primary Examiner, Art Unit 2653
Read full office action

Prosecution Timeline

Mar 21, 2025
Application Filed
Oct 21, 2026
Non-Final Rejection mailed — §103, §DOUBLEPATENT (current)

Precedent Cases

Applications granted by this same examiner with similar technology

Patent 12784745
APPLICATION VOCABULARY INTEGRATION WITH A DIGITAL ASSISTANT
2y 7m to grant Granted Nov 03, 2026
Patent 12776305
ADVANCED CLUSTERING FOR SELF-SUPERVISED LEARNING IN SPEECH RECOGNITION
2y 2m to grant Granted Oct 27, 2026
Patent 12771894
WIRELESS DISPLAY DEVICE, WIRELESS SET-TOP BOX, AND WIRELESS DISPLAY SYSTEM
2y 8m to grant Granted Oct 20, 2026
Patent 12744034
ELECTRONIC DEVICE AND CONTROL METHOD THEREOF
2y 8m to grant Granted Sep 22, 2026
Patent 12744038
SYSTEMS AND TECHNIQUES FOR USING A DIGITAL ASSISTANT WITH AN ENHANCED ENDPOINTER
2y 6m to grant Granted Sep 22, 2026
Study what changed to get past this examiner. Based on 5 most recent grants.

Strategy Recommendation AI-generated — please review before filing

Get a prosecution strategy drawn from examiner precedents, rejection analysis, and claim mapping.
Typically takes 5-10 seconds — AI-generated, attorney review required before filing

Prosecution Projections

1-2
Expected OA Rounds
74%
Grant Probability
99%
With Interview (+38.4%)
2y 5m (~10m remaining)
Median Time to Grant
Low
PTA Risk
Based on 373 resolved cases by this examiner. Grant probability derived from career allowance rate.

Sign in with your work email

Enter your email to receive a magic link. No password needed.

Personal email addresses (Gmail, Yahoo, etc.) are not accepted.

Free tier: 3 strategy analyses per month