DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Application
This is a Final Action in response to the claims and remarks submitted on 04/28/2026.
Claims 1-22 are examined herein.
Claim Objections
Claim 1 is objected to because of the following informalities: Claim 1 recites the typographical error "the customer associated with the [[ interaction” in line 24. Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-22 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 1, 10 and 19 recites the limitation "the first performance parameter in the log" in line 18. There is insufficient antecedent basis for this limitation in the claim. It is unclear if this term is the same or different from the “first performance parameter of the plurality of performance parameters” as introduced in line 13 since the term as recited in line 18 is described as being the first parameter in the log which has not been introduced.
Claims 1, 10 and 19 recites the limitations "corresponding interaction metadata" in line 5, “the interaction metadata” in line 9, “the corresponding interaction metadata” in line 11, “the interaction metadata corresponding to the target interaction” in line 22. It is unclear if these terms are the same or different. It is unclear the amount of “interaction metadata” that the claim requires.
Claims 1, 10 and 19 recites the limitation “the interaction metadata corresponding to the target interaction” in line 22. There is insufficient antecedent basis for this limitation in the claim.
Claims 1, 10 and 19 recites the limitation "the first performance parameter in the performance in the performance metadata" in line 24. There is insufficient antecedent basis for this limitation in the claim. The requirement of the claim is unclear since the term was not introduced properly.
Claims 1, 10 and 19 recites the limitation "the customer associated with the interaction” in line 24. There is insufficient antecedent basis for this limitation in the claim. It is unclear if the term is the same or different to “a particular customer” as introduced in line 4.
Claims 1, 10 and 19 recites the limitation "the interaction" in line 30. There is insufficient antecedent basis for this limitation in the claim. It is unclear if the term is related to “a target interaction” or “an interaction metadata”.
Claims 1, 10 and 19 recites the limitation “the customer” in line 32. It is unclear if the term is related to the term “a particular customer” as introduced in line 4, or if the term is related to the term "the customer associated with the [[ interaction” in line 24. It is unclear if the claim requires 2 different customers or if the terms are all related.
Claims 2, 11 and 20 discloses the limitation “the first performance parameter comprises a sentiment score corresponding to at least one of the plurality of interactions”, however it is unclear if the term “the first performance parameter” is related to “first performance parameter of the plurality of performance parameters” as introduced in line 13, "the first performance parameter in the log" in line 18 or to the term "the first performance parameter in the performance in the performance metadata" in line 24. The requirements of the claim are unclear as to the amount of “first performance parameters” required by the claim.
Claims 3, 12 and 20 recites the limitation "the sentiment score corresponding to the target interaction" in line 4. There is insufficient antecedent basis for this limitation in the claim. The requirement of the claim is unclear since the term was not introduced properly.
Claims 4, 13 and recites the limitation “the customer” in line 3. It is unclear if the term is related to the term “a particular customer” as introduced in line 4, or if the term is related to the term "the customer associated with the [[ interaction” in line 24 of independent claims from which claim 4 and 13 depends. It is unclear if the claims require 2 different customers or if the terms are all related.
Claims 7, 16 recites the limitation “the customer” in line 3. It is unclear if the term is related to the term “a particular customer” as introduced in line 4, or if the term is related to the term "the customer associated with the [[ interaction” in line 24 of independent claims from which claim 7 and 16 depends. It is unclear if the claims require 2 different customers or if the terms are all related.
Claim 21 discloses the limitation “the personalized survey question” in line 3, however it is unclear if the term is related to the “a personalized survey question” introduced in line 20 or “a personalized survey question for the customer” introduced in line 29.
Claim 21 discloses the limitation “the corresponding engineered prompts” in line 9. There is insufficient antecedent basis for this limitation in the claim. It is unclear if the term is related to “a corresponding engineer prompt” as introduced in line 6.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-22 are rejected under 35 U.S.C. 101 because the claims are directed to an abstract idea without significantly more.
With respect to Step 1 of the eligibility inquiry (as explained in MPEP 2106), it is first noted that the claims are directed to at least one potentially eligible category of subject matter (i.e., process and machine, respectively). Thus, Step 1 of the Subject Matter Eligibility test for claims 1-20 is satisfied.
With respect to Step 2A Prong One, it is next noted that the claims recite an abstract idea that falls under the “Mental Processes” and “Certain Methods Of Organizing Human Activity” groups within the enumerated groupings of abstract ideas set forth in the MPEP 2106 since the claims set forth steps that recite concepts performed in the human mind (including an observation, evaluation, judgment, opinion); and managing personal behavior or relationships or interactions between people (including social activities, teaching, and following rules or instructions).
Claims 1 and 10 and 19 recites the abstract idea of generating satisfaction surveys for particular customers that are personalized to the customers’ interactions with a customer service system [018]. In claims 1, 10 and 19, this idea is described by the following claim steps:
receiving a first message from a particular customer including a service request;
recording a plurality of interactions associated with the service request and corresponding interaction metadata;
determining performance metadata using the interaction metadata corresponding to the plurality of interactions, the performance metadata comprising a plurality of performance parameters quantifying the interaction metadata;
storing one or more records comprising a log of the plurality of interactions in association with the corresponding: (a) interaction metadata and (b) performance metadata;
detecting if a first performance parameter of the plurality of performance parameters exceeds a threshold value;
in response to determining that the first performance parameter exceeds the threshold value, triggering a personalized survey question generation process;
identifying a target interaction in the plurality of interactions that corresponds to the first performance parameter in the log;
generating a model prompt to determine a personalized survey question, at least by:
generating a first portion of the model prompt based on interaction metadata corresponding to the target interaction; and
generating a second portion of the model prompt based on the first performance parameter in the performance metadata corresponding to the target interaction;
applying the first portion of the model prompt and the second portion of the model prompt to a model prompt template, to obtain the model prompt; and
using the model prompt to obtain a personalized survey question for the customer associated with the interaction;
generating a survey comprising the personalized survey question; and
transmitting the survey to the customer for completion.
This idea falls within the mental processes grouping because it is directed towards concepts performed in the human mind such as an observation, evaluation, judgment, opinion of the data collected. In the instant case a person can mentally analyze service requests, including interactions and performance metadata of the service request in order to create a personalized model to further generate a survey. This process could easily be performed mentally perhaps with the aid of pen and paper when analyzing the received data and when generating the survey. The noted abstract idea is also directed to managing interactions between people such as that required during communications when transmitting the generated survey conforms to the requirements of more than one party.
Because the above-noted limitations recite steps falling within the Mental Processes and Certain Methods of Organizing Human Activity abstract idea groupings of the MPEP 2106, they have been determined to recite at least one abstract idea when evaluated under Step 2A Prong One of the eligibility inquiry.
Therefore, because the limitations above set forth activities falling within the C Mental Processes and Certain Methods of Organizing Human Activity abstract idea groupings described in the MPEP 2106, the additional elements recited in the claims are further evaluated, individually and in combination, under Step 2A Prong Two and Step 2B below. Claim 10 and 19 recites similar limitations as claim 1 and is therefore determined to recite the same abstract idea.
With respect to Step 2A Prong Two, the judicial exception is not integrated into a practical application. The additional elements that fail to integrate the abstract idea into a practical application are:
one or more non-transitory computer readable media comprising instructions;
hardware processors;
electronic message;
large language model (LLM);
submitting the LLM prompt to an LLM;
However, using a computer environment such as a processor, and one or more non-transitory computer readable media and other recited computer elements amounts to no more than generally linking the use of the abstract idea to a particular technological environment. Generating satisfaction surveys for particular customers that are personalized to the customers’ interactions with a customer service system can reasonably be performed by pencil and paper until limited to a computerized environment by requiring an electronic message and the use of a large language model to perform the steps.
In relation with the limitations using a large language model, specifically “generating a large language model (LLM) prompt to determine a personalized survey question, at least by :generating a first portion of the LLM prompt based on the interaction metadata corresponding to the target interaction; and; generating a second portion of the LLM prompt based on the first performance parameter in the performance metadata corresponding to the target interaction” The examiner views these additional elements as results-oriented steps given that there is no restriction on how the result is accomplished and no description of the mechanism for accomplishing the result are currently present such that this is viewed as equivalent to “apply it” for merely implementing the abstract idea using generic computing components (See Id.). Therefore, the claims are also non-statutory subject matter.
All the additional elements have been evaluated, but fail to integrate the abstract idea into a practical application because they amount to using generic computing elements or computer-executable instructions (software) to perform the abstract idea, similar to adding the words “apply it” (or an equivalent), and alternatively serve to link the use of the judicial exception to a particular technological environment. See MPEP 2106.05(f) and 2106.05(h).
In addition, these limitations fail to provide an improvement to the functioning of a computer or to any other technology or technical field, fail to apply the exception with a particular machine, fail to apply the judicial exception to effect a particular treatment or prophylaxis for a disease or medical condition, fail to effect a transformation of a particular article to a different state or thing, and fail to apply/use the abstract idea in a meaningful way beyond generally linking the use of the judicial exception to a particular technological environment.
Accordingly, because the Step 2A Prong One and Prong Two analysis resulted in the conclusion that the claims are directed to an abstract idea, additional analysis under Step 2B of the eligibility inquiry must be conducted in order to determine whether any claim element or combination of elements amount to significantly more than the judicial exception.
With respect to Step 2B of the eligibility inquiry, it has been determined that the claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As noted above, the claims as a whole merely describes a method, computer system, and computer program product that generally “apply” the concepts discussed in prong 1 above. (See MPEP 2106.05 f (II)) In particular applicant has recited the computing components at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using generic computer components. As the court stated in TLI Communications v. LLC v. AV Automotive LLC, 823 F.3d 607, 613 (Fed. Cir. 2016) merely invoking generic computing components or machinery that perform their functions in their ordinary capacity to facilitate the abstract idea are mere instructions to implement the abstract idea within a computing environment and does not add significantly more to the abstract idea. Accordingly, these additional computer components do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. Therefore, even when viewed as a whole, nothing in the claim adds significantly more (i.e. an inventive concept) to the abstract idea and as a result the claim is not patent eligible.
In addition, when taken as an ordered combination, the ordered combination adds nothing that is not already present as when the elements are taken individually. There is no indication that the combination of elements integrates the abstract idea into a practical application. Their collective functions merely provide generic computer implementation. Therefore, when viewed as a whole, these additional claim elements do not provide meaningful limitations to transform the abstract idea into a practical application of the abstract idea or that, as an ordered combination, amount to significantly more than the abstract idea itself.
For the reasons identified with respect to Step 2A, prong 2, claims 1, 10 and 20 fail to recite additional elements that amount to an inventive concept. For example, use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general-purpose computer or computer components after the fact to an abstract idea (e.g., a commercial or legal interaction or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more (see MPEP 2106.05(g)). In addition, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application (see MPEP 2106.05(h)).
Dependent claims 2-9, 11-18 and 20 recite the same abstract idea as recited in the independent claims, and when evaluated under Step 2A Prong One are found to merely recite details that serve to narrow the same abstract idea recited in the independent claims accompanied by the same generic computing elements or software as those addressed above in the discussion of the independent claims, which is not sufficient to amount to a practical application or add significantly more, or other additional elements that fail to amount to a practical application or add significantly more, as noted above.
Dependent claim 2, 6 and 11, 15 further limits the abstract idea by introducing the first performance parameter comprises a sentiment score corresponding to at least one of the plurality of interactions; and the operations further comprise identifying the target interaction from the plurality of interactions based on the sentiment score and wherein generating the LLM prompt comprises specifying a selection of a survey question format for the personalized survey question from a set comprising: a Likert Scale, a binary, multiple choice, or open-ended. Describing the performance data and identifying a target interaction is a process that could be performed manually until limited by a processor. Further embellishing that the invention is capable of processing information in a generic computing environment does not integrate the abstract idea into a practical application or adds significantly more to the abstract idea. Therefore, the claims are also non-statutory subject matter.
Dependent claims 3, 5 and 12, 14 further limits the abstract idea by linking the judicial exception to a particular field of use by introducing the limitation wherein generating the LLM prompt comprises specifying the target interaction, the interaction metadata corresponding to the target interaction, and the sentiment score corresponding to the target interaction and wherein generating the LLM prompt comprises specifying a selection of a survey question format for the personalized survey question from a set comprising: a Likert Scale, a binary, multiple choice, or open-ended. The examiner views these additional elements as results-oriented steps given that there is no restriction on how the result is accomplished and no description of the mechanism for accomplishing the result are currently present such that this is viewed as equivalent to “apply it” for merely implementing the abstract idea using generic computing components (See Id.). Therefore, the claims are also non-statutory subject matter.
Dependent claim 4 and 13 further limits the abstract idea by linking the judicial exception to a particular field of use by introducing the limitation comparing the personalized survey question to previous survey questions for the customer; and based on the comparing, combining the personalized survey question with one or more predetermined survey questions. Comparing the survey questions and combining the data is a process that could be performed manually until limited by a processor. Further embellishing that the invention is capable of comparing and processing information in a generic computing environment does not integrate the abstract idea into a practical application or adds significantly more to the abstract idea. Therefore, the claims are also non-statutory subject matter.
Dependent claims 7-8 , 16-17 and 20 further limits the abstract idea by linking the judicial exception to a particular field of use by introducing the limitation responsive to receiving a completed survey from the customer, applying a machine learning model to the completed survey to determine an overall satisfaction score for the survey and an explanation of the overall satisfaction score; updating a training dataset for a machine learning model with the overall satisfaction score; and applying the updated training dataset to a machine learning algorithm to train the machine learning model; and the performance metadata comprises sentiment metadata corresponding to the plurality of interactions; the operations further comprise identifying a target interaction from the plurality of interactions based on the sentiment metadata; and generating the LLM prompt comprises specifying the target interaction, the sentiment metadata corresponding to the target interaction, and the performance metadata corresponding to the target interaction. The examiner views these additional elements as results-oriented steps given that there is no restriction on how the result is accomplished and no description of the mechanism for accomplishing the result are currently present such that this is viewed as equivalent to “apply it” for merely implementing the abstract idea using generic computing components (See Id.). Therefore, the claims are also non-statutory subject matter.
Dependent claim 9 and 18 further limits the abstract idea by introducing for individual survey questions of a plurality of survey questions included in the survey: determining a type of an individual survey question and a response to the individual survey question; and associating the types of the individual survey questions and the responses to the individual survey questions with the overall satisfaction score. Determining a type of survey question is a process that could be performed manually until limited by a processor. Further embellishing that the invention is capable of processing information in a generic computing environment does not integrate the abstract idea into a practical application or adds significantly more to the abstract idea. Therefore, the claims are also non-statutory subject matter.
Dependent claims 21-22 further limits the abstract idea by linking the judicial exception to a particular field of use by introducing the limitation generating a training dataset comprising, for each of a plurality of past service requests: (i) attributes extracted from interactions and performance metadata of the past service request, and (ii) a corresponding engineered prompt that caused an LLM to generate a survey question for the past service request; computing feature vectors from the attributes; and feeding the feature vectors and the corresponding engineered prompts as labels to a machine learning algorithm to train the machine learning model to generate LLM prompts and prior to the triggering, evaluating the plurality of performance parameters against a plurality of threshold values corresponding to respective performance parameters, the plurality of threshold values comprising thresholds for one or more of: sentiment score, response time, on-hold time, number of agent transfers, and number of escalations; and triggering the personalized survey question generation process only when at least one performance parameter exceeds the corresponding threshold value. The examiner views these additional elements as results-oriented steps given that there is no restriction on how the result is accomplished and no description of the mechanism for accomplishing the result are currently present such that this is viewed as equivalent to “apply it” for merely implementing the abstract idea using generic computing components (See Id.). Therefore, the claims are also non-statutory subject matter.
The ordered combination of elements in the dependent claims (including the limitations inherited from the parent claim(s)) add nothing that is not already present as when the elements are taken individually. There is no indication that the combination of elements improves the functioning of a computer or improves any other technology, and the collective functions merely provide high level of generality computer implementation. Therefore, whether taken individually or as an order combination, the claims are nonetheless rejected under 35 U.S.C. 101 as being directed to non-statutory subject matter.
For more information see MPEP 2106.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-22 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jeffs (US Patent Publication 2014/0143157) in view of Lin (US Patent Publication 2025/0202844).
Regarding claims 1, 10 and 19 Jeffs discloses:
one or more non-transitory computer readable media comprising instructions that, when executed by one or more hardware processors ([021] “non-transitory storage media”),
a method (abstract); and
a system comprising: at least one device including a hardware processor ([008] system comprising a “processor”);
the system being configured to perform operations comprising:
receiving a first electronic message from a particular customer including a service request ([0023] As described in further detail herein, the computing system 200 receives customer interaction content 220. Exemplarily, the customer interaction content 220 may include one or more audio files which may be audio recordings of a customer service interactions, which may exemplarily be between two speakers, although the audio recording may be of any other variety of other audio records, including multiple speakers, a single speaker, or an automated or recorded auditory message. In an embodiment as disclosed herein, the processor 200 may operate to transform the received customer interaction content 220 into a text form and further receive rules and analytics 240 for the analysis of the customer interaction content. [0025] At 102 a customer and a customer service agent have an interaction. In exemplary embodiments, the customer service interaction may include one or more communication channels, including but not limited to email, phone calls, text messages, web chat, or social media. In an exemplary embodiment, the interaction is a result of a customer calling a customer service center where a customer service agent responds to the customer's concerns.);
recording a plurality of interactions associated with the service request and corresponding interaction metadata ([0012] The method 300 begins at 302 in which customer interaction content is acquired. In an exemplary embodiment, a customer interaction may occur over a period of time, exemplarily an hour, day, week, month, or longer and may also occur across a plurality communication channels. The communication channels may include email, telephone calls, and web chat, but may also include purchases or returns executed through a website or at a brick and mortar facility. Exemplarily with the creation of a customer identification number, each of the customer interactions across these different channels can be linked and the customer interaction content extracted for analysis. In an exemplary embodiment, the customer interaction content 302 is converted to or extracted as textual data suitable to the evaluated through text analytics 304, although it is understood that analytics may also be performed as audio or speech analytics.);
determining performance metadata using the interaction metadata corresponding to the plurality of interactions, the performance metadata comprising a plurality of performance parameters quantifying the interaction metadata ([0013] In the text analytics at 304, the customer interaction content is analyzed with an ontology to identify content, themes, and/or customer sentiments in the customer interaction content. In exemplary embodiments, the analytics at 304 may include, but are not limited to keyword spotting, phonetic indexing, and phonetic searching. It is to be understood that these analytics may be performed against textual or audio data. The textual analytics can identify further information regarding the customer and the customer's experience up to the point of analysis or exemplarily through annotation of purchases and returns, other customer service concerns, requests for upgrade or a decrease of services, customer service or purchase history, reward membership status, or other reasons for customer service interaction. At 306, text analytics from 304 are used to identify the specific issue driving the customer interaction. These specific issues may be product warranties or returns, product help or questions, inquiries regarding competitive products or services, increases in service, decreases in service, or cancellation of service. In a still further exemplary embodiment, the identified issues at 306 may further include complaints or concerns regarding previous customer service interactions themselves, either for responsiveness, available channels, available customer service hours, or waiting times. The identified specific issues at 306 are then used at 308 in order to construct one or more surveys to be sent to the customer that are tailored to the identified specific issues at 306, and may further include information from the text analytics at 304.);
storing one or more records comprising a log of the plurality of interactions in association with the corresponding: (a) interaction metadata and (b) performance metadata ([012] In an exemplary embodiment, a customer interaction may occur over a period of time, exemplarily an hour, day, week, month, or longer and may also occur across a plurality communication channels. [0026] At 104 a call recorder records the conversation. The call recorder 104 may also collect metadata related to the customer service interaction. Such metadata may include an agent identification, agent extension, customer identification, time, date, or any other metadata available within the information technology infrastructure. In one non-limiting embodiment, building off of the customer identification metadata, the system may access a database storing more detailed account information of the customer using information such as a customer identification account number received during the conversation, if such information is available. The call recordings and metadata can be stored in a computer readable media at 106 within a data network and the stored recordings and metadata are indexed within the network for subsequent retrieval and analysis. Further see Figure 1 and [024] );
detecting if a first performance parameter of the plurality of performance parameters exceeds a threshold value ([0028] In one embodiment, the analytics performed at 108 can be a batch processing in which the analytics are applied to the recorded conversations after the conversations are completed or in an off-line process in a system that deals with many recorded conversations either simultaneously or in sequence. In some embodiments, this may happen as soon as the call is completed or up to 24 hours from when a conversation takes place. In alternative embodiments, the analytics are applied in a later time frame, or as needed for other applications or objectives. Alternatively, at 110 real-time speech recognition processes may be applied to the conversation as it happens. In some such embodiments, speech-to-text analytics techniques may be used, including, but not limited to, LVCSR, or other speech analytics techniques that are tailored to real-time recognition. While in other embodiments, other forms of speech analytics may be used. In such embodiments, the real-time speech analytics can be used to provide in-call guidance to the customer service agent. In one embodiment, a plurality of rules are stored in a computer readable medium at 112 that are applied during the real-time speech recognition at 110 and the rules from 112 are compared against the recognized real-time speech in order to identify triggering events that are represented by the rules. The rules can include, but are not limited to, spoken conversation in the form of words and phrases, metadata values pulled from the speech recognition, agent desktop activity as monitored through desktop analytics and tracking, and/or historical information about the customer.);
in response to determining that the first performance parameter exceeds the threshold value, triggering a personalized survey question generation process ([0029] From either processing technique, speech or textual recognition is used to identify specific topics, events, sentiment and metadata from a conversation between a customer and a customer service agent. This information is provided to a survey decision engine at 114. The survey decision engine uses the information received from the application of the batch or real-time speech recognition and analytics to decide on the content of a survey to be provided to the customer participating in the conversation. The survey decision engine 114 may exemplarily decide between pre-formed surveys stored in a computer readable medium at 116 or from a plurality of survey questions stored at a computer readable medium at 118. If the survey decision engine 114 decides, based upon the received information about the conversation, that the customer's experience and service inquiries are best represented in a standardized survey, then the survey decision engine 114 selects one of these pre-made surveys. For example, an organization could create a plurality of standardized surveys, dealing with customer service events such as purchase, a return, a product complaint, or cancelling service.) comprising:
identifying a target interaction in the plurality of interactions that corresponds to the first performance parameter in the log (See [028-029] wherein the system identifies target interactions that correspond to performance parameters [0028] In one embodiment, the analytics performed at 108 can be a batch processing in which the analytics are applied to the recorded conversations after the conversations are completed or in an off-line process in a system that deals with many recorded conversations either simultaneously or in sequence. In some embodiments, this may happen as soon as the call is completed or up to 24 hours from when a conversation takes place. In alternative embodiments, the analytics are applied in a later time frame, or as needed for other applications or objectives. Alternatively, at 110 real-time speech recognition processes may be applied to the conversation as it happens. In some such embodiments, speech-to-text analytics techniques may be used, including, but not limited to, LVCSR, or other speech analytics techniques that are tailored to real-time recognition. While in other embodiments, other forms of speech analytics may be used. In such embodiments, the real-time speech analytics can be used to provide in-call guidance to the customer service agent. In one embodiment, a plurality of rules are stored in a computer readable medium at 112 that are applied during the real-time speech recognition at 110 and the rules from 112 are compared against the recognized real-time speech in order to identify triggering events that are represented by the rules. The rules can include, but are not limited to, spoken conversation in the form of words and phrases, metadata values pulled from the speech recognition, agent desktop activity as monitored through desktop analytics and tracking, and/or historical information about the customer. [0029] From either processing technique, speech or textual recognition is used to identify specific topics, events, sentiment and metadata from a conversation between a customer and a customer service agent. This information is provided to a survey decision engine at 114. The survey decision engine uses the information received from the application of the batch or real-time speech recognition and analytics to decide on the content of a survey to be provided to the customer participating in the conversation. The survey decision engine 114 may exemplarily decide between pre-formed surveys stored in a computer readable medium at 116 or from a plurality of survey questions stored at a computer readable medium at 118. If the survey decision engine 114 decides, based upon the received information about the conversation, that the customer's experience and service inquiries are best represented in a standardized survey, then the survey decision engine 114 selects one of these pre-made surveys. For example, an organization could create a plurality of standardized surveys, dealing with customer service events such as purchase, a return, a product complaint, or cancelling service.);
responsive to a trigger event, determine a personalized survey question (See Figure 1 and [0030] In an alternative, the survey decision engine 114 decides from the received customer interaction content that it is desirable that the customer be asked survey questions that are not represented in a standard survey. In such an embodiment, based upon specifically identified information or events in the conversation, more specific or detailed survey questions are required. In this event, the survey decision engine 114 can select questions from the stored survey questions at 118 in order to create a customized survey. In a non-limiting example of a customer service call requesting cancellation of service, while a generalized survey stored at 116 may be selected, in an alternative, the conversation information provided to the survey decision engine 114 may identify that the customer was switching to a service of a known competitor and questions specifically about that competitor can be selected from survey questions at 118 to specialize the survey. These more specific questions may be able to draw more specific information based upon the customer statements during the recorded conversation. In a further non-limiting embodiment, the speech analytics of the conversation may identify that the customer calling in for service was highly frustrated with the product/service/organization. In this event, a customized survey may be created by the decision engine 114 that has additional questions related to customer service/satisfaction. These additional questions may be in the form of a standardized customer satisfaction survey stored at 116 or individualized customer survey questions that may be specifically tailored to a particular product or service selected from 118. See also [029]), at least by:
generating a first portion based on the based on the interaction metadata corresponding to the target interaction ([0027] At 108 analytics are applied to the stored call recordings and metadata in order to analyze the recorded conversations to identify topics within the spoken conversations as well as to identify any related trending information against those topics… Other analytic approaches may include, but are not limited to, keyword spotting, phonetic indexing, and phonetic searching... Analytics can be used at 108 to identify topics, trends, content, or sentiment of the conversation. In embodiments, the analytics may apply an ontology or predefined set of rules or structures for interpreting the customer interaction content. The results of the analytics can be gleaned for all transcripts, metadata pulled from open responses of the customer, categorized spoken content, and indexing/frequency of words used in the conversation. See also [0280-030] );
generating a second portion based on the first performance parameter in the performance metadata corresponding to the target interaction ([0027] At 108 analytics are applied to the stored call recordings and metadata in order to analyze the recorded conversations to identify topics within the spoken conversations as well as to identify any related trending information against those topics… The metadata included with the call recording can be used in embodiments of the analytics in order to improve the quality of the analysis and information pulled from the recorded conversation. Analytics can be used at 108 to identify topics, trends, content, or sentiment of the conversation. In embodiments, the analytics may apply an ontology or predefined set of rules or structures for interpreting the customer interaction content. The results of the analytics can be gleaned for all transcripts, metadata pulled from open responses of the customer, categorized spoken content, and indexing/frequency of words used in the conversation. See also [028-030]);
applying the first portion of and the second portion to a survey decision engine ([0029] From either processing technique, speech or textual recognition is used to identify specific topics, events, sentiment and metadata from a conversation between a customer and a customer service agent. This information is provided to a survey decision engine at 114. The survey decision engine uses the information received from the application of the batch or real-time speech recognition and analytics to decide on the content of a survey to be provided to the customer participating in the conversation.);
generating a personalized survey question for the customer associated with the interaction ([0030] In an alternative, the survey decision engine 114 decides from the received customer interaction content that it is desirable that the customer be asked survey questions that are not represented in a standard survey. In such an embodiment, based upon specifically identified information or events in the conversation, more specific or detailed survey questions are required. In this event, the survey decision engine 114 can select questions from the stored survey questions at 118 in order to create a customized survey. In a non-limiting example of a customer service call requesting cancellation of service, while a generalized survey stored at 116 may be selected, in an alternative, the conversation information provided to the survey decision engine 114 may identify that the customer was switching to a service of a known competitor and questions specifically about that competitor can be selected from survey questions at 118 to specialize the survey. These more specific questions may be able to draw more specific information based upon the customer statements during the recorded conversation. In a further non-limiting embodiment, the speech analytics of the conversation may identify that the customer calling in for service was highly frustrated with the product/service/organization. In this event, a customized survey may be created by the decision engine 114 that has additional questions related to customer service/satisfaction. These additional questions may be in the form of a standardized customer satisfaction survey stored at 116 or individualized customer survey questions that may be specifically tailored to a particular product or service selected from 118. [0034] Once the specific questions to be used in the customer feedback survey are determined by the survey decision engine 114, dynamic survey delivery at 120 determines the form and channel across which the survey is provided to the customer. );
generating a survey comprising the personalized survey question ([0030] In an alternative, the survey decision engine 114 decides from the received customer interaction content that it is desirable that the customer be asked survey questions that are not represented in a standard survey. In such an embodiment, based upon specifically identified information or events in the conversation, more specific or detailed survey questions are required. In this event, the survey decision engine 114 can select questions from the stored survey questions at 118 in order to create a customized survey. In a non-limiting example of a customer service call requesting cancellation of service, while a generalized survey stored at 116 may be selected, in an alternative, the conversation information provided to the survey decision engine 114 may identify that the customer was switching to a service of a known competitor and questions specifically about that competitor can be selected from survey questions at 118 to specialize the survey. These more specific questions may be able to draw more specific information based upon the customer statements during the recorded conversation. In a further non-limiting embodiment, the speech analytics of the conversation may identify that the customer calling in for service was highly frustrated with the product/service/organization. In this event, a customized survey may be created by the decision engine 114 that has additional questions related to customer service/satisfaction. These additional questions may be in the form of a standardized customer satisfaction survey stored at 116 or individualized customer survey questions that may be specifically tailored to a particular product or service selected from 118. [0034] Once the specific questions to be used in the customer feedback survey are determined by the survey decision engine 114, dynamic survey delivery at 120 determines the form and channel across which the survey is provided to the customer. ); and
transmitting the survey to the customer for completion ([0034] Once the specific questions to be used in the customer feedback survey are determined by the survey decision engine 114, dynamic survey delivery at 120 determines the form and channel across which the survey is provided to the customer. A number of communication platforms and survey styles are available and each have benefits and limitations. Non-limiting examples of customer feedback survey formats include interactive voice response (IVR) surveys delivered as a part of the customer service conversation, short message service (SMS), surveys wherein the customer is prompted with a series of text messages and replies by text message, web chat surveys, email surveys where the customer receives an email and responds in an email, and web based surveys wherein the customer visits a web site and submits responses through the web site. Of course, it is understood that this is a non-limiting list of survey examples and other survey formats may be used within the scope of this disclosure, including hybrid formats that use a combination of one or more of the above noted methods. [0035] The dynamic survey delivery at 120 uses the information from the conversation and/or the selected questions for the survey in order to determine the appropriate format for the survey. For example, some survey formats, such as a web based formats, are more conducive to present greater numbers of survey questions, whereas if only a few survey questions are asked, formats such as SMS can be effective, particularly among technology savvy customers. It is to be noted that while IVR can be used at any time, including a scheduled follow up IVR customer feedback survey, if an IVR survey is going to be offered as a part of the original customer service conversation, then an embodiment with real-time speech recognition is necessary in order for the survey questions to be selected and surveys to be prepared for presentation to the customer by the time that the customer service conversation ends. ).
Jeffs is related to provide customer assistance with the design and analysis of customer feedback surveys. The system in Jess uses different portions of the customer interactions in order to customize the surveys transmitted to the customers. Jeffs uses technology such general purpose central processing units, application specific processors, and logic devices [019] to perform the process and analytics such as large-vocabulary continuous speech recognition (LVCSR) or other speech-to-text techniques. Other analytic approaches may include, but are not limited to, keyword spotting, phonetic indexing, and phonetic searching [027]. Jeffs fails to disclose the use of large language model to generate the customized survey.
However, Lin, which is similarly directed to providing customer assistance by using a large language model to generate a chat environment to assist a customer further teaches:
generating a large language model (LLM) prompt by:
generating a first portion of the LLM prompt (See Figures 1-2 and 0037] In various examples, the prompt templates 102 may include one or more variables. Prior to use of the prompt templates 102 with the LLM 120, in other words before customized prompts generated using the prompt templates 102 being provided to the LLM 120, a value of the one or more variables may be set. The value may be set based at least on the user data 104 and/or other historical data regarding a user, current user input (such as user input received via the user interface 110), and so on.);
generating a second portion of the LLM prompt (See Figures 1-2 and [0037] In various examples, the prompt templates 102 may include one or more variables. Prior to use of the prompt templates 102 with the LLM 120, in other words before customized prompts generated using the prompt templates 102 being provided to the LLM 120, a value of the one or more variables may be set. The value may be set based at least on the user data 104 and/or other historical data regarding a user, current user input (such as user input received via the user interface 110), and so on.):
applying the first portion of the LLM prompt and the second portion of the LLM prompt to an LLM prompt template, to obtain the LLM prompt and submitting the LLM prompt to an LLM (See Figures 1-2 and [0036] The system 100a may control interaction with the LLM 120 using one or more of the prompt templates 102 that may be selected, customized, and/or modified based on information stored in the user data 104 and that is known about the user with whom the LLM 120 will be interacting. The prompt templates may control the role that the LLM 120 plays, implement one or more boundaries on the LLM 120 (such as one or more boundaries indicating what is relevant for output), configure the format of LLM 120 output, configure the tone of the LLM 120 output, and so on. In an example where the LLM 120 may be used for a food chat, the prompt template may specify what the user ate, the user's known preferences, the user's known allergies, and so on. The prompt templates may be used to generate customized prompts that have been customized for specific users before being provided to the LLM 120. [0038] The variables may enable additional customization of responses for particular users beyond the customization already provided by selecting prompt templates based at least on the information stored in the user data 104 and that is known about the user with whom the LLM 120 will be interacting without having to generate a different prompt template for each user. In other words, prompt templates plus values set for variables may be used to generate specific customized prompts for particular users programmatically. This may balance the amount of storage resources necessary to store prompt templates with the amount of processing time required to generate individual prompts for particular users.).
Therefore, it would have been obvious to one of ordinary skill in the art at the time the invention was filled to replace basic computer logic and improve the system in Jeffs with the use large language model (LLM) technology since such improvement in the system of Jeffs is a simple substitution of a known technology (generic software) for another providing predictable results such as a controlled artificial intelligence environment that further may control interaction with the LLM using prompt templates that may be selected, customized, and/or modified based on information known about the user with whom the LLM will be interacting as disclosed by Lin [033]
Regarding claims 2, and 11 Jeffs disclose:
the first performance parameter comprises a sentiment score corresponding to at least one of the plurality of interactions ([027] Analytics can be used at 108 to identify topics, trends, content, or sentiment of the conversation. In embodiments, the analytics may apply an ontology or predefined set of rules or structures for interpreting the customer interaction content. The results of the analytics can be gleaned for all transcripts, metadata pulled from open responses of the customer, categorized spoken content, and indexing/frequency of words used in the conversation. ); and
the operations further comprise identifying the target interaction from the plurality of interactions based on the sentiment score (See [027-028] disclosing the system identifying a customer interaction based on the sentiment analytics.).
Regarding claims 3, and 12 Jeffs disclose:
specifying the target interaction, the interaction metadata corresponding to the target interaction, and the sentiment score corresponding to the target interaction (See [027-028] disclosing the system identifying a customer interaction based on the sentiment analytics and interaction metadata [0029] From either processing technique, speech or textual recognition is used to identify specific topics, events, sentiment and metadata from a conversation between a customer and a customer service agent. ).
Lin further teaches:
generating the LLM prompt [0036] The system 100a may control interaction with the LLM 120 using one or more of the prompt templates 102 that may be selected, customized, and/or modified based on information stored in the user data 104 and that is known about the user with whom the LLM 120 will be interacting. The prompt templates may control the role that the LLM 120 plays, implement one or more boundaries on the LLM 120 (such as one or more boundaries indicating what is relevant for output), configure the format of LLM 120 output, configure the tone of the LLM 120 output, and so on. In an example where the LLM 120 may be used for a food chat, the prompt template may specify what the user ate, the user's known preferences, the user's known allergies, and so on. The prompt templates may be used to generate customized prompts that have been customized for specific users before being provided to the LLM 120. [0038] The variables may enable additional customization of responses for particular users beyond the customization already provided by selecting prompt templates based at least on the information stored in the user data 104 and that is known about the user with whom the LLM 120 will be interacting without having to generate a different prompt template for each user. In other words, prompt templates plus values set for variables may be used to generate specific customized prompts for particular users programmatically. This may balance the amount of storage resources necessary to store prompt templates with the amount of processing time required to generate individual prompts for particular users.).
Therefore, it would have been obvious to one of ordinary skill in the art at the time the invention was filled to replace basic computer logic and improve the system in Jeffs with the use large language model (LLM) technology since such improvement in the system of Jeffs is a simple substitution of a known technology (generic software) for another providing predictable results such as a controlled artificial intelligence environment that further may control interaction with the LLM using prompt templates that may be selected, customized, and/or modified based on information known about the user with whom the LLM will be interacting as disclosed by Lin [033]
Regarding claims 4, and 12 Jeffs disclose:
wherein generating the survey comprises:
comparing the personalized survey question to previous survey questions for the customer; and based on the comparing, combining the personalized survey question with one or more predetermined survey questions ([0029] From either processing technique, speech or textual recognition is used to identify specific topics, events, sentiment and metadata from a conversation between a customer and a customer service agent. This information is provided to a survey decision engine at 114. The survey decision engine uses the information received from the application of the batch or real-time speech recognition and analytics to decide on the content of a survey to be provided to the customer participating in the conversation. The survey decision engine 114 may exemplarily decide between pre-formed surveys stored in a computer readable medium at 116 or from a plurality of survey questions stored at a computer readable medium at 118. If the survey decision engine 114 decides, based upon the received information about the conversation, that the customer's experience and service inquiries are best represented in a standardized survey, then the survey decision engine 114 selects one of these pre-made surveys. For example, an organization could create a plurality of standardized surveys, dealing with customer service events such as purchase, a return, a product complaint, or cancelling service. [0030] In an alternative, the survey decision engine 114 decides from the received customer interaction content that it is desirable that the customer be asked survey questions that are not represented in a standard survey. In such an embodiment, based upon specifically identified information or events in the conversation, more specific or detailed survey questions are required. In this event, the survey decision engine 114 can select questions from the stored survey questions at 118 in order to create a customized survey. In a non-limiting example of a customer service call requesting cancellation of service, while a generalized survey stored at 116 may be selected, in an alternative, the conversation information provided to the survey decision engine 114 may identify that the customer was switching to a service of a known competitor and questions specifically about that competitor can be selected from survey questions at 118 to specialize the survey. These more specific questions may be able to draw more specific information based upon the customer statements during the recorded conversation. In a further non-limiting embodiment, the speech analytics of the conversation may identify that the customer calling in for service was highly frustrated with the product/service/organization. In this event, a customized survey may be created by the decision engine 114 that has additional questions related to customer service/satisfaction. These additional questions may be in the form of a standardized customer satisfaction survey stored at 116 or individualized customer survey questions that may be specifically tailored to a particular product or service selected from 118. [0031] In a still further embodiment, the survey may be constructed by completing question structures with content or other information obtained through the customer service interactions to tailor the questions to the experiences and concerns of the customer. In other embodiments, a combination of stock surveys, survey questions, or question structures may be used to construct the survey for the customer.).
Regarding claims 5, and 14 Jeffs disclose:
specifying a selection of a survey question format for the personalized survey question from a set comprising: a Likert Scale, a binary, multiple choice, or open-ended ([0035] The dynamic survey delivery at 120 uses the information from the conversation and/or the selected questions for the survey in order to determine the appropriate format for the survey. For example, some survey formats, such as a web based formats, are more conducive to present greater numbers of survey questions, whereas if only a few survey questions are asked, formats such as SMS can be effective, particularly among technology savvy customers. [036] In one non-limiting example, a plurality of specifically selected survey questions from 118 may be presented to the customer as part of the original conversation using an IVR survey, while a more generalized survey from 116 is presented to the customer in an email format in the days after the customer service interaction. Claim 5 wherein the common format is a text format).
Lin further teaches:
generating the LLM prompt [0036] The system 100a may control interaction with the LLM 120 using one or more of the prompt templates 102 that may be selected, customized, and/or modified based on information stored in the user data 104 and that is known about the user with whom the LLM 120 will be interacting. The prompt templates may control the role that the LLM 120 plays, implement one or more boundaries on the LLM 120 (such as one or more boundaries indicating what is relevant for output), configure the format of LLM 120 output, configure the tone of the LLM 120 output, and so on. In an example where the LLM 120 may be used for a food chat, the prompt template may specify what the user ate, the user's known preferences, the user's known allergies, and so on. The prompt templates may be used to generate customized prompts that have been customized for specific users before being provided to the LLM 120. [0038] The variables may enable additional customization of responses for particular users beyond the customization already provided by selecting prompt templates based at least on the information stored in the user data 104 and that is known about the user with whom the LLM 120 will be interacting without having to generate a different prompt template for each user. In other words, prompt templates plus values set for variables may be used to generate specific customized prompts for particular users programmatically. This may balance the amount of storage resources necessary to store prompt templates with the amount of processing time required to generate individual prompts for particular users.).
Therefore, it would have been obvious to one of ordinary skill in the art at the time the invention was filled to replace basic computer logic and improve the system in Jeffs with the use large language model (LLM) technology since such improvement in the system of Jeffs is a simple substitution of a known technology (generic software) for another providing predictable results such as a controlled artificial intelligence environment that further may control interaction with the LLM using prompt templates that may be selected, customized, and/or modified based on information known about the user with whom the LLM will be interacting as disclosed by Lin [033]
Regarding claims 6, and 15 Jeffs disclose:
wherein the performance metadata comprises one or more of the following: sentiment score, response time, number of transfers, number of escalations, number of agents, or response interval ([0013] In the text analytics at 304, the customer interaction content is analyzed with an ontology to identify content, themes, and/or customer sentiments in the customer interaction content. [027] The metadata included with the call recording can be used in embodiments of the analytics in order to improve the quality of the analysis and information pulled from the recorded conversation. Analytics can be used at 108 to identify topics, trends, content, or sentiment of the conversation).
Regarding claims 7, and 16 Jeffs disclose:
responsive to receiving a completed survey from the customer, determine an overall satisfaction score for the survey and an explanation of the overall satisfaction score ([0037] The customer's feedback survey responses are collected at 122. As previously disclosed, the customer feedback surveys can come in a variety of formats, and each of these formats solicit different types of formatted survey responses. The responses from across numerous channels, such as IVR, SMS, email, and web based surveys, are collected from these various platforms and are analyzed to identify and interpret the content of those responses. In non-limiting examples, if a question calls for a numerical or quantitative answer, these quantitative answers are all recorded in relation to the specific questions. In free form responses, which may be either spoken responses, as in the case of IVR, or textual responses as in the case of email or web based surveys, the text is recorded and analyzed to identify and characterize the content of the responses. All of the survey responses collected at 122 are then stored at 124. In the storage of the survey responses, not only is each response stored, but also the relationship of the responses both to the customer that provided them as well as the survey question that solicited the recorded response. In some embodiments, the survey response may further be stored as customer interaction content to be used in the generation and use of future customer feedback surveys. [0038] At 126 the survey results can be analyzed either on their own or in connection with the stored speech analytics results, or other customer metadata collected by the organization. In one example, the survey responses to a particular question can be separated by a particular type of metadata about the customers that provide those responses. As a non-limiting example, the customers are divided into two groups based upon whether the customer cancels service or increases service. The survey responses can then also be divided based on the customers that exhibit this criteria. In this manner, a better picture or understanding of the customer feedback of particular sets of customers can be looked at together or comparatively against another group. In a particular advantage of embodiments, association of customer metadata to the surveys or survey response offer more available categories for sorting and analysis rather than reliance upon survey demographic or contextual question answers. ).
Lin further teaches:
applying a machine learning model to the completed survey ([0087] At operation 330, the electronic device may use the model to evaluate LLM interaction. The evaluation may be performed while the LLM is operating or offline. In some examples, the evaluation of the LLM interaction may be used to modify one or more of the prompts, the user data, the LLM, and so on. Various configurations are possible and contemplated without departing from the scope of the present disclosure. )
Therefore, it would have been obvious to one of ordinary skill in the art at the time the invention was filled to replace basic computer logic and improve the system in Jeffs with the use machine learning technology since such improvement in the system of Jeffs is a simple substitution of a known technology (generic software) for another providing predictable results such as a controlled artificial intelligence environment that further may control interaction with the LLM using prompt templates that may be selected, customized, and/or modified based on information known about the user with whom the LLM will be interacting as disclosed by Lin [033].
Regarding claims 8, and 17 Jeffs further disclose
overall satisfaction score ([0037] The customer's feedback survey responses are collected at 122. As previously disclosed, the customer feedback surveys can come in a variety of formats, and each of these formats solicit different types of formatted survey responses. The responses from across numerous channels, such as IVR, SMS, email, and web based surveys, are collected from these various platforms and are analyzed to identify and interpret the content of those responses. In non-limiting examples, if a question calls for a numerical or quantitative answer, these quantitative answers are all recorded in relation to the specific questions. In free form responses, which may be either spoken responses, as in the case of IVR, or textual responses as in the case of email or web based surveys, the text is recorded and analyzed to identify and characterize the content of the responses. All of the survey responses collected at 122 are then stored at 124. In the storage of the survey responses, not only is each response stored, but also the relationship of the responses both to the customer that provided them as well as the survey question that solicited the recorded response. In some embodiments, the survey response may further be stored as customer interaction content to be used in the generation and use of future customer feedback ).
Lin further teaches:
updating a training dataset for a machine learning model and applying the updated training dataset to a machine learning algorithm to train the machine learning model ([0087] At operation 330, the electronic device may use the model to evaluate LLM interaction. The evaluation may be performed while the LLM is operating or offline. In some examples, the evaluation of the LLM interaction may be used to modify one or more of the prompts, the user data, the LLM, and so on. Various configurations are possible and contemplated without departing from the scope of the present disclosure. )
Therefore, it would have been obvious to one of ordinary skill in the art at the time the invention was filled to replace basic computer logic and improve the system in Jeffs with the use machine learning technology since such improvement in the system of Jeffs is a simple substitution of a known technology (generic software) for another providing predictable results such as a controlled artificial intelligence environment that further may control interaction with the LLM using prompt templates that may be selected, customized, and/or modified based on information known about the user with whom the LLM will be interacting as disclosed by Lin [033].
Regarding claims 9, and 18 Jeff further teaches:
for individual survey questions of a plurality of survey questions included in the survey, determining a type of an individual survey question and a response to the individual survey question; and associating the types of the individual survey questions and the responses to the individual survey questions with the overall satisfaction score ([0037] The customer's feedback survey responses are collected at 122. As previously disclosed, the customer feedback surveys can come in a variety of formats, and each of these formats solicit different types of formatted survey responses. The responses from across numerous channels, such as IVR, SMS, email, and web based surveys, are collected from these various platforms and are analyzed to identify and interpret the content of those responses. In non-limiting examples, if a question calls for a numerical or quantitative answer, these quantitative answers are all recorded in relation to the specific questions. In free form responses, which may be either spoken responses, as in the case of IVR, or textual responses as in the case of email or web based surveys, the text is recorded and analyzed to identify and characterize the content of the responses. All of the survey responses collected at 122 are then stored at 124. In the storage of the survey responses, not only is each response stored, but also the relationship of the responses both to the customer that provided them as well as the survey question that solicited the recorded response. In some embodiments, the survey response may further be stored as customer interaction content to be used in the generation and use of future customer feedback ).
Lin further teaches:
training dataset for the machine learning model ([0087] At operation 330, the electronic device may use the model to evaluate LLM interaction. The evaluation may be performed while the LLM is operating or offline. In some examples, the evaluation of the LLM interaction may be used to modify one or more of the prompts, the user data, the LLM, and so on. Various configurations are possible and contemplated without departing from the scope of the present disclosure. )
Therefore, it would have been obvious to one of ordinary skill in the art at the time the invention was filled to replace basic computer logic and improve the system in Jeffs with the use machine learning technology since such improvement in the system of Jeffs is a simple substitution of a known technology (generic software) for another providing predictable results such as a controlled artificial intelligence environment that further may control interaction with the LLM using prompt templates that may be selected, customized, and/or modified based on information known about the user with whom the LLM will be interacting as disclosed by Lin [033].
Regarding claim 20 Jeffs disclose:
the first performance parameter comprises a sentiment score corresponding to at least one of the plurality of interactions ([027] Analytics can be used at 108 to identify topics, trends, content, or sentiment of the conversation. In embodiments, the analytics may apply an ontology or predefined set of rules or structures for interpreting the customer interaction content. The results of the analytics can be gleaned for all transcripts, metadata pulled from open responses of the customer, categorized spoken content, and indexing/frequency of words used in the conversation. ); and
the operations further comprise identifying the target interaction from the plurality of interactions based on the sentiment score (See [027-028] disclosing the system identifying a customer interaction based on the sentiment analytics.);
specifying the target interaction, the interaction metadata corresponding to the target interaction, and the sentiment score corresponding to the target interaction (See [027-028] disclosing the system identifying a customer interaction based on the sentiment analytics and interaction metadata [0029] From either processing technique, speech or textual recognition is used to identify specific topics, events, sentiment and metadata from a conversation between a customer and a customer service agent. ).
Lin further teaches:
generating the LLM prompt [0036] The system 100a may control interaction with the LLM 120 using one or more of the prompt templates 102 that may be selected, customized, and/or modified based on information stored in the user data 104 and that is known about the user with whom the LLM 120 will be interacting. The prompt templates may control the role that the LLM 120 plays, implement one or more boundaries on the LLM 120 (such as one or more boundaries indicating what is relevant for output), configure the format of LLM 120 output, configure the tone of the LLM 120 output, and so on. In an example where the LLM 120 may be used for a food chat, the prompt template may specify what the user ate, the user's known preferences, the user's known allergies, and so on. The prompt templates may be used to generate customized prompts that have been customized for specific users before being provided to the LLM 120. [0038] The variables may enable additional customization of responses for particular users beyond the customization already provided by selecting prompt templates based at least on the information stored in the user data 104 and that is known about the user with whom the LLM 120 will be interacting without having to generate a different prompt template for each user. In other words, prompt templates plus values set for variables may be used to generate specific customized prompts for particular users programmatically. This may balance the amount of storage resources necessary to store prompt templates with the amount of processing time required to generate individual prompts for particular users.).
Therefore, it would have been obvious to one of ordinary skill in the art at the time the invention was filled to replace basic computer logic and improve the system in Jeffs with the use large language model (LLM) technology since such improvement in the system of Jeffs is a simple substitution of a known technology (generic software) for another providing predictable results such as a controlled artificial intelligence environment that further may control interaction with the LLM using prompt templates that may be selected, customized, and/or modified based on information known about the user with whom the LLM will be interacting as disclosed by Lin [033].
Response to Arguments
Applicant's arguments filed 04/28/2026 have been fully considered but they are not persuasive.
In regards 35 USC 101, Applicant argues:
- “Amended claim 1 does not recite organizing human activity… claim 1 does not involve human activities” Examiner respectfully disagrees. The limitations recited in the claims are reciting the collection of interaction information, related to a service request, for the purpose of managing the interaction between a service provider and a customer, which falls within the judicial exception of certain methods of organizing human activity. The collection and evaluation of information regarding interactions between participants in a service process (customer and service provider) are activities that constitute managing interpersonal interactions and commercial interactions, since the received information is used to determine follow-up questions that should be presented to the customer in order to receive additional information related to the service request. These processes could be performed mentally or through ordinary human judgment by a person reviewing the interaction data received and identified and selecting appropriate survey questions based on the gathered data related to the service request. Accordingly, the claimed limitations are directed to organizing and managing human interactions associated with providing and evaluating a service.
- “Applicant respectfully submits that claim 1 is analogous to SRI Int’l, Inc. v. Cisco Systems, Inc., 930.3d 1295, 1304 (Fed. Cir. 2019)” Examiner respectfully disagrees. Applicant’s claimed invention is not analogous to the claims found eligible in SRI International Inc. v. Cisco Systems. In SRI, the Federal Circuit determined that the claims were directed to a specific technological improvement in computer network security by providing a hierarchical framework of network monitors capable of detecting suspicious activity across an enterprise network in a manner that improved the operation of the network itself. The claimed addressed a technological problem arising in computer networks and recited a specific technological solution that improved network intrusion detection capabilities. In contrast, the claims at hand are directed to receiving information related to interactions associated with a service request of a customer and using that information to trigger customized survey questions. As previously presented, the claimed invention is directed to managing customer service interactions and collecting feedback, which is a business practice involving interpersonal or commercial interactions. The claims do not address a problem specific to computer technology and do not recite any improvement to the functioning of a computer or recited technology. Rather, the technology is invoked merely a s a tool to perform the abstract process of receive information, evaluate conditions and present a customized survey. Unlike the claims in SRI, the present claims do not improve the underlying operation, performance or security of a computer system. Instead, any improvement is directed to the efficiency or effectiveness of administering customer surveys, which is an improvement to a business process rather than to computer technology.
-“Claim 1 is patent-eligible under Step 2A, Prong 2 for reasons similar to Ex Parte Desjardins.” Examiner respectfully disagrees. Applicant’s reliance on the use of interaction metadata to generate a prompt for a Large Language Model does not render the claims analogous to Desjardins. Although the claims recite generating an LLM prompt and using metadata associated with a target interaction to control when and how personalized survey questions are generated, the claimed remains directed to the abstract idea of evaluating information regarding a service interaction and determining appropriate customized questions. The interaction metadata merely serves as an input to a decision-making process, and the LLM is invoked as a tool for producing survey questions based on that information. It is noted that the claims do not recite an improvement in the operation of the LLM, prompt generation techniques, computer architecture, data storage, or any other computer technology. It is further noticed that the claims do not recite a particular technological mechanism that improves how metadata is collected, processes, or transmitted. Instead, the LLM performed its ordinary function of generating text in response to a prompt, while the metadata provides content for the prompt. Using available information to formulate instructions for a generative model constitutes use of generic technology to automate a process that could otherwise be performed by a human reviewer selecting or drafting customized survey questions based on the circumstances of the interaction. Therefore, the recited LLM prompt generation merely limits the abstract idea to implementation in a particular technological environment.
- “Unlike the claims in Ameranth, amended claim 1 does not merely define a desired result. Instead, amended claim 1 recites a specific, parameter-triggered computational process that controls when and how the system invokes an LLM to generate survey content. Thus, claim 1 recites a technical process that limits survey generation and controls survey content based on interactions associated with performance parameters that satisfy threshold conditions. As described in the specification, doing so reduces the computational overhead associated with survey generation for interactions that do not meet the defined threshold. (See Specification at II 22-23.) Accordingly, amended claim 1 improves the functioning of a computer under MPEP § 2106.05(a). Therefore, claim 1 integrates any alleged abstract idea into a practical application under Step 2A, Prong Two.” Examiner respectfully disagrees. As explained above, the claims at hand are directed to managing customer service interactions and collecting feedback, which is a business practice involving interpersonal or commercial interactions. The claims do not address a problem specific to computer technology and do not recite any improvement to the functioning of a computer or recited technology. Rather, the technology is invoked merely a s a tool to perform the abstract process of receive information, evaluate conditions and present a customized survey, any improvement is directed to the efficiency or effectiveness of administering customer surveys, which is an improvement to a business process rather than to computer technology. Applicant claims that the invention reduces the “the computational overhead associated with survey generation for interactions that do not meet the defined threshold”, however any reduction in computational activity results solely from choosing not to execute the conventional survey generation process when the defined threshold is not met. Such a reduction is a consequence of performing fewer instances of the claimed business process, rather than an improvement on how the computer performs the process. Therefore, the alleged reduction in computational overhead is related to any system that conserves resources by performing fewer tasks but not to a system that improved on how the computer performs the tasks. This is obtained by a business decision regarding when surveys are generated not an improvement on how the computer or the LLM technology performs the survey generation process.
In regards 35 USC 103, Applicant argues:
-“In response to applicant’s argument that there is no teaching, suggestion, or motivation to combine the references, the examiner recognizes that obviousness may be established by combining or modifying the teachings of the prior art to produce the claimed invention where there is some teaching, suggestion, or motivation to do so found either in the references themselves or in the knowledge generally available to one of ordinary skill in the art. See In re Fine, 837 F.2d 1071, 5 USPQ2d 1596 (Fed. Cir. 1988), In re Jones, 958 F.2d 347, 21 USPQ2d 1941 (Fed. Cir. 1992), and KSR International Co. v. Teleflex, Inc., 550 U.S. 398, 82 USPQ2d 1385 (2007). In this case, Jeffs is related to provide customer assistance with the design and analysis of customer feedback surveys. The system in Jess uses different portions of the customer interactions in order to customize the surveys transmitted to the customers. Jeffs uses technology such general purpose central processing units, application specific processors, and logic devices [019] to perform the process and analytics such as large-vocabulary continuous speech recognition (LVCSR) or other speech-to-text techniques. Other analytic approaches may include, but are not limited to, keyword spotting, phonetic indexing, and phonetic searching [027]. Jeffs fails to disclose the use of large language model to generate the customized survey. However Lin, which is similarly directed to providing customer assistance by using a large language model to generate a chat environment to assist a customer, is introduced to demonstrate the well-known practice of using LLM technology in providing customer assistance.
-“ Jeffs does not disclose or suggest, at least: detecting if a first performance parameter of the plurality of performance parameters exceeds a threshold value; in response to determining that the first performance parameter exceeds the threshold value, triggering a personalized survey question generation process.” Examiner respectfully disagrees. Jeff, at least on paragraphs [028-029] disclose the system detecting performance parameters being analyzed and determined to be exceeding a threshold in order to trigger survey questions. Jeff discloses analytics applied to the received interactions, wherein such analysis, if determined to be exceeding a threshold related to performance (i.e. specific topics, events, sentiment and metadata from a conversation between a customer and a customer service agent), trigger the generation of the content of a survey provided to the customer associated with the service request.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
GUIN, US 2025/0238842 SYSTEM AND METHOD FOR PROVIDING PERSONALIZED CONVERSATION-CONTENT FOR OUTBOUND MARKETING CAMPAIGNS OPERATED VIA A CLOUD-BASED CONTACT CENTER PLATFORM. The present disclosure relates to the field of data analysis and more specifically, to computerized systems and methods for providing personalized content for an interaction of an agent with a customer during an outbound conversational marketing campaign of a tenant that is operated via a cloud-based contact center platform.
Ellison US 2021/0350385 ASSISTANCE FOR CUSTOMER SERVICE AGENTS, Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for implementing assistance for customer service agents are disclosed. In one aspect, a method includes the actions of receiving, by a computing device, customer interaction data that reflects an interaction between a first user and a second user.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MARIA C SANTOS-DIAZ whose telephone number is (571)272-6532. The examiner can normally be reached Monday-Friday 8:00AM-5:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sarah Monfeldt can be reached at 571-270-1833. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MARIA C SANTOS-DIAZ/ Primary Examiner, Art Unit 3629
Prosecution Insights