DETAILED ACTION
Acknowledgement
This office action is in response to the request for continued examination (RCE) filed on 05/29/2026.
Status of Claims
Claims 3, 4, 7, 14, and 19 have been cancelled.
Claims 1, 6, 11, 13, 16, 18, and 21-24 have been amended.
Claims 1, 6, 8-11, 13, 15, 16, 18, and 20-24 are now pending.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 05/29/2026 has been entered.
Response to Arguments
Applicant's arguments filed on 05/29/2026 regarding the 35 U.S.C. 101 and 103 rejections of the claims have been fully considered. The Applicant argues the following:
(1) As per the 101 rejection, the Applicant argues, in summary, that (i) the claims are not directed to a mental process or other judicial exception, the claims are directed to a specific neural-network based technique that improves a technological process, (ii) the claims are similar to MPEP Example 39 in being directed to a specific neural-network based technique that improves a technological process, and (iii) the claims provides a technical solution to a technical problem by “automatically classifying sources of repeats interactions by transforming data into vectors…and applying ML models”.
The Examiner respectfully disagrees. The Examiner analyzed the claims as a whole and determined that the claims are still directed to the abstract groupings of Mental Processes and Certain Methods of Organizing Human Activity because the claims recite and describe a process of receiving and analyzing customer data to identify/classify causes of repeat interaction (i.e. mental processes) and to route customer to a particular agent (i.e. certain methods of organizing human activity). Per MPEP 2106.04(a), a claim recites a judicial exception when the judicial exception is “set forth” or “described” in the claim. The Applicant’s claims being directed to an abstract idea is the main difference between the Applicant’s claims and Example 39. Example 39 did not recite an abstract idea and therefore, the claims were eligible at step 2A prong 1. It was not due to the claims reflecting an abstract idea and then improving a technological process in step 2A prong 2.
The Examiner maintains the position that the additional elements recited in the claims and listed in Steps 2A(2) and 2B do not integrate the abstract idea into a practical application nor provide significantly more because the additional elements do not improve the functioning of a computer, improve another technology or technical field, or provide a technical solution to a technical problem. The claims reflect the use of ML/neural network technology to perform an abstract process and generally links the use this abstract idea to a call center environment. Applying an abstract idea on a computer and/or generally linking the use of the abstract idea to a particular technological environment does not integrate a judicial exception into a practical application or provide an inventive concept (see MPEP 2106.05 (f) and (h)). Automating an abstract process of classifying sources of repeat interaction is not considered a technological improvement. Mere automation of a manual process is not sufficient to show an improvement in technology (MPEP 2106.05(a)(I)).
In summary, the Examiner considered the claims as a whole and did not dissect the claims into old vs. new elements. The 101 analysis requires examiners to identity abstract elements in step 2A(1) and additional elements in steps 2A(2) and 2B and determine how they relate as whole in terms of integration of the abstract idea into a practical application (e.g. technical improvement) and/or providing significantly more (e.g. an inventive concept) to determine eligibility. The Examiner determined that the claims as a whole reflect the use of specific technology to perform an abstract idea. Therefore, the 35 U.S.C. 101 rejection is maintained.
(2) As per the 103 rejection, the Applicant argues, in summary, that independent claims 1, 11, and 16, as amended, are non-obvious over the cited references of Pham and Revanur.
The Examiner somewhat disagrees with the Applicant. Pham teaches the main concept of the Applicant’s claims of classifying sources (e.g. agent, customer, subject, topics, interaction drivers/reasons) of repeat interactions using ML/neural network models to transform/sequence, vectorize, and analyze heterogeneous repeat interaction data sets and to determine a response to reduce repeat call and/or improve FCR. Pham’s analysis output includes without limitation: (i) the identities of conversation participants or "content sources;" (ii) a list of subjects addressed within the content data and that identify the reasons or "driver" for why a customer initiated a shared experience; (iii) weighting data showing the relative importance or engagement associated with certain subjects; and (iv) frequency data defining the proportion of shared experiences that relate to a particular subject identifier or driver for a support request [0083]. All of Pham’s outputs/sources are related to either the agent, customer, or contact center. The Applicant’s main argument is that these output sources or causes of the repeat interactions are not explicitly recited as “customer-related”, “agent-related”, and “contact-center related”. Therefore, the Examiner has included an additional prior art reference in combination with Pham that attributes the cause of repeat interactions as customer, agent, or contact center related. Hence, the Examiner withdraws the previous 103 rejections and add a new ground of 103 rejections. See details below.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claims 8-10 are objected to because of the claims recite the limitation of “The classification and resolution system of claim 1” instead of “The system of claim 1” as recited in dependent claim 6. Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 8-10, 15, and 20 are rejected under 35 U.S.C. 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
Claims 8-10, 15, 20 include the limitation of “the reason ranking model”. There is insufficient antecedent basis for this limitation because “a reason ranking model” was first introduced in preceding claims 1 , 11, nor 16 in which claims 8-10, 15, and 20 depend upon respectively. Therefore, claims 8-10, 15, and 20 are considered indefinite and are rejected under 35 U.S.C. 112(b).
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 6, 8-11, 13, 15, 16, 18, and 20-24 are rejected under 35 U.S.C. 101 because the claimed invention, “Effortless Customer Contact and Increased First Call Resolution System and Methods”, is directed to the abstract ideas, specifically Mental Processes and Certain Methods of Organizing Human Activity, without significantly more. The claims as a whole do not include additional elements that integrate the abstract idea into a practical application or are sufficient to amount to significantly more than the abstract idea because the additional elements individually or in combination provide mere instructions to implement the abstract idea on a computer.
Step 1: Claims 1, 6, 8-11, 13, 15, 16, 18, and 20-24 are directed to a statutory category, namely a machine (claims 1, 6, 8-10, and 21-22), a process (claims 11, 13, 15, and 23), and a manufacture (claims 16, 18, 20, and 24).
Step 2A (1): Independent claims 1, 11, and 16 are directed to an abstract idea of Mental Processes and Certain Methods of Organizing Human Activity, based on the following claim limitations: “ classifying sources of repeat interactions…; receiving,…, a repeat interaction from a customer after a first interaction with a first agent, wherein the repeat interaction comprises a contact information sequence associated with previous instances of interactions from the customer, and wherein the contact information sequence comprises a chronological sequence of previous interactions associated with the customer; retrieving, from the repeat interaction, metadata associated with the repeat interaction, wherein the metadata comprises a history of the customer with the contact center, historical statistics of the first agent, skill statistics of the first agent, and contact center information on the first interaction; determining a sequence of concatenated vectors by concatenating and vectorizing the history of the customer with the contact center, the historical statistics of the first agent, the skill statistics of the first agent, and the contact center information on the first interaction; generating,…, a single vector using the sequence of concatenated vectors; automatically determining, …, a probability for each source of a plurality of sources, wherein the plurality of sources of the repeat interaction comprises a customer-related factor, an agent-related factor, or a contact center-related factor, wherein each probability indicates a likelihood that a corresponding source is a cause of the repeat interaction; classifying, … based on the generated probabilities, the source of the repeat interaction as one or more sources having probabilities greater than a threshold probability; automatically routing, based on the classified source, the repeat interaction to at least one of the first agent or a different agent”. These claim limitations describe a process of receiving and analyzing customer, agent, and contact center data, creating vectors or vectorization (i.e. numerical arrays) via modelling to determine causes of repeat customer interactions, which can be practically performed in the human mind via observation, evaluation, and judgements with pen and paper (e.g. root cause analysis). Routing (i.e. assigning) repeat calls to an agent or different agents based on analysis of performance is an act of managing employee work behavior and managing the interaction between a customer and an agent (i.e. people). Dependent claims 6, 8-10, 13, 15, 18, and 20-24 further describe the process of the data analysis (e.g. reason ranking and agent statistics), modelling (e.g. training models, evaluating/verifying accuracy), identification of the cause (e.g. customer, agent, or contact center related), and the response to the cause (e.g. routing/assigning, modifying KPIs) with the limitations of “generating,… based on the determined source of the repeat interaction, one or more ranked reasons for the repeat interaction; and automatically routing, based on the one or more ranked reasons, the repeat interaction (i) to the first agent when within a reconnection buffer period or (ii) to the different agent when the classified source corresponds to the agent-related factor (claims 6, 13, and 18); wherein the operations further comprise training the source classification model and the reason ranking model (claims 8, 15, and 20); wherein training the source classification model and the reasons ranking model comprises evaluating an accuracy of the source classification model and the reason ranking model until the accuracy reaches a threshold value (claim 9); wherein the operations further comprise periodically verifying an accuracy of the source classification model and the reason ranking model (claim 10); wherein the historical statistics of the first agent comprise, for a relevant agent skill, a percentage of incomplete communication, a percentage of lacking skills or proficiency, and a percentage of incorrect information on previous interactions (claim 21); wherein the performed action comprises… modifying a repeat interaction key performance indicator (KPI) of the first agent…(claims 22-24). These are steps that a human person can perform during a root cause analysis procedure, modelling/training/validation procedure, and/or an agent/contact center performance evaluation/routing procedure. Therefore, these limitations, under the broadest reasonable interpretation, fall within the abstract groupings of Mental Processes which include concepts performed in the human mind such as observations, evaluations, judgments, and opinions and Certain Methods of Organizing Human Activity which encompasses managing personal behavior or relationships or interactions between people including social activities, teaching, and following rules or instructions. Mental Processes include claims directed to collecting information, analyzing it, and displaying certain results of the collection and analysis even if they are claimed as being performed on a computer. Certain Methods of Organizing Human Activity can encompass the activity of a single person (e.g. a person following a set of instructions), activity that involve multiple people (e.g. a commercial interaction), and certain activity between a person and a computer (e.g. a method of anonymous loan shopping). Therefore, claims 1, 6, 8-11, 13, 15, 16, 18, and 20-24 as a whole are directed to an abstract idea and are not patent eligible.
Step 2A (2): The claims as a whole do not integrate this abstract idea into a practical application. In particular, claims 1, 6, 11, 13, 16, and 18 recite additional elements of “a system… using one or more machine learning (ML) models comprising a processor and a non-transitory computer readable medium operably coupled thereto, the computer readable medium comprising a plurality of instructions stored in associated therewith that are accessible to, and executable by, the processor, to perform operations; at least one of an interactive voice response (IVR) system, an automatic call distributor (ACD), or a computer telephony integration (CTI) component; by the one or more ML models comprising a recurrent neural network (RNN); by the one or more ML models comprising a source classification model (claims 1, 11, and 16); by the one or more ML models comprising a reason ranking model (claims 6, 13, and 18); and a non-transitory computer-readable medium having stored thereon computer-readable instructions executable by a processor to perform operations (claim 16)”. These additional elements do not integrate the abstract idea into a practical application because the claims do not recite (a) an improvement to another technology or technical field and (b) an improvement to the functioning of the computer itself and (c) implementing the abstract idea with or by use of a particular machine, (d) effecting a particular transformation or reduction of an article, or (e) applying the judicial exception in some other meaningful way beyond generally linking the use of an abstract idea to a particular technological environment. These additional elements evaluated individually and in combination are viewed as computing devices that are used to perform the abstract process of receiving and analyzing customer, agent, and contact center data via modelling to determine causes of repeat customer interactions and to route repeat calls to improve FCR metrics and customer satisfaction. Per MPEP 2106.05(a), an improvement in the abstract concept (e.g. first call resolution, repeat interaction, agent performance, data transformation) is not an improvement in technology. Limitations that recite mere instructions to implement an abstract idea on a computer or merely uses a computer as a tool to perform an abstract idea are not indicative of integration into a practical application (see MPEP 2106.05(f)). Also, limitations that amount to merely indicating a field of use or technological environment (e.g. contact center) in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application (see MPEP 2106.05(h)). Therefore, claims 1, 6, 8-11, 13, 15, 16, 18, and 20-24 as a whole do not include individual or a combination of additional elements that integrate the abstract idea into a practical application and thus are not patent eligible.
Step 2B: The claims as a whole do not include additional elements that are sufficient to amount to significantly more than the abstract idea. Claims 1, 6, 11, 13, 16, and 18 recite additional elements of “a system… using one or more machine learning (ML) models comprising a processor and a non-transitory computer readable medium operably coupled thereto, the computer readable medium comprising a plurality of instructions stored in associated therewith that are accessible to, and executable by, the processor, to perform operations; at least one of an interactive voice response (IVR) system, an automatic call distributor (ACD), or a computer telephony integration (CTI) component; by the one or more ML models comprising a recurrent neural network (RNN); by the one or more ML models comprising a source classification model (claims 1, 11, and 16); by the one or more ML models comprising a reason ranking model (claims 6, 13, and 18); and a non-transitory computer-readable medium having stored thereon computer-readable instructions executable by a processor to perform operations (claim 16)”. These additional elements are viewed as mere instructions to implement an abstract idea on a computer and merely indicates a field of use or technological environment in which to apply a judicial exception. Applying an abstract idea on a computer does not integrate a judicial exception into a practical application or provide an inventive concept (see MPEP 2106.05(f)). Therefore, claims 1, 6, 8-11, 13, 15, 16, 18, and 20-24 as a whole do not include individual or a combination of additional elements that are sufficient to amount to significantly more than the abstract idea and thus are not patent eligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 11, 16, and 22-24 are rejected under 35 U.S.C. 103 as being unpatentable over Pham et al. (US 2024/0168918 A1) in view of Pearce et al. (US 2020/0329142 A1).
As per claims 1, 11, and 16 (Currently Amended) Pham teaches a system for classifying sources of repeat interactions using one or more machine learning (ML) models comprising: a processor and a non-transitory computer readable medium operably coupled thereto, the non-transitory computer readable medium comprising a plurality of instructions stored in association there with that are accessible to, and executable by, the processor, to perform operations which comprise; a method for classifying sources of repeat interactions using one or more machine learning (ML) models, which comprises; and A non-transitory computer-readable medium having stored thereon computer-readable instructions executable by a processor to perform operations which comprise (Pham e.g. The invention relates to systems and methods that automatically classify, segment, filter, and display alphanumeric content data generated during a user-provider interaction through the use of artificial intelligence and natural language processing technology [0001]. The results of the analysis in turn allow for identification of system and service problems and the implementation of system enhancements (Abstract). Fig. 1 hardware system 100 configuration according to one embodiment generally includes a user 110 that benefits through use of services and products offered by a provider through an enterprise system 200 [0085]. The storage device 124 includes at least one of a non-transitory storage medium for long-term, intermediate-term, and short term storage of computer-readable instructions 126 for execution by the processor 120 [0087].):
Pham teaches receiving, from at least one of an interactive voice response (IVR) system, an automatic call distributor (ACD), or a computer telephony integration (CTI) component, a repeat interaction from a customer after a first interaction with a first agent, wherein the repeat interaction comprises a contact information sequence associated with previous instances of interactions from the customer, and wherein the contact information sequence comprises a chronological sequence of previous interactions associated with the customer; (Pham e.g. The embodiments discussed in this specification are described with reference to systems and methods utilized in a call center environment where provider personnel are engaging in shared experiences and performing customer service activities [0084]. The provider system can be configured to generate content data manually or to obtain content data from a third party source, such as a cloud storage service or remote database [0106]. Content data is generated from a transcript of a written or verbal interactive exchange between conversation participants or "content sources." Examples of content data include, but are not limited to, an exchange of instant chat messages between two or more participants or recorded audio data generated during a telephone call (e.g., a consumer support request or help desk call), or a video conference [0082]. To further analyze instances of multiple shared experiences, the content data files stored to an interaction database can include a user identifier. The provider network computing device performs the operation of identifying interaction database records having matching user identifiers-i.e., two interactions involving the same customer or end user [0021]. The system generates a repeat driver set that is made up of those interaction database records within the repeat interaction set that have matching interaction driver identifiers. In other words, the final data set includes shared experience records where a user called multiple times (i.e., matching user identifiers) for the same reason (i.e., matching interaction driver identifiers) [0022]. The content data files are generated by recording telephonic communications between a user and an agent and converting the recorded telephonic communications to alphanumeric content data. The telephonic communications are each initiated by an incoming call comprising incoming telephonic interaction data, such as Internet Protocol ("IP") data packets (for a Voice-over-IP call) or multi-frequency tones (for a conventional phone call) [0023]. A provider can also modify its interactive voice response ("IVR") software, which outputs automated options for selection by a user, to include an option for users requiring assistance with the provider mobile application electronic transfer function. Thus, the IVR software is modified to reflect the interaction driver identifiers [0025]. Provider-user interactions generally commence when a user initiates contact with a provider by telephone or written electronic communication (e.g., email, SMS text message, an instant chat message, or a social media message) [0108]. Content metadata can include, for example: (i) sequencing data representing the date and time when the content data was created or otherwise representing an order or sequence in which a shared experience reflected in the content data occurred relative to other shared experiences; etc. [0110]. In one particular use-case, the system can display data from user-provider interactions involving a user that has initiated multiple interactions with the provider, for the same or similar reasons, within a limited time frame. For instance, identifying a user who made multiple calls to technical support in a six week time period [0020]. In this manner, the provider is able to identify subjects or interaction drivers that required multiple contacts between a user and a provider and, therefore, could represent problems or issues that are difficult to address [0020].)
Pham teaches retrieving, from the repeat interaction, metadata associated with the repeat interaction, wherein the metadata comprises a history of the customer with the contact center, historical statistics of the first agent, skill statistics of the first agent, and contact center information on the first interaction; (Pham e.g. To further analyze instances of multiple shared experiences, the content data files stored to an interaction database can include a user identifier. The provider network computing device performs the operation of identifying interaction database records having matching user identifiers-i.e., two interactions involving the same customer or end user [0021]. The system generates a repeat driver set that is made up of those interaction database records within the repeat interaction set that have matching interaction driver identifiers. In other words, the final data set includes shared experience records where a user called multiple times (i.e., matching user identifiers) for the same reason (i.e., matching interaction driver identifiers) [0022]. In other cases, the subject classification analysis can utilizes data from prior shared experiences involving the same user. If the user has contacted a provider multiple times to request an electronic transfer, that data may factor into the potential subject identifiers or interaction driver identifiers [0045]. The system can include an interaction database having interaction database records that include content data files and interaction driver identifiers generated from prior shared experiences between a provider and a plurality of end users [0045]. The network computing device retrieves the interaction database records from the interaction database, and utilizes the interaction database records to generate interaction driver identifier [0045]. The content data is stored to a database on the provider system or to a remote storage location. The content data is stored as content data files that include the substance of an exchange of communications along with content metadata [0206]. The content data is stored to a relational database that maintains the content data in a manner that permits the content data files to be associated with certain information, such as one or more subject identifiers and content metadata [0109]. Content metadata can include, for example: (i) sequencing data representing the date and time when the content data was created or otherwise representing an order or sequence in which a shared experience reflected in the content data occurred relative to other shared experiences; (ii) subject identifier data that characterizes the subjects or topics addressed within the content data (e.g., "technical support" or "new product launch demonstration"); (iii) interaction driver identifier data, which can be a subset or subcategory of subject identifier data, and that identifies the reasons why a shared experience was initiated (i.e., the reason a customer initiated the interaction can be, and typically is, a subject or topic addressed within the content data); (iv) weighting data representing the relative importance of subject identifiers through, for example, an analysis of the frequency of communication elements contributing to the subject identifier; (v) content source identifier data that identifies one or more participants to the interaction, which can include a name, an affiliated employer or business, or a job title or role and can further comprise agent identifier data or user identifier data that identifiers an agent or customer by name or identification number; (vi) provider identifier data that identifies owner of the content data; (vii) user source data, such as a telephone number, email address, or user device IP Address; (viii) sentiment data, including sentiment identifiers; (ix) polarity data indicating the relative positive or negative degree of sentiment occurring during a shared experience; (x) resolution data indicating whether a particular user issue was resolved or not, and if so, how the issue was resolved (e.g., the issue is a user forgot a password, and the resolution was a password reset); (xi) an agent identifier indicating the provider agent that participated in the shared experience; or (xii) other types of data useful for provider service to a user or processing content data [0110]. Agent attribute data can include, without limitation: ...(ii) an agent identifier,...(iv) agent service line identifier data indicating a provider department, branch, or division to which an agent is assigned; (v) an agent role designation (e.g., junior agent, senior agent, supervisor, etc.) (vii) agent experience data indicating the duration of professional experience an agent has in one or more relevant roles or in working for a provider (e.g., 2 years' experience in new account creation or 5 years and 2 months working for the provider overall); and (vii) agent training data indicating particular certifications, products, or services that an agent is trained to handle (e.g., an agent is qualified to provide technical support for a provider mobile application, or the agent is qualified to offer advice concerning a particular product or service) [0124].)
Pham teaches determining a sequence of concatenated vectors by concatenating and vectorizing the history of the customer with the contact center, the historical statistics of the first agent, the skill statistics of the first agent, and the contact center information on the first interaction; (Pham e.g. The present invention to provide systems and methods that automate the process of characterizing a user-provider interaction by converting the interaction to an alphanumeric content data format and using artificial intelligence and natural language processing ("NPL") technology to generate subject matter identifiers and sentiment identifiers that characterize the interaction [0003]. The subject classification analysis can be implemented using supervised learning techniques that require training the neural networks with labeled, known training data. The neural network can be a recurrent neural network having a long short-term memory neural network architecture [0016]. FIG. 4 is a diagram of a Recurrent Neural Network RNN, according to at least one embodiment, utilized in machine learning [0064]. An RNN may allow for analysis of sequences of inputs rather than only considering the current input data set [0184]. To implement natural language processing technology, suitable neural network architectures can include, without limitation:...(iv) recurrent neural networks; (v) Long Short-Term Memory ("LSTM") network architecture; (vi) Bidirectional Long Short-Term Memory network architecture, which is an improvement upon LSTM by analyzing word, or communication element, sequences in forward and backward directions; [0197]. The provider system then performs a subject classification analysis by processing the content data using NPL and artificial intelligence software processing techniques that are implemented using neural networks [0208]. The subject classification analysis can be implemented by neural networks that execute unsupervised learning software processing techniques that do not require substantial volumes of known and labeled training data [0213].)
Pham teaches generating, by the one or more ML models comprising a recurrent neural network (RNN), a single vector using the sequence of concatenated vectors; (Pham e.g. The content driver software service receives a plurality of interaction database records and content parameter data that includes sequencing identifiers [0010]. The content driver software service can be implemented with a neural network that executes the subject classification analysis [0014]. The content driver software service processes the content data using natural language processing technology that is implemented by one or more artificial intelligence software applications and systems [0127]. The content data is first pre-processes using a reduction analysis to create reduced content data [0135]. Following a reduction analysis, the reduced content data is vectorized to map the alphanumeric text into a vector form [0138]. One approach to vectorising content data includes applying "bag-of-words" modeling [0138]. The content data is, thus, turned into a bag-of-words that includes integer values and the number of times the integers occur in content data. The bag-of-words is turned into a unit vector (i.e. single vector), where all the occurrences are normalized to the overall length [0151]. A similar analysis can be performed on vectors created through other processing, such as Kmeans clustering or techniques that generate vectors where each word in the vector is replaced with a probability that the word represents a subject identifier or request driver data [0151].The content driver software service can also use term frequency-inverse document frequency software processing techniques to vectorize the content data and generating weighting data that weight words or particular subjects [0156]. The subject classification analysis can be implemented using supervised learning techniques that require training the neural networks with labeled, known training data. The neural network can be a recurrent neural network having a long short-term memory neural network architecture [0016]. FIG. 4 is a diagram of a Recurrent Neural Network RNN, according to at least one embodiment, utilized in machine learning [0064]. The training set content data is then fed to the content driver software service neural networks to identify subjects, content sources, or sentiments and the corresponding probabilities [0169]. An RNN may allow for analysis of sequences of inputs rather than only considering the current input data set [0184]. The provider system then performs a subject classification analysis by processing the content data using NPL and artificial intelligence software processing techniques that are implemented using neural networks [0208]. The subject classification analysis determines one or more subject identifiers that reflect subjects or topics addressed in the content data. The subject identifiers can be interaction driver identifiers, which are the reasons why a user initiated a shared experience. Unlike conventional systems, the present system is capable of efficiently and accurately identifying multiple subject or interaction driver identifiers [0208].)
Pham teaches automatically determining, by the one or more ML models comprising a source classification model using the single vector, a probability for each source of a plurality of sources, wherein the plurality of sources of the repeat interaction comprises a customer-related factor, an agent-related factor, or a contact center-related factor, wherein each probability indicates a likelihood that a corresponding source is a participant of the repeat interaction; (Pham e.g. The invention relates to systems and methods that automatically classify, segment, filter, and display alphanumeric content data generated during a user-provider interaction through the use of artificial intelligence and natural language processing technology [0001]. The content driver software service executes a subject classification analysis using the concentrated content data [0008]. The content driver software service receives a plurality of interaction database records and content parameter data that includes sequencing identifiers [0010]. The content driver software service can be implemented with a neural network that executes the subject classification analysis. The neural network performs operations that implement a Kmeans clustering analysis to execute the subject classification analysis [0014]. The content data is analyzed using natural language processing techniques that are implemented by artificial intelligence technology. The resulting outputs can include, without limitation: (i) the identities of conversation participants or "content sources;" (ii) a list of subjects addressed within the content data and that identify the reasons or "driver" for why a customer initiated a shared experience; (iii) weighting data showing the relative importance or engagement associated with certain subjects; and (iv) frequency data defining the proportion of shared experiences that relate to a particular subject identifier or driver for a support request [0083]. Content sources can include an agent and a customer or end user generating content data as part of a shared experience [0077]. Content source identifier data identifies one or more participants to the interaction [0110]. A subject is then represented by a specified number of words or phrases having the highest probabilities (i.e., the words with the five highest probabilities), or the subject is represented by text data having probabilities above a pre-determined subject probability threshold [0145]. The clustering analysis yields a group of words or communication elements associated with each cluster, which can be referred to as subject vectors. Subjects may each include one or more subject vectors where each subject vector includes one or more identified communication elements (i.e., keywords, phrases, symbols, etc.) within the content data as well as a frequency of the one or more communication elements within the content data [0147]. The content driver software service can be configured to perform an additional concentration analysis following the clustering analysis that selects a pre-defined number of communication elements from each cluster to generate a descriptor set, such as the five or ten words having the highest weights in terms of frequency of appearance (or in terms of the probability that the words or phrases represent the true subject when neural networking architecture is used). In one embodiment, the descriptor sets were analyzed to determine if the reasons driving a customer support request were identified by the descriptor set subject identifiers [0147]. An interaction driver identifier can be determined by, for example, first determining the subject identifiers having the highest weight quantifiers (e.g., frequencies or probabilities) and comparing such subject identifiers against a database of known interaction driver identifiers [0144]. To illustrate, the subject identifiers from a shared experience having the five (5) highest frequencies or probabilities might include "forgot password," "report fraud," "the weather," "children," and "covid-19." [0144]. The provider system compares the top five subject identifiers against a list of known interaction driver identifiers that includes "forgot password" and "report fraud" as a known support driver but not "weather," "children," and "covid-19." In that instance, the provider system identifiers the two support drivers as being "forgot password" and "report fraud." [0144]. In one embodiment, the subject classification analysis is performed on the content data using a Latent Drichlet Allocation analysis to identify subject data that includes one or more subject identifiers (e.g., topics addressed in the underlying content data) [0145]. Performing the LDA analysis on the reduced content data may include transforming the content data into an array of text data representing key words or phrases that represent a subject (e.g., a bag-of-words array) and determining the one or more subjects through analysis of the array. Each cell in the array can represent the probability that given text data relates to a subject [0145]. A subject is then represented by a specified number of words or phrases having the highest probabilities (i.e., the words with the five highest probabilities), or the subject is represented by text data having probabilities above a pre-determined subject probability threshold [0145]. Process content data to determine content sources within the content data (e.g. persons participating in the discussion) ([0160] and [0169]).)
Pham teaches classifying, by the source classification model and based on the generated probabilities, the source of the repeat interaction as one or more sources having probabilities greater than a threshold probability; and (Pham e.g. An interaction driver identifier can be determined by, for example, first determining the subject identifiers having the highest weight quantifiers (e.g., frequencies or probabilities) and comparing such subject identifiers against a database of known interaction driver identifiers [0144]. To illustrate, the subject identifiers from a shared experience having the five (5) highest frequencies or probabilities might include "forgot password," "report fraud," "the weather," "children," and "covid-19." [0144]. The provider system compares the top five subject identifiers against a list of known interaction driver identifiers that includes "forgot password" and "report fraud" as a known support driver but not "weather," "children," and "covid-19." In that instance, the provider system identifiers the two support drivers as being "forgot password" and "report fraud." [0144]. In one embodiment, the subject classification analysis is performed on the content data using a Latent Drichlet Allocation analysis to identify subject data that includes one or more subject identifiers (e.g., topics addressed in the underlying content data) [0145]. Performing the LDA analysis on the reduced content data may include transforming the content data into an array of text data representing key words or phrases that represent a subject (e.g., a bag-of-words array) and determining the one or more subjects through analysis of the array. Each cell in the array can represent the probability that given text data relates to a subject [0145]. A subject is then represented by a specified number of words or phrases having the highest probabilities (i.e., the words with the five highest probabilities), or the subject is represented by text data having probabilities above a pre-determined subject probability threshold [0145].)
Pham teaches automatically routing, based on the classified source, the repeat interaction to at least one of the first agent or a different agent. (Pham e.g. The system can perform a routing analysis using, for instance, agent data, customer data, or information relating to the shared experience [0039]. As an example, the routing analysis can ascertain an agent that has relevant training or experience that makes the agent most likely to successfully resolve any problems arising during the interaction, as determined by the subject identifiers or interaction driver identifiers [0039]. To execute a routing analysis, the network computing device can pass a query containing agent identifiers of available agents to an agent identity management service. In response to the query, the network computing device can receive agent attribute data, such as data relating to the training, experience level or role of particular agents. The routing analysis utilizes the agent attribute data to generate an optimal agent identifier [0048]. The routing analysis can utilize the subject identifiers, interaction driver identifiers sentiment identifiers, polarity data, end user data, agent attribute data, or other relevant data and information [0223].)
While Pham teaches using a subject classification analysis/model to identifying sources (e.g. customer, agent, content, topic, subject, interaction drivers) of repeat interactions. Pham does not explicitly teach that a customer, agent, contact center source is a cause of the repeat interaction. However, Pearce teaches identifying a customer, agent, or contact center source is a cause of the repeat interaction (Pearce e.g. A system, method, and computer program product for customer contact management via voice, chat, e-mail and social network contacts includes a balanced service process (BSP) that includes a plurality of cause or response codes for maximizing first contact resolution (FCR) and CSAT. The BSP in real-time determines dispositions of such contacts, monitors and manages the performance of individual resolvers by incorporating machine learning in said BSP (Abstract). The balanced service process (BSP) according to embodiments of the present invention generates FCR, CSAT and cause codes—issues that are driving call volume data, all in actionable or real-time. The data, when analyzed, creates action plans to address process, behavioral, and recurrent training problems and also serves as the basis for an agent's reward and recognition program [0038]. Fundamentally, the BSP relies on “customer contact points” (contact center agents, BOTs, etc.) to generate FCR and CSAT data. At the end of every contact the customer contact point must disposition each contact before another is presented: “YES” if the issue was resolved and “NO” if the issue was not resolved. Low “exception rates” (i.e., where there are different Agent and customer dispositions for the same record) and high “resolution rates” (i.e., where the Agent and customer both disposition the record as “YES”) is the goal [0042]. Low exception rates and low resolution rates suggest training is required. High exception rates and low resolution rates indicate a process or policy breakdown, and high exception rates with high resolution rates suggest Agent behavior issues (Source identifiers) [0042]. An “Exception” may comprise a different disposition/response for the same record(s) as dispositioned by the Agent and the customer [0069]. An “Exception Analysis” may comprise the determination made by QA regarding the root cause of the Exception. Among such root causes are: (a) client process; (b) CRM process; (c) Agent training; and (d) Agent behavior. The “Exception Rate” may comprise the percentage of variances in responses between the customer and the Agent (calculated as TOTAL EXCEPTIONS/TOTAL CDP's) [0070]. The three categories of Exception root cause are process, training and behavior. The following are example for each (Examples 1-3) ([0083]-[0086]). As Exception analyses are produced, trends are analyzed, and stack ranked in descending order. In the event that a CRM process is impeding FCR, and can be adjusted within the realm of fair practice, client approval may be sought, if required, and the process is adjusted. If Agent training is required to affect the process change, it is scheduled and executed upon as quickly as possible [0090].)
The Examiner submits that before the effective filing date, it would have been obvious to one of ordinary skill in the art to combine Pham’s classification analysis model with Pearce’s root cause analysis process that identify sources such as contact center processes, agent behavior, or agent training that drive call volume in order to create action plans to address process, behavioral, and recurrent training problems and lower operating expense, reduce service-related contacts and improve customer satisfaction (Pearce e.g. [0013] and [0038]).
As per claims 22, 23, and 24 (Currently Amended), Pham in view of Pearce teach the system of claim 1, wherein the processor, to perform the operations further comprises, the method of claim 11, wherein the method further comprises, and the non-transitory computer-readable medium of claim 16, wherein the processor is configured to perform the operations further comprises, modifying a repeat interaction key performance indicator (KPI) of the first agent (Pearce e.g. A system, method, and computer program product for customer contact management via voice, chat, e-mail and social network contacts includes a balanced service process (BSP) that includes a plurality of cause or response codes for maximizing first contact resolution (FCR) and CSAT. The BSP in real-time determines dispositions of such contacts, monitors and manages the performance of individual resolvers by incorporating machine learning in said BSP (Abstract). Accordingly, it is generally an object of certain embodiments of the present invention to provide a system, method, and computer program product to accurately measure and manage first contact resolution (FCR) and customer satisfaction (CSAT) at an actionable (i.e., from agent to customer) level [0010]. The balanced service process (BSP) according to embodiments of the present invention generates FCR, CSAT and cause codes-issues that are driving call volume data, all in actionable or real-time [0038]. BSP makes FCR and CSAT an intraday-managed metric, analogous to service level or handle time [0044]. Zacoustic (i.e., the performance management software which powers the balanced service process) supports and runs on BSP 208, and displays key performance metrics in real-time, including-and most importantly-FCR statistics and CSAT [0075]. Zacoustic resides on every Agent's, supervisor's, and client's desktop 230, 234, providing the necessary feedback required optimizing intraday FCR performance. Its primary function is to calculate customer satisfaction and call resolution metrics, with various associations to those metrics, into certain formats and reports [0075]. The FCR Rate and CSAT rate are immediately populated in the Zacoustic database 218 along with the number of contacts the agent/enterprise has handled within the same time period [0078].)
The Examiner submits that before the effective filing date, it would have been obvious to one of ordinary skill in the art to combine Pham’s classification analysis model with Pearce’s Balanced Service Process that generates and tracks FCR rates for agents in order to improve FCR and reduce the total volume of repeat calls, thus improving customer satisfaction (CSAT) (Pearce e.g. [0008]).
Claims 6, 8-10, 13, 15, 18, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Pham et al. (US 2024/0168918 A1) in view of Pearce et al. (US 2020/0329142 A1) and in further view of Revanur et al. (US 2020/0137231 A1).
As per claims 6, 13, and 18 (Currently Amended), Pham in view of Pearce teach the system of claim 1, wherein the processor, to perform the operations further comprises; the method of claim 11, wherein the method further comprises; the non-transitory computer-readable medium of claim 16, wherein the processor is configured to perform the operations further comprising:
Pham teaches generating, by the one or more ML models, …., and based on the determined source of the repeat interaction, one or more ranked reasons for the repeat interaction; and (Pham e.g. The invention relates to systems and methods that automatically classify, segment, filter, and display alphanumeric content data generated during a user-provider interaction through the use of artificial intelligence and natural language processing technology [0001]. The content driver software service can be implemented with a neural network that executes the subject classification analysis. The neural network performs operations that implement a Kmeans clustering analysis to execute the subject classification analysis [0014]. A subject is then represented by a specified number of words or phrases having the highest probabilities (i.e., the words with the five highest probabilities), or the subject is represented by text data having probabilities above a pre-determined subject probability threshold [0145]. The clustering analysis yields a group of words or communication elements associated with each cluster, which can be referred to as subject vectors. Subjects may each include one or more subject vectors where each subject vector includes one or more identified communication elements (i.e., keywords, phrases, symbols, etc.) within the content data as well as a frequency of the one or more communication elements within the content data [0147]. The content driver software service can be configured to perform an additional concentration analysis following the clustering analysis that selects a pre-defined number of communication elements from each cluster to generate a descriptor set, such as the five or ten words having the highest weights in terms of frequency of appearance (or in terms of the probability that the words or phrases represent the true subject when neural networking architecture is used). In one embodiment, the descriptor sets were analyzed to determine if the reasons driving a customer support request were identified by the descriptor set subject identifiers [0147]. The software model was evaluated according to three categories, including a "good match" where the support request reason(s) were identified by the top words in the subject vector (i.e., the words with the highest weight or frequency), a "moderate" match where the support request reason(s) were identified by the second tier of words in the subject vector (i.e., words six to ten), and a "poor" match where, for instance, the top words in a subject vector did not match or identify the reasons the support request was initiated [0148]. The subject classification analysis can specifically identify one or more interaction driver identifiers that are the reason why a user initiated a shared experience or support service request [0144]. An interaction driver identifier can be determined by, for example, first determining the subject identifiers having the highest weight quantifiers (e.g., frequencies or probabilities) and comparing such subject identifiers against a database of known interaction driver identifiers. To illustrate, the subject identifiers from a shared experience having the five (5) highest frequencies or probabilities might include "forgot password," "report fraud," "the weather," "children," and "covid-19." The provider system compares the top five subject identifiers against a list of known interaction driver identifiers that includes "forgot password" and "report fraud" as a known support driver but not "weather," "children," and "covid-19." In that instance, the provider system identifiers the two support drivers as being "forgot password" and "report fraud." [0144].)
Pham teaches automatically routing, based on the one or more ranked reasons, the repeat interaction (i) to the first agent when within a reconnection buffer period or (ii) to the different agent when the classified source corresponds to the agent-related factor. (Pham e.g. The provider is able to identify subjects or interaction drivers that required multiple contacts between a user and a provider and, therefore, could represent problems or issues that are difficult to address [0020]. The system can perform a routing analysis using, for instance, agent data, customer data, or information relating to the shared experience [0039]. As an example, the routing analysis can ascertain an agent that has relevant training or experience that makes the agent most likely to successfully resolve any problems arising during the interaction, as determined by the subject identifiers or interaction driver identifiers [0039]. The interaction identifiers are a subcategory of subject identifiers that focus on characterizing the reasons an end user initiated a shared experience (e.g., to purchase a new service, seek technical support, or ask for assistance in rendering a service) [0008]. To execute a routing analysis, the network computing device can pass a query containing agent identifiers of available agents to an agent identity management service. In response to the query, the network computing device can receive agent attribute data, such as data relating to the training, experience level or role of particular agents. The routing analysis utilizes the agent attribute data to generate an optimal agent identifier. The optimal agent identifier can represent an IP address, name, or other information that identifies an agent qualified to assist with a particular shared experience [0048].)
Pham nor Pearce explicitly teach, however, Revanur teaches a ML ranking model (Revanur e.g. Revanur teaches a computer system that routes contact center interactions. Interactions between contact center agents and contact center queries that are received at a contact center are monitored (Abstract). A ranking model is trained according to the categories of the contact center queries and the interaction scores of each handled query using machine learning (Abstract). A ranking model is trained according to the categories of the contact center queries, one or more selected business outcomes, and the interaction scores of each agent for each handled query using machine learning [0010]. The ranking model is tested according to various metrics in order to gauge the performance of the ranking model [0010]. Testing module 120 may test the efficacy of a ranking model in order to ensure that the ranking model properly ranks agents according to desired business outcomes [0020]. The machine learning ranking model is trained and tested at operation 330 by generating interaction scores [0044]. Model generating module 115 may train the ranking model using conventional or other machine learning techniques. In some embodiments, the ranking model is trained using a machine learning approach based on matrix factorization, restricted Boltzmann machines, and/or singular value decomposition [0044]. Once model generating module 115 trains the ranking model, testing module 120 may test the model [0044].)
The Examiner submits that before the effective filing date, it would have been obvious to one of ordinary skill in the art to combine Pham in view of Pearce’s classification analysis model with Revanur’s ranking model in order to analyze and route interactions more effectively (Revanur e.g. [0053]).
As per claims 8, 15, and 20 (Original), Pham in view of Pearce teach the classification and resolution system of claim 1, wherein the operations further comprise; the method of claim 11, which further comprises; and the non-transitory computer-readable medium of claim 16, wherein the operations further comprise training the source classification model and the reason ranking model.
Pham teaches training the source classification model (Pham e.g. Disclosed are systems and methods that automate the process of analyzing interactive content data using artificial intelligence and natural language processing technology to generate subject matter identifiers and sentiment identifiers that characterize the interaction represented by the content data (Abstract). The content driver software service can be implemented with a neural network that executes the subject classification analysis. The neural network performs operations that implement a Kmeans clustering analysis to execute the subject classification analysis [0014]. There are various types of neural network architectures that can be used to implement a clustering analysis, and in particular, implement a Kmeans clustering analysis [0015]. The subject classification analysis can be implemented using supervised learning techniques that require training the neural networks with labeled, known training data. The neural network can be a recurrent neural network having a long short-term memory neural network architecture [0016]. With regard to neural network training, the system can implement supervised learning by performing a labeling analysis on a training set of content data files to generate annotated content data files [0017]. That is, the content data files are labeled to ascertain known subjects, interaction drivers, sentiments, or sentiment polarity, or other information [0017]. The training subject, interaction driver, sentiment, or polarity classification identifiers are compared against the annotated training set content data files to generate an error rate. The weights of the neural network node formulas (i.e., network parameters) of the neural network are adjusted so as to reduce the error rate [0017]. In this manner, the neural network is trained to optimize the parameters that implement the subject classification, sentiment, or other analyses [0017]. FIG. 6 is a flow chart representing a method model development and deployment by machine learning ([0066] and [0199]). The content data is analyzed using natural language processing techniques that are implemented by artificial intelligence technology. The resulting outputs can include, without limitation: (i) the identities of conversation participants or "content sources;" (ii) a list of subjects addressed within the content data and that identify the reasons or "driver" for why a customer initiated a shared experience; (iii) weighting data showing the relative importance or engagement associated with certain subjects; and (iv) frequency data defining the proportion of shared experiences that relate to a particular subject identifier or driver for a support request [0083].)
Pham nor Pearce explicitly teach, however, Revanur teaches training the ranking model (Revanur e.g. Revanur teaches a computer system that routes contact center interactions. Interactions between contact center agents and contact center queries that are received at a contact center are monitored (Abstract). A ranking model is trained according to the categories of the contact center queries and the interaction scores of each handled query using machine learning (Abstract). A ranking model is trained according to the categories of the contact center queries, one or more selected business outcomes, and the interaction scores of each agent for each handled query using machine learning [0010]. The ranking model is tested according to various metrics in order to gauge the performance of the ranking model [0010]. Testing module 120 may test the efficacy of a ranking model in order to ensure that the ranking model properly ranks agents according to desired business outcomes [0020]. The machine learning ranking model is trained and tested at operation 330 by generating interaction scores [0044]. Model generating module 115 may train the ranking model using conventional or other machine learning techniques. In some embodiments, the ranking model is trained using a machine learning approach based on matrix factorization, restricted Boltzmann machines, and/or singular value decomposition [0044]. Once model generating module 115 trains the ranking model, testing module 120 may test the model [0044].)
The Examiner submits that before the effective filing date, it would have been obvious to one of ordinary skill in the art to combine Pham in view of Pearce’s classification analysis model with Revanur’s ranking model in order to analyze and route interactions more effectively (Revanur e.g. [0053]).
As per claim 9 (Original), Pham in view of Pearce and Revanur teach the classification and resolution system of claim 8, wherein training the source classification model and the reasons ranking model comprises evaluating an accuracy of the source classification model and the reason ranking model until the accuracy reaches a threshold value.
Pham teaches wherein training the source classification model comprises evaluation an accuracy of the source classification model until the accuracy reaches a threshold value (Pham e.g. The training subject, interaction driver, sentiment, or polarity classification identifiers are compared against the annotated training set content data files to generate an error rate. The weights of the neural network node formulas (i.e., network parameters) of the neural network are adjusted so as to reduce the error rate [0017]. In this manner, the neural network is trained to optimize the parameters that implement the subject classification, sentiment, or other analyses [0017]. In yet other embodiments, the subject classification analysis can be implemented using supervised learning techniques [0035]. Supervised learning software systems are trained using content data that is well-labeled or "tagged." Supervised learning software systems often require extensive and iterative optimization cycles to adjust the input-output mapping until they converge to an expected and well-accepted level of performance, such as an acceptable threshold error rate between a calculated probability and a desired threshold probability [0129]. The training set content data is then fed to the content driver software service neural networks to identify subjects, content sources, or sentiments and the corresponding probabilities [0169]. For example, the analysis might identify that particular text represents a question with a 35% probability. If the annotations indicate the text is, in fact, a question, an error rate can be taken to be 65% or the difference between the calculated probability and the known certainty. Then parameters to the neural network are adjusted (i.e., constants and formulas that implement the nodes and connections between node), to increase the probability from 35% to ensure the neural network produces more accurate results, thereby reducing the error rate. The process is run iteratively on different sets of training set content data to continue to increase the accuracy of the neural network [0169].)
Pham nor Pearce explicitly teach, however, Revanur teaches wherein training the ranking model comprises evaluating accuracy of the ranking model until the accuracy reaches a threshold value (Revanur e.g. A ranking model is trained according to the categories of the contact center queries and the interaction scores of each handled query using machine learning (Abstract). The ranking model is tested according to various metrics to ensure that the ranking model ranks the agents according to one or more selected business outcomes (Abstract). A ranking model is trained according to the categories of the contact center queries, one or more selected business outcomes, and the interaction scores of each agent for each handled query using machine learning [0010]. The ranking model is tested according to various metrics in order to gauge the performance of the ranking model [0010]. Testing module 120 may test the efficacy of a ranking model in order to ensure that the ranking model properly ranks agents according to desired business outcomes [0020]. FIG. 4 is a flow chart depicting a method 400 of routing queries in accordance with an example embodiment [0046]. The ranking model is updated at operation 460. Feedback may be collected upon resolution of an interaction in order to update the ranking model [0053]. In some embodiments, the ranking model is continually updated by training the model with fresh feedback data. In order to determine whether an updated ranking model should replace a currently-deployed model, the two models may be compared using AB testing; if a new model reliably performs better than a deployed model, the new model may be deployed [0053]. As the ranking model is continually updated, additional latent information may be discovered for each agent, thus enabling successive ranking models to route interactions more effectively [0053].)
The Examiner submits that before the effective filing date, it would have been obvious to one of ordinary skill in the art to combine Pham in view of Pearce’s classification analysis model with Revanur’s ranking model in order to analyze and route interactions more effectively (Revanur e.g. [0053]).
As per claim 10 (Original), Pham in view of Pearce teach the classification and resolution system of claim 1, Pham in view of Pearce and Revanur teach wherein the operations further comprise periodically verifying an accuracy of the source classification model and the reason ranking model.
Pham teaches periodically verifying an accuracy of the source classification model (Pham e.g. FIG. 6 is a flow chart representing a method model development and deployment by machine learning ([0066] and [0199]). Step 606 can include data validation to confirm that the statistics of the ingested data are as expected, such as that data values are within expected numerical ranges, that data sets are within any expected or required categories, and that data comply with any needed distributions such as within those categories [0202]. In step 610, training test data such as a target variable value is inserted into an iterative training and testing loop. For example, features in the training test data are used to train the model based on weights and iterative calculations in which the target variable may be incorrectly predicted in an early iteration as determined by comparison in step 614, where the model is tested. Subsequent iterations of the model training, in step 612, may be conducted with updated weights in the calculations [0203].).
Pham nor Pearce explicitly teach, however, Revanur teaches periodically verifying an accuracy of the ranking model (Revanur e.g. The ranking model is tested according to various metrics in order to gauge the performance of the ranking model [0010]. Testing module 120 may test the efficacy of a ranking model in order to ensure that the ranking model properly ranks agents according to desired business outcomes [0020]. FIG. 4 is a flow chart depicting a method 400 of routing queries in accordance with an example embodiment [0046]. The ranking model is updated at operation 460. Feedback may be collected upon resolution of an interaction in order to update the ranking model [0053]. In some embodiments, the ranking model is continually updated by training the model with fresh feedback data. In order to determine whether an updated ranking model should replace a currently-deployed model, the two models may be compared using AB testing; if a new model reliably performs better than a deployed model, the new model may be deployed [0053]. As the ranking model is continually updated, additional latent information may be discovered for each agent, thus enabling successive ranking models to route interactions more effectively [0053].)
The Examiner submits that before the effective filing date, it would have been obvious to one of ordinary skill in the art to combine Pham in view of Pearce’s classification analysis model with Revanur’s ranking model in order to analyze and route interactions more effectively (Revanur e.g. [0053]).
Claim 21 is rejected under 35 U.S.C. 103 as being unpatentable over Pham et al. (US 2024/0168918 A1) in view of Pearce et al. (US 2020/0329142 A1) and in further view of Conway et al. (US 10,129,402 B1).
As per claim 21 (Currently Amended), Pham in view of Pearce teach the system of claim 1, Pham or Pearce explicitly teach, however Conway teaches wherein the historical statistics of the first agent comprise, for a relevant agent skill, a percentage of incomplete communication, a percentage of lacking skills or proficiency, and a percentage of incorrect information on previous interactions (Conway e.g. A method for analyzing caller interaction events that includes receiving, by a processor, a caller interaction event between an agent and a caller, extracting, by a processor, caller event data from the caller interaction event, analyzing, by a processor, the caller event data, and generating, by a processor, a report displaying one or more selected categories of the caller event data (Abstract). Generally, a customer is in contact with a customer service representative ("CSR") or call center agent who is responsible for answering the customer's inquiries and/or directing the customer to the appropriate individual, department, information source, or service as required to satisfy the customer's needs (col. 1 lines 40-46). The caller event information can also include additional information concerning each call, such as statistical data relating to the caller interaction event (e.g., time, date and length of call, caller identification, agent identification, hold times, transfers, etc.), and a recording of the caller interaction event (col. 7 lines 11-16). This event data is comprised of a call assessment data corresponding to at least one identifying indicia (e.g., a CSR name, a CSR center identifier, a customer, a customer type, a call type, etc.) and at least one predetermined time interval (col. 23 lines 54-58). In the embodiment shown in FIG. 15, the system 1 includes a PROFILES tab 410, a REVIEW tab 412 and a METRICS tab 414. A variety of the other tabs with additional information can also be made available (col. 24 lines 56-60). The REVIEW tab 412 also includes a visual link to call center or CSR agent folders 422. This includes a list of calls divided by call center or CSR agents (col. 25 lines 36-38). The user could choose an agent from a drop down menu or list of available agents. This returns all calls from the selected agent in the date range specified (col. 25 lines 51-53). The user can also generate a number of CALL CENTER or CSR AGENT REPORTS. These include the following summary reports: corporate summary by location; CSR agent performance; and non-analyzed calls (col. 27 lines 34-37). A CORPORATE SUMMARY BY LOCATION REPORT 502 is shown in FIG. 26 (col. 27 lines 40-41). The CORPORATE SUMMARY BY LOCATION REPORT 502 includes a location column 504 (this identifies the call center location that received the call), a number of calls column 506 (total number of calls received by the associated call center location during the specified reporting interval, an average duration column 508 (total analyzed talk time for all calls analyzed for the associated CSR agent divided by the total number of calls analyzed for the agent), a greater than 150% duration column 510 (percentage of calls for a CSR agent that exceed 150% of the average duration for all calls, a greater than 90 second hold column 512 (percentage of calls for a CSR agent where the CSR places the caller on hold for greater than 90 seconds), a greater than 30 second silence column 514 (percentage of calls for a CSR agent where there is a period of continuous silence within a call greater than 30 seconds), a call transfer column 516 (percentage of calls for a CSR agent that result in the caller being transferred), an inappropriate response column 518 (percentage of calls where the CSR agent exhibits inappropriate behavior or language), an appropriate response column 520 (percentage of calls where the CSR agent exhibits appropriate behavior or language that result in the dissipation of caller distress-these calls can be found in the upset caller/issue resolved folder), a no authentication column 522 (percentage of calls where the CSR agent does not authenticate the caller's identity to prevent fraud), and a score column 524 (a composite score that represents overall call center performance for all calls in the associated call center location.) (cols. 27-28 lines 45-6). The values 526 in the score column 524 are based on the weighted criteria shown in FIG. 27. All weighted values are subtracted from a starting point of 100 except for "appropriate response," which is an additive value (col. 28 lines 7-10). A CSR PERFORMANCE REPORT 528 is shown in FIG. 28. This is a detail level report that identifies analysis results by CSR for the specified time interval. This Report 528 contains a composite score that ranks relative CSR performance for each call type across event filter criteria (col. 28 lines 11-15). FIG. 31 shows a TEAM BY AGENT REPORT 534. This is a summary level report that identifies analysis results by team and agent for the specified time interval. These Reports 534 contain a composite performance score that ranks relative CSR performance across event filter criteria by agent (col. 28 lines 26-31). The Examiner submits that % call transfers could reflect incomplete/unresolved communications, the score could reflect proficiency, and % of no authentication/inappropriate response could reflect incorrect information.)
The Examiner submits that before the effective filing date, it would have been obvious to one of ordinary skill in the art to modify Pham in view of Pearce’s provider/contact center system’s historical statistics of agents to include a percentage of incomplete communication, a percentage of lacking skills or proficiency, and a percentage of incorrect information on previous interactions as taught by Conway in order to monitor the performance of the call center agents to identify possible training needs (Conway e.g. col. 1 lines 62-64).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure include US: Konig et al. (US 2018/0159982 A1) “System and Method For Performance-Based Routing of Interactions in a Contact Center”, NPL: N. Mehrbod, A. Grilo and A. Zutshi, "Caller-Agent Pairing in Call Centers Using Machine Learning Techniques with Imbalanced Data," 2018 IEEE International Conference on Engineering, Technology and Innovation (ICE/ITMC), Stuttgart, Germany, 2018, pp. 1-6, and FOR: JP2023540970A “Systems And Methods Related To Predicting And Preventing High Agent Turnover In Contact Centers”.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Ayanna Minor whose telephone number is (571)272-3605. The examiner can normally be reached M-F 9am-5 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jerry O'Connor can be reached at 571-272-6787. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/A.M./Examiner, Art Unit 3624
/Jerry O'Connor/Supervisory Patent Examiner,Group Art Unit 3624
Prosecution Insights