DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Introduction
The following is a non-final Office Action in response to Applicant’s communications received on July 9, 2026. Claims 1-2, 6, 8-9 and 15-16 have been amended, claims 3-5, 7, 10-14 and 17-23 have been canceled, and claims 24-30 have been added.
Currently claims 1-2, 6, 8-9, 15-16 and 24-30 are pending. Claims 1, 8 and 15 are independent.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submissions filed on July 9, 2026 has been entered.
Response to Amendments
Applicant’s amendments to claims 1-2, 6, 8-9 and 15-16 are NOT sufficient to overcome the 35 U.S.C. § 101 rejection as set forth in the previous Office Action. Therefore, the 35 U.S.C. § 101 rejection to claims 1-2, 6, 8-9, 15-16 and 24-30 has been maintained.
Response to Arguments
Applicant’s arguments filed on July 9, 2026 have been fully considered but are not persuasive.
In the Remarks on page 11, Applicant’s arguments regarding the 35 U.S.C. § 101 rejection that the features of the amended independent claims provide a technological improvement to artificial intelligence routing systems and machine learning resource allocation. A claim that improves the functioning of a computer or other technology is integrated into a practical application and is patent eligible.
In response to Applicant’s arguments, the Examiner respectfully disagrees. Claim 1 recites limitations of “identifying a matching stored question, retrieving stored performance data associated with the matching stored question…, and determining which model of the plurality of models to use for the at least one question based on the stored performance data associated with the matching stored question…” can be performed in the mind or by a human using a pen and paper. Thus, the claim still recites an abstract idea.
Further, in order for a claim to integrate the exception into a practical application, the additional claimed elements must, for example, improve the functioning of a computer or any other technology or technical field (see MPEP § 2106.05(a)), apply the judicial exception with a particular machine (see MPEP § 2106.05(b)), affect a transformation or reduction of a particular article to a different state or thing (see MPEP § 2106.05(c)), or apply or use the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment (see MPEP § 2106.05(e)). See Revised 2019 Guidance. Here, the claim recites the additional element of “a computer device” and “a trained large language model” for performing the steps. These additional elements are recited at a high level of generality and merely invoked as tools to perform generic computer functions including receiving, storing, displaying and transmitting information over a network. However, none of the claim elements reflects an improvement to the functioning of the computer device or another technology or technical field. In this regard, the courts held that “Automating manual and mental processes on generic computers does not make an abstract idea patent eligible.” See Credit Acceptance Corp. v. Westlake Servs., 859 F.3d 1044, 1055 (Fed. Cir. 2017) (“[A]utomation of manual processes using generic computers does not constitute a patentable improvement in computer technology.”). Even allowing the computing device to perform the steps in a more quick and efficient way, the improvement is not on the functioning of the computing device itself, the focus of the claims is not on such an improvement in computers as tools, but on certain independently abstract ideas that use computers as tools. See FairWarning IP, LLC v. Iatric Sys., Inc., 839 F.3d 1089, 1095 (Fed. Cir. 2016). Therefore, using generic computer components to implement an abstract idea does not integrate the abstract idea into a practical application.
In the Remarks on page 13, Applicant argues that the Chu does not disclose “identifying a matching stored question that matches or partially matches the at least one question” or “retrieving stored performance data associated with the matching stored question.” However, Applicant’s arguments are directed to the newly amended claims, and therefore, the newly amended claims will be fully addressed in this Office Action.
Claim Objections
Claims 12-13 are objected to because of the following informalities.
Claim 12 recites (original) and claim 13 recites (anceled) appear to be a typography error of: Claim 12 (Canceled) and Claim 13 (Currently Amended) the system of claim 9. Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefore, subject to the conditions and requirements of this title.
Claims 1-2, 6, 8-9, 15-16 and 24-30 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
As per Step 1 of the subject matter eligibility analysis, it is to determine whether the claim is directed to one of the four statutory categories of invention, i.e., process, machine, manufacture, or composition of matter.
In this case, claims 1-2, 6 and 24-30 are directed to a method for quality management, which falls within the statutory category of a process. Claims 8-9 are directed to a system comprising a computing device and at a computer-readable medium, which falls within the statutory category of a machine. Claims 15-16 are directed to a non-transitory computer-readable medium with computer-executable instructions stored, which falls within the statutory category of a product.
In Step 2A of the subject matter eligibility analysis, it is to “determine whether the claim at issue is directed to a judicial exception (i.e., an abstract idea, a law of nature, or a natural phenomenon). Under this step, a two-prong inquiry will be performed to determine if the claim recites a judicial exception (an abstract idea enumerated in the 2019 Guidance), then determine if the claim recites additional elements that integrate the exception into a practical application of the exception. See 2019 Revised Patent Subject Matter Eligibility Guidance (2019 Guidance), 84 Fed. Reg. 50, 54-55 (January 7, 2019).
In Prong One, it is to determine if the claim recites a judicial exception (an abstract idea enumerated in the 2019 Guidance, a law of nature, or a natural phenomenon).
Taking the method as representative, the claims recite the limitations of “receiving an evaluation including at least one question, removing the metadata from the interaction data, adding text representations of the removed metadata, identifying a matching stored question, retrieving stored performance data, determining which model of the plurality of models to use for the at least one question, determining whether the untrained large language model performs better for the matching stored question than the trained large language model, generating a quality metric for the at least one question for the current interaction, associating the quality metric for the at least one question, providing the evaluation including the associated quality metric, receiving interaction data representing previous interactions with one or more agents, using a first portion of the interaction data…training the first large language model, using a second portion of interaction data…generating performance indicator for the first classifier, generating the quality metric for the question, determining that the generated performance indicators for the first large language model, generating the quality metric for the question…, generating performance indicator…, selecting one of the first large language model, the second large language model, and the first classifier, based on the generated performance indicators, selecting one of the first large language model, the second large language model, and the first classifier; generating a quality metric for the at least one question for the current interaction using the selected one of the first large language model, the second large language model, and the first classifier, transforming a component question of the plurality of question into a plurality of single answer question, dividing the text transcript into two or more transcripts, generating a quality metric for each portion of the transcript, combining similar questions, using a large language model, and applying an administrator-defined weight to each of the key performance indicators”. None of the limitations recites technological implementation details for any of these steps, but instead recite only results desired by any and all possible means. The limitations, as drafted, are directed to methods that allow users to monitor and evaluate agent performance, and managing interactions between agent and customer, which fall within the certain methods of organizing human activity grouping. See Under the 2019 Guidance, 84 Fed. Reg. 52. The mere recitation of a computing device does not take the claims out of the certain method of organizing human activity group. Accordingly, the claims recite an abstract idea, and the analysis is proceeding to Prong Two.
In Prong Two, it is to determine if the claim recites additional elements that integrate the exception into a practical application of the exception.
Beyond the abstract idea, claim 1 recites an additional element of “a computing device” for performing the steps, and “a large language model”. The specification describes these additional elements at a high level of generality and merely invoked as tools to perform the generic computer functions including receive interaction data over a network. For example, “The components of the AQM engine 180 may be implemented together or separately using one or more general purpose computing devices such as the computer 500 shown in Fig. 5.” (See ¶ 123). Thus, merely adding a generic computer, generic computer components, or programmed computer to perform generic computer functions does not automatically overcome an eligibility rejection. Alice Corp. Pty. Ltd. V. CLS Bank Int’l, 134 S. Ct. 2347, 2358-59, 110 USPQ2d 1976, 1983-84 (2014); see also Bancorp Servs., L.L.C. v. Sun Life Assurance Co. of Canada (U.S.), 687 F.3d 1266, 1278 (Fed. Cir. 2012) (A computer “employed only for its most basic function… does not impose meaningful limits on the scope of those claims”). As to the learning/training per se, such an argument overlooks the entire education system. Reciting machine learning is placing such learning in a computer context, offering no technological implementation details beyond the conceptual idea to use a machine learning. However, simply implementing the abstract idea on a generic computer does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. Further, nothing in the claims that reflects an improvement to the functioning of a computer itself or another technology, effects a transformation or reduction of a particular article to a different state or thing, or applies or uses the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effect designed to monopolize the exception. Therefore, the additional elements do not integrate the judicial exception into a practical application. The claims are directed to an abstract idea, the analysis is proceeding to Step 2B.
In Step 2B of Alice, it is "a search for an ‘inventive concept’—i.e., an element or combination of elements that is ‘sufficient to ensure that the patent in practice amounts to significantly more than a patent upon the [ineligible concept’ itself.’” Id. (alternation in original) (quoting Mayo Collaborative Servs. v. Prometheus Labs., Inc., 132 S. Ct. 1289, 1294 (2012)).
The claims as described in Prong Two above, nothing in the claims that integrates the abstract idea into a practical application. The same analysis applies here in Step 2B.
Beyond the abstract idea, claim 1 recites an additional element of “a computing device” for performing the steps, and “a large language model”. The specification describes these additional elements at a high level of generality and merely invoked as tools to perform the generic computer functions including receive interaction data over a network. For example, “The components of the AQM engine 180 may be implemented together or separately using one or more general purpose computing devices such as the computer 500 shown in Fig. 5.” (See ¶ 123). Taking the claim elements separately and as an ordered combination, the computing device, at best, may perform the generic computer functions including receiving, manipulating, and transmitting information over a network. However, generic computer for performing generic computer functions have been recognized by the courts as merely well-understood, routine, and conventional functions of generic computers. See MPEP 2106.05 (d) (II) (Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)). Thus, simply implementing the abstract idea on a generic computer for performing generic computer functions do not amount to significantly more than the abstract idea. (MPEP 2106.05(a)-(c), (e-f) & (h)).
For the foregoing reasons, claims 1-2 and 24-30 cover subject matter that is judicially-excepted from patent eligibility under § 101 as discussed above, the other claims 8-9 and 15-16 parallel claims 1-2 and 24-30—similarly cover claimed subject matter that is judicially excepted from patent eligibility under § 101.
Therefore, the claims as a whole, viewed individually and as a combination, do not provide meaningful limitations to transform the abstract idea into a patent eligible application of the abstract idea such that the claims amount to significantly more than the abstract idea itself. The claims are not patent eligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 6, 8-9, 15-16 and 24-30 are rejected under 35 U.S.C. 103 as being unpatentable over Tapuhi et al., (US 2018/0096617, hereinafter: Tapuhi), and in view of Chu et al., (WO 2024/177735, hereinafter: Chu), and further in view of Davies et al., (US 2018/0196845, hereinafter: Davies), and Moreno et al., (US 2017/0228372 A1, hereinafter: Moreno).
Regarding claim 1, Tapuhi discloses a method for automating quality metrics comprising:
receiving an evaluation including at least one question of a plurality of questions and interaction data representing a current interaction with an agent by a receiving component of a computing device (see Abstract; ¶ 5, ¶ 9, ¶ 15, ¶ 86, ¶ 122), wherein the interaction data comprises a text transcript of the interaction and metadata (see ¶ 56, ¶ 63, ¶ 161-163);
identifying a matching stored question that matches or partially matches the at least one question (see ¶ 53, ¶ 63, ¶ 68);
retrieving stored performance data associated with the matching stored question, wherein the stored performance data indicates performance metrics of a plurality of models for the matching stored question (see ¶ 59, ¶ 75, ¶ 89, ¶ 94-95), the plurality of models including at least a trained large language model, an untrained large language model, and a trained classifier;
determining which model of the plurality of models to use for the at least one question based on the stored performance data associated with the matching stored question (see ¶ 71, ¶ 95, 110, ¶ 122), wherein determining which model to use comprises determining whether the untrained large language model performs better for the matching stored question than the trained large language model;
generating a quality metric for the at least one question for the current interaction using the determined model of the plurality of models and the interaction data representing the current interaction by a generating component of the computing device (see ¶ 75-76, ¶ 98, ¶ 105, ¶ 113, ¶ 122, ¶ 169);
associating the quality metric for the at least one question with the current interaction by an associating component of the computing device (see ¶ 12, ¶ 16, ¶ 89, ¶ 169); and
providing the evaluation including the associated quality metric for the at least one question to the agent by the computing device (see ¶ 75-76, ¶ 93, ¶ 121-122).
Tapuhi discloses training a neural network to compute an overall evaluation score for an interaction based on automatic answered questions (see ¶ 28).
Tapuhi does not explicitly disclose using a first large language model or a first classifier; however, Chu in an analogous art for ranking recommender discloses
identifying a matching stored question that matches or partially matches the at least one question (see ¶ 15, ¶ 100);
the plurality of models including at least a trained large language model, an untrained large language model, and a trained classifier (see ¶ 9-13, ¶ 33);
wherein determining which model to use comprises determining whether the untrained large language model performs better for the matching stored question than the trained large language model (see ¶ 40, ¶ 104-105, ¶ 134).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Tapuhi to include the teaching of Chu in order to gain the commonly understood benefit of such adaption, such as providing the benefit of a more optimal solution with a specified model, in turn of operational efficiency. Since the combination of each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Tapuhi discloses “storing one or more databases relating to agent data, customer data and interaction data (e.g., details of each interaction with a customer, reason for the interaction, disposition data, time on hold, handle time, etc.) (see ¶ 56); “detecting any of the phrases within an interaction (e.g., speech to text transcript of a voice interaction or chat session) containing the phrases as relating to the associated topic. Topics can be grouped into meta topics and meta topics may be grouped with other meta topics and/or topics from semantic hierarchy or taxonomy of meta topics and topics” (see ¶ 63); and “the quality monitoring system also stores the identified portions include the locations of the identified portions within the interactions (e.g., start and end timestamps)” (see ¶ 70).
Tapuhi and Chu do not explicitly disclose using a first large language model or a first classifier; however, Davies in an analogous art for providing interaction data discloses
removing the metadata from the interaction data by an adding component of the computing device (see Fig. 8, # 820-825; ¶ 25, ¶ 28, ¶ 30, ¶ 51), wherein the removed metadata comprises indicators of times during the current interaction when the agent was put on hold and one or more sentiments detected during the current interaction (see ¶ 39-40, ¶ 44-45, ¶ 60);
adding text representations of the removed metadata, including time stamps corresponding to the indicators of the times during the interaction when the agent was put on hold and the one or more sentiments, into the text transcript by the adding component of the computing device (see ¶ 25-26, ¶ 30, ¶ 42-44, ¶ 48, ¶ 58-60).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Tapuhi and in view of Chu to include the teaching of Davies in order to gain the commonly understood benefit of such adaption, such as providing the benefit of time-specific data locations, and enabling better decision making. Since the combination of each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Regarding claim 2, Tapuhi discloses the method of claim 1, further comprising:
receiving interaction data representing previous interactions with one or more agents by the computing device, wherein each interaction is associated with a question of the plurality of questions and a quality metric (see ¶ 15, ¶ 110, ¶ 145, ¶ 162);
using a first portion of the interaction data representing the previous interactions (see Fig. 5, # 506; ¶ 65-68, ¶ 87, ¶ 149).
Tapuhi discloses training a neural network to compute an overall evaluation score for the interaction (see ¶ 28).
Tapuhi does not explicitly disclose the following limitations; however, Chu discloses
Training the trained large language model or the trained classifier by the computing device classifier (see Abstract; Fig. 15, # 1502-1506; ¶ 9, ¶ 13, ¶ 79, ¶ 136).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Tapuhi to include the teaching of Chu in order to gain the commonly understood benefit of such adaption, such as providing the benefit of a more optimal solution with a specified model, in turn of operational efficiency. Since the combination of each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Regarding claim 6, Tapuhi does not explicitly disclose following limitations; however, Chu discloses the method of claim 2, wherein the trained classifier comprises a neural network classifier, or XGBoost (see ¶ 78-80, ¶ 124). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Tapuhi to include the teaching of Chu in order to gain the commonly understood benefit of such adaption, such as providing the benefit of a more optimal solution with a specified model, in turn of operational efficiency. Since the combination of each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Regarding claim 8, Tapuhi discloses a system for automating quality metrics comprising:
a computing device (see ¶ 171); and
a computer-readable medium with computer-executable instructions stored thereon that when executed by the computing device cause the computing device (see ¶ 19, ¶ 172) to:
receive an evaluation including at least one question of a plurality of questions and interaction data representing a current interaction with an agent (see Abstract; ¶ 5, ¶ 15, ¶ 86, ¶ 122), wherein the interaction data comprises a text transcript of the interaction and metadata (see ¶ 56, ¶ 63, ¶ 161-163);
identify a matching stored question that matches or partially matches the at least one question (see ¶ 53, ¶ 63, ¶ 68);
retrieve stored performance data associated with the matching stored question, wherein the stored performance data indicates performance metrics of a plurality of models for the matching stored question (see ¶ 59, ¶ 75, ¶ 89, ¶ 94-95), the plurality of models including at least a trained large language model, an untrained large language model, and a trained classifier;
determine which model of the plurality of models to use for the at least one question based on the stored performance data associated with the matching stored question (see ¶ 71, ¶ 95, 110, ¶ 122), wherein determining which model to use comprises determining whether the untrained large language model performs better for the matching stored question than the trained large language model;
generate a quality metric for the at least one question for the current interaction using the determined model of the plurality of models and the interaction data representing the current interaction (see ¶ 75-76, ¶ 98, ¶ 105, ¶ 113, ¶ 119-122, ¶ 163); and
associate the quality metric for the at least one question with the current interaction (see ¶ 12, ¶ 16, ¶ 89, ¶ 169); and
provide the evaluation including the associated quality metric for the at least one question to the agent by the computing device (see ¶ 75-76, ¶ 93, ¶ 121-122).
Tapuhi discloses training a neural network to compute an overall evaluation score for an interaction based on automatic answered questions (see ¶ 28).
Tapuhi does not explicitly disclose using a first large language model or a first classifier; however, Chu in an analogous art for ranking recommender discloses
identify a matching stored question that matches or partially matches the at least one question (see ¶ 15, ¶ 100);
the plurality of models including at least a trained large language model, an untrained large language model, and a trained classifier (see ¶ 9-13, ¶ 33);
wherein determining which model to use comprises determining whether the untrained large language model performs better for the matching stored question than the trained large language model (see ¶ 40, ¶ 104-105, ¶ 134).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Tapuhi to include the teaching of Chu in order to gain the commonly understood benefit of such adaption, such as providing the benefit of a more optimal solution with a specified model, in turn of operational efficiency. Since the combination of each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Tapuhi discloses “storing one or more databases relating to agent data, customer data and interaction data (e.g., details of each interaction with a customer, reason for the interaction, disposition data, time on hold, handle time, etc.) (see ¶ 56); “detecting any of the phrases within an interaction (e.g., speech to text transcript of a voice interaction or chat session) containing the phrases as relating to the associated topic. Topics can be grouped into meta topics and meta topics may be grouped with other meta topics and/or topics from semantic hierarchy or taxonomy of meta topics and topics” (see ¶ 63); and “the quality monitoring system also stores the identified portions include the locations of the identified portions within the interactions (e.g., start and end timestamps)” (see ¶ 70).
Tapuhi and Chu do not explicitly disclose using a first large language model or a first classifier; however, Davies in an analogous art for providing interaction data discloses
remove the metadata from the interaction data by an adding component of the computing device (see Fig. 8, # 820-825; ¶ 25, ¶ 28, ¶ 30, ¶ 51), wherein the removed metadata comprises indicators of times during the current interaction when the agent was put on hold and one or more sentiments detected during the current interaction (see ¶ 39-40, ¶ 44-45, ¶ 60);
add text representations of the removed metadata, including time stamps corresponding to the indicators of the times during the interaction when the agent was put on hold and the one or more sentiments, into the text transcript by the adding component of the computing device (see ¶ 25-26, ¶ 30, ¶ 42-44, ¶ 48, ¶ 58-60).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Tapuhi and in view of Chu to include the teaching of Davies in order to gain the commonly understood benefit of such adaption, such as providing the benefit of time-specific data locations, and enabling better decision making. Since the combination of each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Regarding claim 9, Tapuhi discloses the system of claim 8, further comprising computer-executable instructions that when executed by the computing device cause the computing device to:
receive interaction data representing previous interactions with one or more agents by the computing device, wherein each interaction is associated with a question of the plurality of questions and a quality metric (see ¶ 15, ¶ 110, ¶ 145, ¶ 162);
use a first portion of the interaction data representing the previous interactions (see Fig. 5, # 506; ¶ 65-68, ¶ 87, ¶ 149).
Tapuhi discloses training a neural network to compute an overall evaluation score for the interaction (see ¶ 28).
Tapuhi does not explicitly disclose the following limitations; however, Chu discloses
training the trained large language model or the trained classifier (see Abstract; Fig. 15, # 1502-1506; ¶ 9, ¶ 13, ¶ 79, ¶ 136). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Tapuhi to include the teaching of Chu in order to gain the commonly understood benefit of such adaption, such as providing the benefit of a more optimal solution with a specified model, in turn of operational efficiency. Since the combination of each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Regarding claim 13, Tapuhi does not explicitly disclose following limitations; however, Chu discloses the system of claim 9, wherein the first classifier comprises a neural network classifier, or XGBoost (see ¶ 78-80, ¶ 124). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Tapuhi to include the teaching of Chu in order to gain the commonly understood benefit of such adaption, such as providing the benefit of a more optimal solution with a specified model, in turn of operational efficiency. Since the combination of each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Regarding claim 15, Tapuhi discloses a non-transitory computer-readable medium with computer-executable instructions stored thereon that when executed by a computing device cause the computing device (see ¶ 19, ¶ 170) to:
receive an evaluation including at least one question of a plurality of questions and interaction data representing a current interaction with an agent (see Abstract; ¶ 5, ¶ 15, ¶ 86, ¶ 122), wherein the interaction data comprises a text transcript of the interaction and metadata (see ¶ 56, ¶ 63, ¶ 161-163);
wherein the metadata identifies one or more events that occurred during the current interaction (see ¶ 44, ¶ 63, ¶ 72);
identify a matching stored question that matches or partially matches the at least one question (see ¶ 53, ¶ 63, ¶ 68);
retrieve stored performance data associated with the matching stored question, wherein the stored performance data indicates performance metrics of a plurality of models for the matching stored question (see ¶ 59, ¶ 75, ¶ 89, ¶ 94-95), the plurality of models including at least a trained large language model, an untrained large language model, and a trained classifier;
determine which model of the plurality of models to use for the at least one question based on the stored performance data associated with the matching stored question (see ¶ 71, ¶ 95, 110, ¶ 122), wherein determining which model to use comprises determining whether the untrained large language model performs better for the matching stored question than the trained large language model;
generate a quality metric for the at least one question for the current interaction using the determined model of the plurality of models and the interaction data representing the current interaction (see ¶ 75-76, ¶ 98, ¶ 105, ¶ 113, ¶ 119-122, ¶ 163); and
associate the quality metric for the at least one question with the current interaction (see ¶ 12, ¶ 16, ¶ 89, ¶ 169); and
provide the evaluation including the associated quality metric for the at least one question to the agent by the computing device (see ¶ 75-76, ¶ 93, ¶ 121-122).
Tapuhi discloses training a neural network to compute an overall evaluation score for an interaction based on automatic answered questions (see ¶ 28).
Tapuhi does not explicitly disclose using a first large language model or a first classifier; however, Chu in an analogous art for ranking recommender discloses
identify a matching stored question that matches or partially matches the at least one question (see ¶ 15, ¶ 100);
the plurality of models including at least a trained large language model, an untrained large language model, and a trained classifier (see ¶ 9-13, ¶ 33);
wherein determining which model to use comprises determining whether the untrained large language model performs better for the matching stored question than the trained large language model (see ¶ 40, ¶ 104-105, ¶ 134).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Tapuhi to include the teaching of Chu in order to gain the commonly understood benefit of such adaption, such as providing the benefit of a more optimal solution with a specified model, in turn of operational efficiency. Since the combination of each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Tapuhi discloses “storing one or more databases relating to agent data, customer data and interaction data (e.g., details of each interaction with a customer, reason for the interaction, disposition data, time on hold, handle time, etc.) (see ¶ 56); “detecting any of the phrases within an interaction (e.g., speech to text transcript of a voice interaction or chat session) containing the phrases as relating to the associated topic. Topics can be grouped into meta topics and meta topics may be grouped with other meta topics and/or topics from semantic hierarchy or taxonomy of meta topics and topics” (see ¶ 63); and “the quality monitoring system also stores the identified portions include the locations of the identified portions within the interactions (e.g., start and end timestamps)” (see ¶ 70).
Tapuhi and Chu do not explicitly disclose using a first large language model or a first classifier; however, Davies in an analogous art for providing interaction data discloses
remove the metadata from the interaction data by an adding component of the computing device (see Fig. 8, # 820-825; ¶ 25, ¶ 28, ¶ 30, ¶ 51), wherein the removed metadata comprises indicators of times during the current interaction when the agent was put on hold and one or more sentiments detected during the current interaction (see ¶ 39-40, ¶ 44-45, ¶ 60);
add text representations of the removed metadata, including time stamps corresponding to the indicators of the times during the interaction when the agent was put on hold and the one or more sentiments, into the text transcript by the adding component of the computing device (see ¶ 25-26, ¶ 30, ¶ 42-44, ¶ 48, ¶ 58-60).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Tapuhi and in view of Chu to include the teaching of Davies in order to gain the commonly understood benefit of such adaption, such as providing the benefit of time-specific data locations, and enabling better decision making. Since the combination of each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Regarding claim 16, Tapuhi discloses the computer-readable medium of claim 15, further comprising computer-executable instructions that when executed by the computing device cause the computing device to:
receive interaction data representing previous interactions with one or more agents by the computing device, wherein each interaction is associated with a question of the plurality of questions and a quality metric (see ¶ 15, ¶ 110, ¶ 145, ¶ 162);
use a first portion of the interaction data representing the previous interactions, training the trained large language model or the trained classifier (see Fig. 5, # 506; ¶ 65-68, ¶ 87, ¶ 149).
Tapuhi discloses training a neural network to compute an overall evaluation score for the interaction (see ¶ 28).
Tapuhi does not explicitly disclose the following limitations; however, Chu discloses
training the trained large language model or the trained classifier by the computing device classifier (see Abstract; Fig. 15, # 1502-1506; ¶ 9, ¶ 13, ¶ 79, ¶ 136). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Tapuhi to include the teaching of Chu in order to gain the commonly understood benefit of such adaption, such as providing the benefit of a more optimal solution with a specified model, in turn of operational efficiency. Since the combination of each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Regarding claim 24, Tapuhi discloses the method of claim 1, wherein the removed metadata further comprises an indication of a silent period during the current interaction, and wherein the mapping of the metadata to text generates a text representation comprising a text tag indicating the silent period and a duration of the silent period (see ¶ 95, ¶ 100-101, ¶ 111, ¶ 174).
Regarding claim 25, Tapuhi discloses the method of claim 1, further comprising transforming a compound question of the plurality of questions into a plurality of single answer questions prior to generating the quality metric (see ¶ 12, ¶ 65, ¶ 88, ¶ 184).
Regarding claim 26, Tapuhi discloses the method of claim 1, further comprising dividing the text transcript into two or more transcripts, and generating a quality metric for each portion of the transcript (see ¶ 61-63, ¶ 95).
Regarding claim 27, Tapuhi and Chu do not explicitly disclose the following limitations; however, Davies discloses the method of claim 1, further comprising combining similar questions that use different language or phrasing into a single question prior to identifying the matching stored question (see ¶ 61, ¶ 100, ¶ 107). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Tapuhi and in view of Chu to include the teaching of Davies in order to gain the commonly understood benefit of such adaption, such as providing the benefit of time-specific data locations, and enabling better decision making. Since the combination of each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Regarding claim 28, Tapuhi does not explicitly disclose the following limitations; however, Chu discloses the method of claim 2, wherein training the trained classifier comprises using a large language model (LLM) embedding generated by the untrained large language model (see ¶ 9-10, ¶ 13, ¶ 78-79). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Tapuhi to include the teaching of Chu in order to gain the commonly understood benefit of such adaption, such as providing the benefit of a more optimal solution with a specified model, in turn of operational efficiency. Since the combination of each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Regarding claim 29, Tapuhi discloses the method of claim 2, wherein the historical interaction data further comprises comments from an evaluator explaining the assigned quality metrics and specific phrases from the previous interactions (see ¶ 99, ¶ 110, ¶ 123, ¶ 129).
Regarding claim 30, Tapuhi discloses the method of claim 1, wherein the stored performance data comprises key performance indicators including accuracy, precision, and recall, and wherein determining which model of the plurality of models to use comprises applying an administrator-defined weight to each of the key performance indicators (see ¶ 75, ¶ 91, ¶ 94, ¶ 138).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Hoeg (US 2014/0249813) discloses a method for interfaces allowing limited edits to data corresponding to transcripts based on in response to commands received via a user interface module.
Hasan et al., (US 2023/0342557) discloses a system for training a virtual agent comprising storing conversation between the virtual agent and a user in logs.
Lev-tov et al., (WO 2018/064199) discloses a method for automatic quality management includes receiving agent interactions with customer, evaluating a plurality of answers corresponding to the questions, and identifying one or more underperforming quality metrics in accordance with the comparisons of the aggreged quality metrics with the threshold values.
Licato et al., (US 2024/0403710) discloses a method that includes prompting a first trained large language model to generate a plurality of arguments, determine a ranking of the plurality of arguments using a second trained large language model.
Fan et al., (CN 111382573) discloses a method for quality evaluation comprises extracting the answer of the answer and answer characteristic of aiming at the problem based on the quality metric and a correlation metric to determine the answer to the question of quality score.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PAN CHOY whose telephone number is (571)270-7038. The examiner can normally be reached 5/4/9 compressed work schedule.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jerry O'Connor can be reached on 571-272-6787. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PAN G CHOY/Primary Examiner, Art Unit 3624
Prosecution Insights