Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
1. This in response to an amendment filed 04/28/2026. No claims have been added. Claim 17-20 have been amended. No claims have been canceled. Claims 1-20 are still pending in this application.
2. This office action is made non-final to address claims 17-70.
Claim Rejections - 35 USC § 103
3. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cattaneo et al. (Pub.No.: 2024/0211960 A1) in view of Surdick (US PAT # 9,742,914 B2).
Regarding claims 1 and 8, Cattaneo teaches a method and system of evaluating data items using a machine learning model (see abstract and [0016]), the method comprising, using one or more computer processors:
producing, by a machine learning (ML) model, one or more answers to one or more questions, wherein the one or more questions are applied to an input data item (reads on automatically scoring the quality of an agent-customer interaction. In one embodiment, an interaction quality score may be determined using one or more natural language processing programs and/or machine learning models to analyze a piece of content see [0015]), and wherein the one or more questions and the input data item are input to the machine learning model (see [0016] and [0020]); and
transmitting one or more output data items to a remote computer over a communication network based on the calculated score (reads on outputting evaluation results, including scores for further processing or display, see [0033] and [0035]).
Cattaneo features already addressed in the rejection of claim 1 and 8, however Cattaneo does not specifically teach “calculating a score for the input data item based on one or more of the produced answers”. In other words, Cattaneo does not specifically teach calculating the quality metric from answers to evaluation questions.
Yet, Surdick teaches an evaluation form containing questions and associated answers, wherein the selected answers are assigned point values that are accumulated into an evaluation score (see col. 5, line 62 through col. 6, line 8). For example, the evaluation configuration tool 322 includes an answer column 332. Surdick further teaches the questions 330 in the evaluation form 221, the answer column 332 includes a number of answer rows 339, each answer row 339 being associated with one of a number of discrete answer values 340 for the question (e.g., a yes/no or a pass/fail answer). Each discrete answer value 340 is associated with a number of points. For example, in FIG. 5, if the evaluation agent 116 provides an answer value of ‘Yes’ for the first evaluation question 331, five points are added to the customer service agent's 108 evaluation score. Alternatively, if the evaluation agent 116 provides an answer value of ‘No’ for the first evaluation question 331, then zero points are added to the customer service agent's 108 evaluation (see col. 5, line 62 through col. 6, line 8).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate answer-based scoring, as taught by Surdick, into the Cattaneo’s automated ML evaluation framework to provide a standardized mechanism for computing quality metrics from generated responses. This would also improve consistency of automated evaluations, reduce evaluator subjectivity, and provide a standardized scoring methodology while retaining Cattaneo’ s automated ML-based evaluations.
Independent claim 15 is rejected for the same reasons addressed in independent claims 1 and however, claim 15 substitutes a large language model (LLM) for the ML model of claim 1. Since Cattaneo already teaches language-model-based processing of interaction text, then using an LLM is an obvious substitution.
Regarding claims 2, 9 and 16, the combination of Cattaneo and Surdick teaches wherein the input data item comprises a transcript of a call, and wherein the method comprises routing, by an automatic call dialer (ACD) the call to an agent computing device (routing calls using an ACD is already discussed by Cattaneo [0001] and a transcript of a call discussed in [0019] of Cattaneo as well).
Claims 3 and 10 recite “wherein the questions are sorted in a plurality of levels, and wherein the calculating of a score is performed based on one or more of the answers corresponding to one or more of the levels”. Note that Cattaneo teaches evaluation interactions across multiple evaluation dimension as discussed in [0003], [0015-0016] and [0021] and Surdick teaches grouped evaluation questions in an evaluation form (see col. 7, lines 3-9). Thus, organizing questions into levels is an obvious design choice.
Regarding claims 4 and 11, the combination of Cattaneo and Surdick teaches wherein one or more of the questions are included in a form, the form generated using a graphical user interface (GUI), wherein the form is stored in a structured query language (SQL) database (Surdick teaches he evaluation form 221 includes a number of evaluation questions that are used to evaluate the performance of the agents who work at the customer service call center 104 (see col. 4, line 61 through col. 5, line 1) and the selected queries and/or target media sets 226 are output from the UI module 218 and stored (e.g., in a data storage device) for later use in the evaluation mode of the agent evaluation module 114 (see col. 5, lines 10-14). Also, using SQL as the underlying implementation would have been a routine database design choice.
Regarding claims 5 and 12, the combination of Cattaneo and Surdick teaches wherein the producing of one or more of the answers is performed based on an automatic evaluation plan, wherein the evaluation plan comprises the form and one or more filtering conditions, and wherein the producing of one or more answers is triggered based on a predefined time interval (Cattaneo teaches automatic evaluation of interactions [0032]). Note that Cattaneo repeatedly discusses evaluating interactions across periods of time and aggregating over periods, thus triggering evaluations according to scheduled time periods is a routine implementation for automated evaluation systems.
Regarding claims 6 and 13, the combination of Cattaneo and Surdick teaches producing, by the ML model, a justification to one or more of the answers (Cattaneo teaches explaining or providing reasoning for evaluation results, see [0031]).
Regarding claims 7 and 14, the combination of Cattaneo and Surdick teaches wherein the plurality of levels comprise a level of critical questions, wherein the calculating of a score comprises: if one or more critical questions are unanswered, assigning a score of zero to the data item. For example, Surdick teaches evaluation questions having answer values associated with points, including adding five points for a “YES” answer and zero points for a ‘’No” answer (see Surdick col. 5, line 62 through col. 6, line 8. Surdick further teaches that some evaluation questions may be automatically answered while others may be left for the evaluator, see Surdick col. 5, lines 20-35 and col. 11 lines 35-55. Cattaneo teaches threshold-based and weighted scoring for determining interaction quality scores. See Cattaneo [0036-0039] and [0060-0069]. Thus, it would have been obvious to assign a zero score when a critical evaluation question is unanswered or not satisfied.)
Regarding claim 17, the combination of Cattaneo and Surdick teaches wherein the questions are classified into a plurality of tiers (Surdick teaches evaluation questions organized into forms, queries, target media sets, answer values and associated evaluation criteria, allowing questions to be grouped and processed according to configured evaluation structures, see Surdick col. 4, lines 40-67, col. 5, lines 1-34 and col. 7, lines 1-40. Cattaneo further teaches computing different quality dimensions (conversation score, service score, interaction quality score) from multiple evaluation dimensions, see [0015-0017], [0020-0026] and [0034-0040]), and wherein the computing of a quality metric is performed based on one or more of the replies corresponding to one or more of the tiers (reads on interaction quality score form multiple scored dimensions derived from interaction content/replies, including conversation score and service score, see [0034-0040], [0051-0054] and [0060-0069]).
Regarding claim 18, the combination of Cattaneo and Surdick teaches wherein one or more of the questions are included in a form (reads on evaluation form including multiple evaluation questions, see Surdick col. 4, lines 40-67 and col. 5, lines 1-13), the form generated using a user interface (Surdick teaches a UI module that receives and presents the evaluation form to an administrative user and provides configuration interfaces, see col. 4, lines 40-67, col. 5, lines 35-67 and col. 7, lines 1-40), wherein the form is stored in a structured query language (SQL) database (Surdick stores evaluation forms, selected queries, target media sets, and call records in storage/database for later use, see col. 4, lines 15-35 and col. 5, lines 14-20).
Regarding claim 19, the combination of Cattaneo and Surdick teaches wherein the generating of one or more of the replies is performed based on an evaluation plan (Surdick teaches an evaluation configuration including an evaluation form, selected queries, target media sets, and evaluation criteria that control automatic answer generation, see col. 5, lines 1-34, col. 6, lines 10-45 and col. 7, lines 1-40), wherein the plan comprises the form and one or more filtering criteria (Surdick teaches evaluation from together with target media sets, metadata filters, query constraints, and selection criteria, see col. 8, lines 1-55 and col. 9, lines 1-43), and wherein the generating of one or more replies is initiated based on a predefined time period (Surdick teaches selecting call records based on time-related metadata like call duration thresholds and temporal constraints for automated evaluation, see col. 6, lines 45-67 and col. 7, lines 1-28).
Regarding claim 20, the combination of Cattaneo and Surdick teaches comprising generating, by the LLM, an explanation to one or more of the replies (Cattaneo teaches machine-learning/NLP models to analyze transcript content and identify utterances. Cattaneo further teaches reporting the interaction quality score with information relating to how the score was generated and recommendations for improving performance, see [0015-0017], [0022-0034], [0039-0040] and [0053]. Surdick teaches automatically suggested answers are presented with notes indicating why the answer was suggested, including which query or target media set was satisfied, and a verification hyperlink to the portion of the call record supporting the suggested answer, see col. 10, lines 25-65. Thus, it would have been obvious to implement the explanation/verification information taught by Surdick using LLM/NLP processing of Cattaneo so that the system can explain why a reply/answer was generated, thereby improving traceability, auditability, and review of automated evaluations.).
Response to Arguments
4. Applicant's arguments filed 04/28/2026 have been fully considered but they are not persuasive.
Applicant argues that the cites references fail to teach or suggest producing answers/replies to evaluation questions using a machine learning model or LLM, and further argues that Surdick merely relies on predetermined answers. However, as set forth above, Surdick teaches an evaluation form including evaluation questions and teaches that a call analyzer analyzes an evaluation call record according to selected queries and/or target media sets and generates call analysis results presented as putative answers to the evaluation questions, see Surdick col. 5, lines 16-35. Surdick further teaches answer values associated with points used to calculate an evaluation score, see Surdick col. 5, line 62 through col. 6, line 8. Cattaneo teaches using NLP, large language models, and/or other machine learning techniques to analyze interaction content/transcripts and generate evaluation scores and outputs, see [0015], [0020-0022], [0034-0040] and [0060-0069]). Therefore, the combined teachings of Cattaneo and Surdick teach or render obvious the disputed limitations.
With respect to Applicant’s statement that claims 17-20 were not addressed in the prior Office Action, claims 17-20 are now addressed in the present Office Action.
Conclusion
5. Any inquiry concerning this communication or earlier communications from the examiner should be directed to Rasha S. AL-Aubaidi whose telephone number is (571) 272-7481. The examiner can normally be reached on Monday-Friday from 8:30 am to 5:30 pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Ahmad Matar, can be reached on (571) 272-7488.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
/RASHA S AL AUBAIDI/Primary Examiner, Art Unit 2693
Prosecution Insights