Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 4/16/2026 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement has been considered by the examiner.
Interview Summary
During the interview conducted on May 19, 2026, no agreement was reached regarding any particular issue.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claim Rejections - 35 USC § 103
Claims 1 – 20 are rejected under 35 U.S.C. 103 as being unpatentable over US 20210350384 A1 (Ellison et al, hereinafter “Ellison”) in view of US 20260079919 A1 (Curtis et al, hereinafter “Curtis”).
Ellison teaches [¶ 0018] a system for providing assistance to customer service agents using a service device such as Agent desktop 106, Fig. 1 or 512 in Fig. 5. A voice call between a customer (using client device, see caller 102, Fig, 1 or user device 508, Fig. 5) and an agent (using service device such as 512 in Fig. 5) may be automatically transcribed in real-time with speech to text technology. The voice call and the transcripts may be processed by machine learning ML models (natural language processor) to identify call features such as conversation topics and facts (reads as the claimed “trigger” phrase), important conversation portions such as goal relevant, related to customer satisfaction, related to resolving an issue (may read as the claimed “performance event” of an application) on the first call from a customer, related to the user's equipment, or related to quality of service (this also reads on the claimed “performance event” of the application).
Ellison clearly and positively teaches the use of a first model and a second model sequentially such as demonstrated, for example, in Fig. 11:
First model receives interaction data and outputs a detected characteristic (trigger)
Responsive to receiving that characteristic, it is provided as input to a second model
Second model generates actionable output (instructions/relevant documents)
In Ellison, call features may be correlated with historical conversations and/or conversation portions that have a high or low likelihood of achieving customer service goals such as customer satisfaction, resolution of an issue during a single call, goals related to particular conversation topics (e.g., billing, user equipment, quality of service). In real-time during the call, suggested responses to a customer in the context of the conversation may be presented to the agent engaged in the call. For example, the agent may be presented with 3-4 responses that are historically most likely (“historical log data”) to result in the agent achieving their goals, as well as other related information and/or suggestions for directing the conversation such as relevant offers or anti-churn dialogs. Call features identified by ML models may also be used to draft a call summary (sometimes called a “call memorandum”) including identified call facts and verbatim transcripts of conversation portions identified as important.
Regarding claim 1, Ellison discloses a system (see Figs. 1, 2, 5, 6, 9 and 11) comprising:
a computing system comprising one or more processors, coupled with memory to:
Ellison discloses computing devices 520 (FIG. 5) comprising processors 604 coupled to memory 606 (FIG. 6; ¶ [0070]: “The processors 604 and the memory 606 of the computing devices 520 may implement an operating system 610 and the expert assist engine 518.”). See also ¶ [0067]: “The computing devices 520 may include general purpose computers, servers, or other electronic devices that are capable of receiving inputs, process the inputs, and generate output data.”
parse an electronic transcript generated via a natural language processor from audio samples of a communication session established between a client device and a service device to identify at least a portion of the electronic transcript; see ¶ s 18 and 99, (for example, ¶ 18 states “automatically transcribed in real-time with speech to text technology. The voice call and the transcripts may be processed by machine learning ML models (natural language processor) to identify call features such as conversation topics and facts, important (e.g., goal relevant, related to customer satisfaction, related to resolving an issue on the first call from a customer, related to the user's equipment, related to quality of service, and/or related to revenue) conversation portions, and the like.” See also ¶ [0020]: “Some of the ML models may use call audio as input, for example, to determine caller sentiment. Other ML models may use transcribed text as input.” ¶ [0026]: “The transcription service 120 may be configured to transcribe the caller audio stream 124 and/or the agent audio stream 124.” The transcript is parsed to identify portions: ¶ [0025]: “the stream handler ensemble may use the stream status events to control invocations of the transcription service 120, such as selecting portions of the audio that are selected for transcription.”
detect, prior to termination of the communication session and via input of the at least the portion of the electronic transcript into a first model trained with machine learning on historical log data, a trigger phrase based on the at least the portion of the electronic transcript wherein the trigger phrase maps to a performance event concerning an application :
Ellison discloses detection in real-time prior to termination of the communication session — the entire system operates in real-time during the ongoing call:
Ellison discloses “issue related to user equipment, see¶ 18). In some instances, the expert assist tool may identify a topic change using a rules-based approach, see¶ 53. This approach may identify keywords in the transcription to determine a likely topic change. These keywords can include words and phrases that suggest satisfaction/resolution, transition phrases, change in frequency of keywords (e.g., words relating to devices decrease, words relating to billing/payment increase), and/or other similar keywords. That is, Ellison discloses input of the transcript portion into a first model that detects a trigger phrase (characteristic/topic/keyword). Specifically, with reference to FIG. 11, step 1120:
¶ [0102]: “The computing devices 520 provide the customer interaction data 522 as an input to a first model that is configured to determine a characteristic of the interaction between the first user and the second user (1120). The computing devices 520 receive, from the first model, the characteristic of the interaction between the first user and the second user (1130).” The characteristic determined by the first model includes keywords and topics (i.e., trigger phrases): ¶ [0102]: “the characteristic of the interaction may include keywords of the customer interaction data 522, articles or documents related to the customer interaction data 522… In some implementations, the characteristic of the interaction may include… an initial reason that the first user requested to interact with the second user.” So the first model is trained with machine learning on historical log data:
¶ [0103]: “The first model may be a model trained using machine learning.”
¶ [0107]: “The computing devices 520 may train the models using machine learning and historical data… The historical data may include previous customer interaction data and previous data that includes characteristics of the interaction.”
¶ [0045]: “There may be a substantial historical database or store of such calls and associated data that can be used for training.”
As for “the trigger phrase maps to a performance event concerning an application”— the detected characteristics include issues related to the service provider’s products and services:
¶ [0018]: “call features such as conversation topics and facts, important (e.g., goal relevant, related to customer satisfaction, related to resolving an issue on the first call from a customer, related to the user’s equipment, related to quality of service).”
¶ [0044]: “A conversation between the caller and the agent may have one or more topics such as… billing issues, service issues, device issues and any suitable classification for associating the conversation or conversation portion with a useful and/or helpful action.”
responsive to the detection of the trigger phrase by the first model, input the trigger phrase into a second model;
Ellison explicitly discloses that, responsive to the first model’s detection of the characteristic, the detected characteristic is provided as input to a second model. With specific reference to FIG. 11, steps 1130 → 1140: ¶ [0111]: “The computing devices 520 provide the characteristic of the interaction between the first user and the second user and the customer interaction data as inputs to a second model that is configured to determine instructions for the second user to continue interacting with the first user during the interaction between the first user and the second user (1140).” The input into the second model is responsive to (i.e., conditioned upon) the first model’s detection:
¶ [0112]: “the computing devices 520 may select the second model from a group of multiple models… The computing devices 520 may select a model based on the sentiment of the first user.”
This demonstrates that invocation of the second model is conditional upon and responsive to the output/detection of the first model. The first model’s output (the detected characteristic/trigger phrase) is what triggers and informs the selection and input into the second model.
generate, using the second model trained with a transformer-based neural network on data corresponding to performance events of the application, a search query configured for input into a search engine;
Ellison describes generating queries from detected topics/issues using ML models, including transformer-based models ([0046], [0082]–[0084], FIGS. 7–8). Ellison discloses a second model that receives the first model’s output and generates actionable output, including identifying relevant articles and documents: ¶ [0109]: “the resulting model trained using data samples that included text-based interaction data and data identifying articles and documents relevant to the interaction data may be configured to receive text-based customer interaction data and output data identifying articles and documents relevant to the interaction data.”¶ [0116]: “The computing devices 520 may train the second model and additional models in a similar fashion… The computing devices 520 may generate data samples from historical data.” Ellison further discloses that the system surfaces relevant articles/documents to the agent based on detected topics: ¶ [0051]: “Relevant agent procedures or customer informational articles 306 may be similarly surfaced.” ¶ [0083]: “Additional ML models may be configured to identify or assist in identifying keywords, trending topics, articles and documents relevant to a conversation.”
Ellison further discloses the use of a transformer model:
¶ [0049]: “The social and messaging product development distributed event streaming 208 may use the transformer 206 to access the ML model 202.”
Ellison discloses multiple ML models arranged in a directed acyclic graph (FIG. 9; ¶¶ [0082]–[0084]) where outputs of one model feed as inputs to a subsequent model to perform a specialized task.
However, Ellison does not explicitly disclose that the second model is trained with a transformer-based neural network.
select, via the search engine, an electronic resource responsive to the search query generated via the second model;
Ellison describes selecting relevant articles/resources via search engines based on generated queries ([0051]–[0053], [0084], FIGS. 3–4). Ellison discloses selecting relevant electronic resources (articles/documents) based on the detected topics and ML model outputs:
¶ [0051]: “one or more of the published ML models may be applied to the audio and/or transcription streams associated with a current call to determine one or more call features (including a current topic of conversation) and, based on the determined call features, one or more portions of historical conversations may be presented to the agent… Relevant agent procedures or customer informational articles 306 may be similarly surfaced. The example GUI shows four relevant articles 320, 322, 308, and 310 being presented to the agent.”
transmit, for receipt by the service device prior to termination of the communication session with the client device, the electronic resource or an identification of the electronic resource.
For the above steps of “generate, select and transmit” see the detailed discussion above with respect to the teachings in ¶ 0018 and see also ¶ s 46, 51, 99, 107, Fig. 1, Fig. 5, Fig. 9 and Fig. 11. Ellison describes presenting selected resources to the agent during the call ([0051]–[0053], FIGS. 3–4, 10–11). Ellison discloses transmitting the electronic resource to the service device (agent desktop) during the ongoing call (prior to termination): ¶ [0051]: “in real-time, one or more response options 316 and 318 that are relevant to a call in-progress” are presented to the agent. ¶ [0096]: “the computing devices 520 provide, for output to the second user, the instructions for the second user to continue interacting with the first user.” ¶ [0019]: “one or more microapps 112 may use the output of one or more ML models published with an ML model output publishing service.” See also FIGS. 3–4 depicting the agent-facing GUI displaying articles and response options during the active call, and FIG. 5 showing the proposed response prompt 520 delivered to the customer service agent terminal 512.
While Ellison teaches the use of different ML models (see Fig. 9 and ¶ 83) including ones for detecting keywords to be searched, it does not explicitly teach the use of “transformer-based neural network” specifically trained to generate a search query configured for input into a search engine.
On one hand, the use of transformer-based neural network such as GPT has been well-known in the art for many years.
On the other hand, Curtis teaches the use of logic 160 which acquires training data for training one or more machine learning models to generate one or more search query statements from natural language description of a search query. In some implementations, sources of training data may include community forums, online query language documentation, manually created training data, training data (“electronic resource”) automatically generated by a machine learning model (e.g., a natural language model), etc. In some instances, such as with community forums, the data posted to a community forum may be parsed and searched with examples of natural language descriptions and corresponding search query statements extracted as training data. Various machine learning models may then be trained on the training data including, but not limited to, transformer deep learning models such as bidirectional encoder pretrained systems (BERT) and generative pretrained transformer (GPT). That is, Curtis discloses a system that uses transformer-based neural networks to generate search queries from natural language input. Curtis ¶ [0039]: “The query generation model training logic 160 is generally configured to… acquir[e] training data for training one or more machine learning models to generate one or more search query statements from a natural language description of a search query. In some implementations, sources of training data may include community forums, online query language documentation, manually created training data, training data automatically generated by a machine learning model (e.g., a natural language model), etc… Various machine learning models may then be trained on the training data including, but not limited to, transformer deep learning models such as bidirectional encoder pretrained systems (BERT) and generative pretrained transformer (GPT).”
Curtis ¶ [0023]: “the machine learning model may be a generative pre-trained transformer trained to transform the natural language description to executable software code (an executable search query statement).”
Curtis ¶ [0022]: “The following disclosure provides for systems and methods… directed to automatically translating a natural language description of a search query to a search query statement, such as an SPL query statement, using artificial intelligence (AI).”
Curtis ¶ [0024]: “querying programming languages are complex and have particular syntax rules… crafting executable search query statements that accurately reflect a desired search query or analysis is a difficult task… However, in order to utilize the systems and methods described herein, users only need to have a sense of the search query or analysis they want to be executed and be able to provide a natural language description of that.”
Curtis thus teaches that transformer-based neural networks (BERT, GPT) are effective at receiving natural language input (such as a detected trigger phrase describing a performance issue) and generating a structured search query configured for input into a search engine.
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to implement Ellison’s second model — which receives the first model’s detected characteristic (trigger phrase) and generates output identifying relevant articles and documents (¶¶ [0109], [0111]) — as a transformer-based neural network trained to generate a search query, as taught by Curtis, for the following reasons:
Same field of endeavor: Both Ellison and Curtis relate to using machine learning models to process natural language input and retrieve relevant information for users.
Ellison already teaches the sequential architecture (first model and second model): Ellison explicitly discloses providing the first model’s output as input to a second model for generating actionable output (FIG. 11, ¶¶ [0102]–[0111]). Ellison also discloses the use of a transformer (¶ [0049]). The modification merely specifies the type of neural network (transformer-based) used in the second model.
Ellison identifies the need for a ML models to identify “articles and documents relevant to a conversation” (¶¶ [0083], [0109]) but does not specify the type. Curtis teaches that transformer-based model may be used.
The advantages of using a transformer-based neural network such as GPT are well-known (e.g., superior abilities, efficiency and accuracy).
Claims 9 and 17 are rejected for the same reasons ac claim 1.
Claim 2 recites a voice call and application provided by computing system. See voice calls, agent devices, and in-call assistance (Ellison, ¶¶ [0020]–[0028], [0065]). Ellison explicitly describes voice calls between client and service devices ([0020]–[0024]).
Claim 10 and 18 are rejected for the same reasons as claim 2.
Claim 3 recites a trigger phrase corresponds to a client statement identifying the issue corresponding to performance event of the application. Ellison: explicitly detects client phrases that identify issues/topics (¶¶ [0044]–[0053]). ¶ 18 refers to “resolving an issue”(this may read as the claimed “performance event” of an application) on the first call from a customer, or related to the user's equipment, related to quality of service (this may also read on the claimed “performance event” of the application).
Claim 11 and 19 are rejected for the same reasons as claim 3.
Claim 4 recites that the system is configured to select documents, generate ranking by similarity, and display ordered results. Ellison teaches ranking and displaying during call (¶¶ [0035], [0051]–[0053], FIG.3). Ranking is inherently or at least obviously based on the well-known “similarity search”.
Claim 12 and 20 are rejected for the same reasons as claim 4.
Claim 5 recites multiple trigger phrase detection, second search, render second resource before call ends. Ellison teaches detecting multiple topics, performing repeated searches and surfaces subsequent resources in call (¶¶ [0088]–[0097]).
Claim 13 is rejected for the same reasons as claim 5.
Claim 6 recites that the system is configured to detect call end, generate LLM style summary using responses and provide display. Ellison teaches end of call memos/summaries with LLMs or generative ML and display (¶¶ [0055]–[0059], [0098]–[0120], FIG.4). Furthermore, LLM summarization has been widely used for many years and has been available commercially.
Claim 14 is rejected for the same reasons as claim 6.
Claim 7 recites: select plurality of resources, score according to relation to query, and recommend the best. Ellison teaches scoring, ranking, recommending top results (¶¶ [0046], [0051]–[0053]).
Claim 15 is rejected for the same reasons as claim 7.
Claim 8 recites : route call to second service device based on trigger phrase and trigger the routing. Ellison teaches agent routing and escalation based on detected issues/topics (¶¶ [0066], [0082]). Contact center routing practice was common.
Claim 16 is rejected for the same reasons as claim 8.
Response to Arguments
Applicant's arguments filed 7/1/2026 have been fully considered but they are not persuasive.
Most of Applicant’s arguments regarding the 103 rejection of Ellison and Curtis have been addressed in the above rejection.
Fig. 11 of Ellison, reproduced below with added emphasis should also address many of applicant’s arguments.
I. Ellison Teaches the Sequential Two-Model Architecture
Applicant’s central argument is that Ellison does not teach a first model detecting a trigger phrase and, responsive to that detection, inputting the trigger phrase into a second model that generates a search query. The Examiner respectfully disagrees. The Examiner directs Applicant’s attention to FIG. 11 and ¶¶ [0102]–[0111], which explicitly teach the sequential, two-model architecture:
¶ [0102]: “The computing devices 520 provide the customer interaction data 522 as an input to a first model that is configured to determine a characteristic of the interaction… The computing devices 520 receive, from the first model, the characteristic of the interaction.” (Steps 1120–1130.)
¶ [0111]: “The computing devices 520 provide the characteristic of the interaction… as inputs to a second model that is configured to determine instructions for the second user.” (Step 1140.)
¶ [0114]: “The computing devices 520 may select a second model based on the confidence score output by the first model.”
This is an explicit teaching of: first model detects characteristic (trigger phrase) → responsive to that detection → characteristic is provided as input to a second model. The invocation of the second model is conditioned on the first model’s output.
The detected “characteristic” includes “keywords,” “an initial reason that the first user requested to interact,” and matters “related to resolving an issue… related to the user’s equipment, related to quality of service” (¶¶ [0018], [0102]) — i.e., trigger phrases mapping to performance events.
The second model outputs identification of relevant resources: “the resulting model… may be configured to… output data identifying articles and documents relevant to the interaction data” (¶ [0109]).
II. Curtis Is Relied Upon Solely for Transformer-Based Query Generation
Applicant argues that Curtis does not teach the sequential architecture or trigger phrase detection. Curtis is not relied upon for those limitations. Ellison teaches the sequential two-model architecture. Curtis is relied upon only for the teaching that transformer-based neural networks (BERT, GPT) are effective for generating search queries from natural language input (Curtis ¶¶ [0023], [0039]). In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
III. Motivation to Combine Is Articulated and Non-Conclusory
Applicant argues the motivation to combine is conclusory and does not explain why one would arrange models in the claimed sequential configuration.
Ellison already provides the sequential configuration (FIG. 11). The combination with Curtis does not create this arrangement — it merely specifies the model type (transformer-based) for Ellison’s already-existing second model position. The above rejection demonstrates :
Curtis teaches transformer-based models excel at generating search queries from natural language input (¶¶ [0023], [0039]);
Implementing Ellison’s second model as a transformer-based query generator is a simple substitution of one known model type for another, yielding the predictable result of improved query accuracy. KSR, 550 U.S. at 416.
No “unacknowledged structural modifications” are required. Ellison’s architecture remains intact; only the model type at the second position is specified per Curtis’s teaching.
The use of multiple models (such as a first model to detect, and a second model to search) is old and well-known in the art. This simply falls under a category of “Task Specialization” or “collaboration”: one model specializes in and performs one task while the second model specializes in and performs another task based on input from the first model. This well-known structure (which is clearly taught by Ellison and others) provides well-known and expected results : efficiency and ease of maintenance/repair or update of separate models vs. having one single model perform many specialized tasks. The second model may be an inexpensive one which may only be used when needed.
PNG
media_image1.png
996
682
media_image1.png
Greyscale
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Niu et al (US 12,393,617 B1) discloses a system for “Document Recommendation Based on Conversational Log for Real Time Assistance” and teaches the use of Sequential Model Architecture.
Niu et al discloses a contact center service that provides real-time assistance to agents during ongoing voice calls with customers. The system uses a transcription engine to convert audio samples of the communication session into a parsed electronic transcript (utterances), which are then processed through two sequential machine learning modules. A first model (dual encoder stacked ensemble — transformer-based, trained on historical labeled conversation data) receives portions of the transcript and determines a probability that the utterance contains a trigger phrase identifying a customer’s issue with a product or service (i.e., maps to a performance event). A comparator module compares this probability to a threshold — if and only if the threshold is satisfied, the detected trigger utterance is passed to a second model. This explicit conditional gate means the second model operates only responsive to the first model’s detection.
The second model (retrieval module 184, employing a BERT-based transformer-based neural network trained on enterprise-specific knowledge base data corresponding to product/service issues) receives the trigger utterance, encodes it as an embedding (functioning as a search query), compares it via similarity search against pre-embedded knowledge base documents, selects the highest-scoring electronic resources, and transmits the relevant documents (or identifiers thereof) to the agent’s device prior to termination of the communication session.
Niu states this two-model sequential architecture conserves computational resources by ensuring the expensive retrieval model is only invoked when the first model has affirmatively detected a trigger — the same resource-efficiency rationale articulated in Applicant’s specification
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AHMAD F. MATAR whose telephone number is (571)272-7488. The examiner can normally be reached M-F 9 - 5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AHMAD F. MATAR/ Supervisory Patent Examiner, Art Unit 2693
Prosecution Insights