Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
2. A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on October 14, 2025 has been entered.
Information Disclosure Statement
3. The information disclosure statement (IDS) submitted on March 02, 2022 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Response to Amendment and Arguments
4. The amendment filed on October 14, 2025 has been entered. Claims 1, 10, and 19 are amended.
Applicant argues that the prior art of record does not disclose the amendments to the independent claims. The examiner agrees with this assertion.
Applicant’s arguments with respect to the 35 U.S.C. 103 rejections for claims 1-20 have been considered but are moot because the arguments are directed towards amended claim language, addressed on new grounds of rejection below.
Claim Rejections - 35 USC § 103
5. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically taught as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
6. Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Streit (U.S.
Publication (U.S. Publication No. 20220277064) in view of Murphy (U.S. Publication No. 20190325259).
Regarding claim 1, Streit discloses a method for handling unlabeled interaction data with contextual understanding, the method comprising:
receiving the interaction data describing agent-consumer interactions associated with a contact center ([0133] - privacy enabled, one-to-many identification of a callers' voice finds the associated UUID for the voice or actor of the underlying message. When a customer communicates with a particular entity, such as a contact center, the system can be configured to make a recording of the real-time call (e.g., using Amazon Kinesis Video Streams “KVS” or other capture and/or streaming service) including both the customer's and agent's voices);
analyzing the interaction data to identify a plurality of features ([0076] - new labels and/or encrypted feature vectors can be communicated from local to remote and vice versa allowing classification networks to be trained on the new label (e.g., 361));
automatically performing taxonomy driven classification on the plurality of features to generate a first set of labels associated with the interaction data ([0076] - new labels and/or encrypted feature vectors can be communicated from local to remote and vice versa allowing classification networks to be trained on the new label (e.g., 361));
combining the first and second sets of labels to obtain a combined set of labels associated with the interaction data ([0076] - a “new” local label and embeddings can be communicated to the remote server 351, which identifies an existing match to the same embeddings. In this example, the labels are reconciled (e.g., 362—merged label, retrain network, link label, or distribute classification network with exiting label, etc.) across the system and the networks updated accordingly);
and retraining, using the combined set of labels, one or more machine learning models including the deep learning model to enhance contextual understanding of the agent-consumer interactions associated with the contact center ([0076] - a “new” local label and embeddings can be communicated to the remote server 351, which identifies an existing match to the same embeddings. In this example, the labels are reconciled (e.g., 362—merged label, retrain network, link label, or distribute classification network with exiting label, etc.) across the system and the networks updated accordingly).
However, Streit does not disclose training a deep learning model with input of the interaction data and the first set of labels generated from the taxonomy driven classification to refine the first set of labels to output a second set of labels.
Murphy does teach training a deep learning model with input of the interaction data and the first set of albels generated from the taxonomy driven classification to refine the first set of labels to output a second set of labels ([0021] - the multi-label classification algorithm includes submitting the multi-label object detection labels as inputs to the multi-label classification algorithm and submitting the multi-label ground truth labels from the enterprise taxonomy as outputs from the multi-label classification algorithm. The multi-label classification algorithm includes discriminatively categorizing the multi-label object detection labels using at least one of a support vector machine, a Bayes classifier, a neural network, a random forest method, and a deep learning method neural network).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Streit to incorporate the teachings of Murphy to implement training a deep learning model with input of the interaction data and the first set of labels generated from the taxonomy driven classification to refine the first set of labels to output a second set of labels. Doing so allows improved multi-label prediction performance (Murphy [0057]).
Regarding claim 2, Streit in view of Murphy teaches all limitations of claim 1, above.
Streit discloses the method, wherein analyzing the interaction data comprises:
configuring and performing at least one of word embeddings, topic modeling, or theme mining ([0117] - entities (e.g., systems, processes, etc.) or actors (e.g., individual, groups, people, intelligences, etc.) (802) are responsible for digital activity/content (804) on devices. Where the digital activity includes identifying information (e.g., voice, audio, face images, biometric data, video, etc.) that activity can be processing by embedding networks to produce encrypted feature vectors (806) representing that activity. The system can assign and manage unique labels for respective encrypted feature vectors (808) which allows for training of universal identification networks (e.g., (810) including, for example, classification networks) that output the unique label whenever the respective encrypted feature vectors are input);
categorizing the interaction data into a list of topics or relevant n-grams ([0117] - entities (e.g., systems, processes, etc.) or actors (e.g., individual, groups, people, intelligences, etc.) (802) are responsible for digital activity/content (804) on devices. Where the digital activity includes identifying information (e.g., voice, audio, face images, biometric data, video, etc.) that activity can be processing by embedding networks to produce encrypted feature vectors (806) representing that activity. The system can assign and manage unique labels for respective encrypted feature vectors (808) which allows for training of universal identification networks (e.g., (810) including, for example, classification networks) that output the unique label whenever the respective encrypted feature vectors are input);
and identifying the plurality of features based on categorizing the interaction data ([0117] - entities (e.g., systems, processes, etc.) or actors (e.g., individual, groups, people, intelligences, etc.) (802) are responsible for digital activity/content (804) on devices. Where the digital activity includes identifying information (e.g., voice, audio, face images, biometric data, video, etc.) that activity can be processing by embedding networks to produce encrypted feature vectors (806) representing that activity. The system can assign and manage unique labels for respective encrypted feature vectors (808) which allows for training of universal identification networks (e.g., (810) including, for example, classification networks) that output the unique label whenever the respective encrypted feature vectors are input).
Regarding claim 3, Streit in view of Murphy teaches all limitations of claim 1, above.
Streit discloses the method, wherein the plurality of features comprises one or more taxonomy suggestions, the method further comprising:
transmitting the one or more taxonomy suggestions to an expert for a one-time expert review ([0133] - the system is configured to segment the recording to extract at least a portion of the customer's voice to create an encrypted voice embedding, and can then transmit the encrypted voice embedding (encrypted payload) across the network to a server (e.g., for remote identification). The server is configured to determine any match and returns a matching label (e.g., uuid)),
and wherein the first set of labels is generated responsive to the one-time expert review the system is configured to segment the recording to extract at least a portion of the customer's voice to create an encrypted voice embedding, and can then transmit the encrypted voice embedding (encrypted payload) across the network to a server (e.g., for remote identification). The server is configured to determine any match and returns a matching label (e.g., uuid).
Regarding claim 4, Streit in view of Murphy teaches all limitations of claim 1, above.
Streit discloses the method, wherein the taxonomy driven classification comprises one or more of algorithms including exclusive n-grams extractions, exclusive collocation extractions, fuzzy string match, or custom named entity recognition ([0085] - the identification system can be used to provide validation of identity, and an entity may use that validation to control their own identification environments based on the validation of identity (and for example, returned user ID). As the system can be configured to validate identity and return any arbitrary unique user ID, the system can be configured to map UUIDs to any value usable by an entity seeking to perform identity validation. Mapping can be accomplished as part of training a neural network on a label (e.g., the neural network will then output the UUID on input of encrypted feature vectors), or by returning a label value and mapping the label value to a UUID that the system can communicate to the entity).
Regarding claim 5, Streit in view of Murphy teaches all limitations of claim 1, above.
Streit discloses the method, further comprising:
performing multi-class and multi-label classification using the trained deep learning model ([0077] - the remote identification services can use the various identifications produced locally at a multitude of devices to define neural networks on a multitude of users, and multitudes of activity identities),
and wherein the second set of labels is determined based on the multi-class and multi-label classification ([0077] - the remote identification services can use the various identifications produced locally at a multitude of devices to define neural networks on a multitude of users, and multitudes of activity identities).
Regarding claim 6, Streit in view of Murphy teaches all limitations of claim 5, above.
Streit discloses the method, wherein performing the multi-class and multi-label classification comprises:
detecting multiple contexts within an utterance of the interaction data ([0076] - Local in this context is based on being at or proximate to a device where plaintext identifying information is being captured for identification. Show in FIG. 3 is an identification system that integrates local identification functions (e.g., 302) and remote identification functions (e.g., 352) to establish the private identity system 300. According to some embodiments, each of the identification functions (local and remote) can be used to establish, train, and/or update (e.g., 360) neural networks (304-308 & 354-358) that are configured to generate private identification. The neural networks can then be synchronized in the local and remote settings. In some embodiments, new labels and/or encrypted feature vectors can be communicated from local to remote and vice versa allowing classification networks to be trained on the new label (e.g., 361). In some alternatives, a “new” local label and embeddings can be communicated to the remote server 351, which identifies an existing match to the same embeddings. In this example, the labels are reconciled (e.g., 362—merged label, retrain network, link label, or distribute classification network with exiting label, etc.) across the system and the networks updated accordingly);
and detecting an order of the multiple contexts in the interaction data, wherein the second set of labels is determined based on analyzing the multiple contexts and the order ([0076] - Local in this context is based on being at or proximate to a device where plaintext identifying information is being captured for identification. Show in FIG. 3 is an identification system that integrates local identification functions (e.g., 302) and remote identification functions (e.g., 352) to establish the private identity system 300. According to some embodiments, each of the identification functions (local and remote) can be used to establish, train, and/or update (e.g., 360) neural networks (304-308 & 354-358) that are configured to generate private identification. The neural networks can then be synchronized in the local and remote settings. In some embodiments, new labels and/or encrypted feature vectors can be communicated from local to remote and vice versa allowing classification networks to be trained on the new label (e.g., 361). In some alternatives, a “new” local label and embeddings can be communicated to the remote server 351, which identifies an existing match to the same embeddings. In this example, the labels are reconciled (e.g., 362—merged label, retrain network, link label, or distribute classification network with exiting label, etc.) across the system and the networks updated accordingly).
Regarding claim 7, Streit in view of Murphy teaches all limitations of claim 1, above.
Streit discloses the method of claim 1, wherein:
automatically performing the taxonomy driven classification is based on one or more unsupervised machine learning (ML) approaches ([0147 - Various embodiments use different machine learning models for capturing feature vectors in the first network. According to various embodiments, the feature vector capture is accomplished via a pre-trained neural network (including, for example, a convolutional neural network) where the output is Euclidean measurable. In some examples, this can include models having a softmax layer as part of the model, and capture of feature vectors can occur preceding such layers. Feature vectors can be extracted from the pre-trained neural network by capturing results from the layers that are Euclidean measurable. In some examples, the softmax layer or categorical distribution layer is the final layer of the model, and feature vectors can be extracted from the n−1 layer (e.g., the immediately preceding layer). In other examples, the feature vectors can be extracted from the model in layers preceding the last layer. Some implementations may offer the feature vector as the last layer),
training the deep learning model is based on one or more supervised ML approaches ([0148] - The resulting feature vectors are bound to a specific user classification at 1412. For example, deep learning is executed at 1412 on the feature vectors based on a fully connected neural network (e.g., a second neural network). The execution is run against all the biometric data (i.e., feature vectors from the initial biometric and training biometric data) to create the classification information. According to one example, a fully connected neural network having two hidden layers is employed for classification of the biometric data. In another example, a fully connected network with no hidden layers can be used for the classification. According to one embodiment, process 1400 can be executed to receive an original biometric (e.g., at 1402) generate feature vectors (e.g., 1410), and apply a FCNN classifier to generate a label to identify a person at 1412 (e.g., output #people)),
and obtaining the combined set of labels is based on combining the first set of labels predicted using the one or more unsupervised ML approaches and the second set of labels predicted using the one or more supervised ML approaches ([0076] - a “new” local label and embeddings can be communicated to the remote server 351, which identifies an existing match to the same embeddings. In this example, the labels are reconciled (e.g., 362—merged label, retrain network, link label, or distribute classification network with exiting label, etc.) across the system and the networks updated accordingly).
Regarding claim 8, Streit in view of Murphy teaches all limitations of claim 1, above.
Streit discloses the method, further comprising:
configuring and adjusting, based at least in part on a type of interaction data, one or more algorithms used in each of analyzing the interaction data, automatically performing the taxonomy driven classification, or training the deep learning model ([0076] - a “new” local label and embeddings can be communicated to the remote server 351, which identifies an existing match to the same embeddings. In this example, the labels are reconciled (e.g., 362—merged label, retrain network, link label, or distribute classification network with exiting label, etc.) across the system and the networks updated accordingly),
and wherein the configuring and adjusting comprise changing at least one of a number of the one or more algorithms or an order of the one or more algorithms ([0076] - a “new” local label and embeddings can be communicated to the remote server 351, which identifies an existing match to the same embeddings. In this example, the labels are reconciled (e.g., 362—merged label, retrain network, link label, or distribute classification network with exiting label, etc.) across the system and the networks updated accordingly).
Regarding claim 9, Streit in view of Murphy teaches all limitations of claim 1, above.
Streit discloses the method, wherein prior to analyzing the interaction data, the method comprises:
selecting and customizing pre-processing operations ([0079] - the device can be configured to provide different operations, levels of access, etc., which can be tailored to the respective user identified by the private identity functions. In one example, an administrative user can be given the authority to define what functions a group of users (e.g., 312) are given access to, including the ability to assign or remove functionality for various user/identities. Each time new identification samples are detected on the device (e.g., new user of the device and/or new activity on the device), the local identification functions (302) will attempt to identify the new user (which can include continuously verifying the identity of the current user) and/or various activity monitors will review digital activity on the device and attempt identification of the actor associated with the digital activity (e.g., 314));
and pre-processing the interaction data using the selected pre-processing operations ([0079] - the device can be configured to provide different operations, levels of access, etc., which can be tailored to the respective user identified by the private identity functions. In one example, an administrative user can be given the authority to define what functions a group of users (e.g., 312) are given access to, including the ability to assign or remove functionality for various user/identities. Each time new identification samples are detected on the device (e.g., new user of the device and/or new activity on the device), the local identification functions (302) will attempt to identify the new user (which can include continuously verifying the identity of the current user) and/or various activity monitors will review digital activity on the device and attempt identification of the actor associated with the digital activity (e.g., 314)).
Regarding claim 10, Streit discloses a system for handling unlabeled interaction data with contextual understanding, the system comprising:
a processor ([0243] - The computer system 1900 may include one or more processors 1910 and one or more articles of manufacture that comprise non-transitory computer-readable storage media (e.g., memory 1920 and one or more non-volatile storage media 1930));
and a memory in communication with the processor and comprising instructions which, when executed by the processor ([0243] - The computer system 1900 may include one or more processors 1910 and one or more articles of manufacture that comprise non-transitory computer-readable storage media (e.g., memory 1920 and one or more non-volatile storage media 1930)), program the processor to:
receive the interaction data describing agent-consumer interactions associated with a contact center ([0133] - privacy enabled, one-to-many identification of a callers' voice finds the associated UUID for the voice or actor of the underlying message. When a customer communicates with a particular entity, such as a contact center, the system can be configured to make a recording of the real-time call (e.g., using Amazon Kinesis Video Streams “KVS” or other capture and/or streaming service) including both the customer's and agent's voices);
analyze the interaction data to identify a plurality of features ([0076] - new labels and/or encrypted feature vectors can be communicated from local to remote and vice versa allowing classification networks to be trained on the new label (e.g., 361));
automatically perform taxonomy driven classification on the plurality of features to generate a first set of labels associated with the interaction data ([0076] - new labels and/or encrypted feature vectors can be communicated from local to remote and vice versa allowing classification networks to be trained on the new label (e.g., 361));
combine the first and second sets of labels to obtain a combined set of labels associated with the interaction data ([0076] - a “new” local label and embeddings can be communicated to the remote server 351, which identifies an existing match to the same embeddings. In this example, the labels are reconciled (e.g., 362—merged label, retrain network, link label, or distribute classification network with exiting label, etc.) across the system and the networks updated accordingly);
and retrain, using the combined set of labels, one or more machine learning models including the deep learning model to enhance contextual understanding of the agent-consumer interactions associated with the contact center ([0076] - a “new” local label and embeddings can be communicated to the remote server 351, which identifies an existing match to the same embeddings. In this example, the labels are reconciled (e.g., 362—merged label, retrain network, link label, or distribute classification network with exiting label, etc.) across the system and the networks updated accordingly).
However, Streit does not disclose training a deep learning model with input of the interaction data and the first set of labels generated from the taxonomy driven classification to refine the first set of labels to output a second set of labels.
Murphy does teach training a deep learning model with input of the interaction data and the first set of albels generated from the taxonomy driven classification to refine the first set of labels to output a second set of labels ([0021] - the multi-label classification algorithm includes submitting the multi-label object detection labels as inputs to the multi-label classification algorithm and submitting the multi-label ground truth labels from the enterprise taxonomy as outputs from the multi-label classification algorithm. The multi-label classification algorithm includes discriminatively categorizing the multi-label object detection labels using at least one of a support vector machine, a Bayes classifier, a neural network, a random forest method, and a deep learning method neural network).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Streit to incorporate the teachings of Murphy to implement training a deep learning model with input of the interaction data and the first set of labels generated from the taxonomy driven classification to refine the first set of labels to output a second set of labels. Doing so allows improved multi-label prediction performance (Murphy [0057]).
Dependent claims 11-18 are analogous in scope to claims 2-9, and are rejected according to the same reasoning.
Regarding claim 19, Streit discloses a computer program product for handling unlabeled interaction data with contextual understanding, the computer program product comprising a non-transitory computer readable medium having computer readable program code stored thereon, the computer readable program code configured to:
receive the interaction data describing agent-consumer interactions associated with a contact center ([0133] - privacy enabled, one-to-many identification of a callers' voice finds the associated UUID for the voice or actor of the underlying message. When a customer communicates with a particular entity, such as a contact center, the system can be configured to make a recording of the real-time call (e.g., using Amazon Kinesis Video Streams “KVS” or other capture and/or streaming service) including both the customer's and agent's voices);
analyze the interaction data to identify a plurality of features ([0076] - new labels and/or encrypted feature vectors can be communicated from local to remote and vice versa allowing classification networks to be trained on the new label (e.g., 361));
automatically perform taxonomy driven classification on the plurality of features to generate a first set of labels associated with the interaction data ([0076] - new labels and/or encrypted feature vectors can be communicated from local to remote and vice versa allowing classification networks to be trained on the new label (e.g., 361));
combine the first and second sets of labels to obtain a combined set of labels associated with the interaction data ([0076] - a “new” local label and embeddings can be communicated to the remote server 351, which identifies an existing match to the same embeddings. In this example, the labels are reconciled (e.g., 362—merged label, retrain network, link label, or distribute classification network with exiting label, etc.) across the system and the networks updated accordingly);
and retrain, using the combined set of labels, one or more machine learning models including the deep learning model to enhance contextual understanding of the agent-consumer interactions associated with the contact center ([0076] - a “new” local label and embeddings can be communicated to the remote server 351, which identifies an existing match to the same embeddings. In this example, the labels are reconciled (e.g., 362—merged label, retrain network, link label, or distribute classification network with exiting label, etc.) across the system and the networks updated accordingly).
However, Streit does not disclose training a deep learning model with input of the interaction data and the first set of labels generated from the taxonomy driven classification to refine the first set of labels to output a second set of labels.
Murphy does teach training a deep learning model with input of the interaction data and the first set of albels generated from the taxonomy driven classification to refine the first set of labels to output a second set of labels ([0021] - the multi-label classification algorithm includes submitting the multi-label object detection labels as inputs to the multi-label classification algorithm and submitting the multi-label ground truth labels from the enterprise taxonomy as outputs from the multi-label classification algorithm. The multi-label classification algorithm includes discriminatively categorizing the multi-label object detection labels using at least one of a support vector machine, a Bayes classifier, a neural network, a random forest method, and a deep learning method neural network).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Streit to incorporate the teachings of Murphy to implement training a deep learning model with input of the interaction data and the first set of labels generated from the taxonomy driven classification to refine the first set of labels to output a second set of labels. Doing so allows improved multi-label prediction performance (Murphy [0057]).
Dependent claim 20 is analogous in scope to claim 2, and is rejected according to the same reasoning.
Conclusion
7. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Ho (U.S. Publication No. 20200074984) teaches intent authoring using weak supervision and co-training for automated response systems. Lubowich (U.S. Publication No. 20100246799) teaches methods and apparatus for deep interaction analysis. Moloney (U.S. Publication No. 20210287080) teaches systems and methods for distributed training of deep learning models. Sundaram (U.S. Publication No. 20190355348) teaches system and method for a multiclass approach for confidence modeling in automatic speech recognition systems.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ETHAN DANIEL KIM whose telephone number is (571) 272-1405. The examiner can normally be reached on Monday - Friday 9:00 - 5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Richemond Dorvil can be reached on (571) 272-7602. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ETHAN DANIEL KIM/
Examiner, Art Unit 2658
/RICHEMOND DORVIL/Supervisory Patent Examiner, Art Unit 2658