DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 12/30/2025 with respect to the 35 USC 103 have been fully considered but they are moot as a reference has been added to address the amendment.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Freedman et al. (US 2004/0249650) in view of Jiang (US 2014/0207447) and Seetharaman et al. (US 2020/0286504).
Claim 1
Freedman teaches a method for processing a contact including a plurality of frames of audio including speech of an agent, the method comprising:
receiving content associated with a contact, wherein the content comprises an audio waveform and where in the audio waveform comprises a sequence of frames ([0045] audio signals; [0046] For example, in order to analyze a conversation carried out between two participants and recorded over a telephone line the audio signal should preferably be segmented in order to provide for the suitable analysis. The separated pieces of the information hidden in the signal frames are individually processed.);
determining a first set of frames from the sequence of frames, wherein the first set of frames includes at least a part of speech ([0046] The separated pieces of the information hidden in the signal frames are individually processed. As a result different and inherently integrated participants and conversation elements, such as speaker A 102, speaker B 104, Speaker A+B 102, 104, can be considered individually. [0047] a) signal 1 is a sequence of segments each of which belongs to speaker 1; );
wherein the first set of frames is identified using a first machine learning model, wherein the first machine learning model is trained using exemplary speech waveforms in order to identify frames that comprise speech ([0046] of Freedman, The system uses speaker segmentation either to obtain more data from a particular speaker or to identify the points in time when each speaker is speaking. System speech recognition performances are improved by adapting the functions to the acoustic models using the data obtained a priori from a particular speaker. [0055] of Freedman, automatic learning mechanism concerning the characteristic voice features of a specific speaker being analyzed. When a speaker is known to the system the reference voice characteristics thereof are learned "on-the-fly" during a real time session. In contrast, when a speaker is known to the system in advance of an initiated call the reference voice characteristics of the unknown speaker are extracted from the database with the activation of the call. The database is updated after each call in accordance with results of the learning process.);
determining a second set of frames from the sequence of frames, wherein the second set of frames includes non-standard noise ([0046] In addition segments including holds 106, noise, and silence 100 are also handled individually. [0047] Output includes the following signals or segments marked by a time index: … c) non-voice, d) silence, and e) talk over);
wherein the second set of frames is determined using a noise classification machine learning model, the noise classification machine learning model trained using waveform data related to non-standard noises ([0047], inherent acoustic model of the site; [0062] acoustic environment modeling; [0074] of Freedman, Referring back to FIG. 4, using the innovative solution presented by the invention, the playback application uses the output of the content analysis system, utilizing the results of both the pre-processing stage 82 and the analysis stage 84. These results were previously stored in the organization knowledge base 86 or in the ICS 94. The results of the audio classification functions 90, the analysis ASR 116, the audio analysis 114, the call flow and emotion 119 and speaker identification functions 118 are all obtained and further processed by the rule engine 112.
generating, based on the first set of frames and the second set of frames, a quality score associated with the contact ([0052], The call flow function 119 analyzes the dynamics of the call. The function 119 attempts to provide an indication of the call-flow parameters of the call. The calculated parameters include the percentages of the call's length, complete silence; talk over, agent speaking and customer speaking. The function 119 counts also the number of times the agents interrupts the speech of the customer and vice versa. It also gives details about the silence, talk over, and activity sections during the call. The function 119 is fed with a variety of streams where each stream represents a specific participant of the call; [0053] At step 186 the sections are processed such that statistics are generated concerning each participant activity and the mutual activities are calculated.);
transmitting at least one of the quality score associated with the contact or the agent quality score ([0036]The present invention provides a system for the analysis of at least two interactions captured as a result of the agent's interaction with the client. Analyzing more then one interaction enables system according to the present invention to effectively monitor the interactions between the agent and the client. Such interactions may take place between a business and a customer or between businesses. The interactions captured can be associated there with each other and with other information already present in the organization, such as the organization knowledge base. The interactions may also be associated with data received about the capturing of the interaction such as Computer Telephony Integration (CTI) information or various other data pertaining to the manner of recording and logging of the interaction. [0037], The performance of agents may be analyzed effectively through the capture and analysis of various data associated with the interaction with the client. The present invention provides for such a system. [0054], The system output is cross-referenced with other system outputs in order to improve the accuracy of the system or in order to yield higher order conclusions. Additional, types of system output or interactions may be associated with analyzed speech components to enhance the accuracy of the system and to better identify the speech segments to analyze or the operations and reactions of the contact center agent.).
Freedman discloses for example [0053] Referring now to FIG. 9 describing the operational steps of the call flow function which shows yet another example of the analysis of speech in accordance with the present invention. At step 180 a digital speech segment is introduced into the function. At step 182 the digital speech segment is sliced into frames of a few milliseconds. The energy of each frame is calculated and then compared to an adaptive threshold representing the maximum noise level. Frames with higher energy than the adaptive threshold are marked with an "activity on" flags while frames with energy lower than the threshold are marked with an "activity off" flag. Each participant of the call is represented as a vector of activity frames. At step 184 each participant-specific vector is passed through a filter. The filter yields a vector of "activity sections" where each section is constructed of consecutive or semi-consecutive frames marked with "activity on" flag. At step 186 the sections are processed such that statistics are generated concerning each participant activity and the mutual activities are calculated. However, Freedman may not clearly detail the call flow parameter/quality score is specifically based on the first set of frames and the second set of frames, and that the noise classification model is trained with a specific type of noise.
Jiang teaches a voice identification under a noise environment by obtaining noise scenario including noise type and noise magnitude ([0063][0085]). [0099] According to a VAD determination result, respectively calculate an average energy of the voice frame and the noise frame to obtain a voice energy level (speechLev) and a noise energy level (noiseLev), and then obtain, by calculating, a signal-to-noise ratio (SNR). [0100] where, Ln and Ls respectively represent the total number of noise frames and the total number of voice frames, ener[Ni] represents the energy of the i.sup.th noise frame, and ener[Sj] represents the energy of the j.sup.th voice frame.
Jiang further teaches that the noise classification model is trained with a specific type of noise [0072] of Jiang, In the embodiment of the present invention, a process of establishing the GMM of a certain noise type may be: inputting multiple groups of noise data of a same type, repeatedly training the GMM model according to the noise data, and finally obtaining the GMM of the noise type.);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to incorporate a form of SNR calculation as taught in Jiang with the customer interactions analysis system with the speech noise reduction function of Freedman because doing so would have greatly improved a voice identification rate under a noise environment ([0101] of Jiang).
Freedman discloses in [0052] Similarly, associations between various interactions may be analyzed as well. So for example, audio and video interactions or audio and CRM data associated with the same call may be analyzed to identify various predetermined combinations of events or elements relating to the handling of the call or query (or offer for goods or services), the response by the agent, the appropriate response to a client or entry of data into the CRM at any given time during a call or an interaction between the business and the customer. The person skilled in the art will appreciate the various types of interactions, which may be associated together and analyzed to obtain like result and enhance the ability to analyze and respond to various events. [0059] Using the rule engine a plurality of phenomena included in but not-limited to a session can be sensed, recognized, identified, organized and optionally handled: a) multiple occurrences of events in a certain time frame, b) sequenced or concurrent occurrences of events, c) logical relations between events, [0060] The recognized phenomena could include the following non-limiting exemplary conclusions:…d) the average percent of the agent's talking time…h) long or frequent hold periods or long and frequent silence periods, which imply that the interaction of the agent with the system is inefficient, [0076] The real time monitoring may also examine more than one interaction at the same time. Examiner notes [0052] discusses analysis of the dynamics of the call, including percentages of different characteristics such as agent speaking, activity sections, etc. [0059] further teaches identifying phenomena found across events and not limited to a session, such as average percent of the agent’s talking time). Still the combination does not explicitly detail generating an agent quality score by aggregating the quality score with a plurality of quality scores associated with the agent across a plurality of contacts, wherein the agent quality score identifies a deterioration of hardware equipment associated with the agent.
Seetharaman teaches the calculation of sound quality measure can be performed over time (e.g., running average, median, etc.) to facilitate presentation of feedback about the sound quality measure ([0037]). Seetharaman also teaches sound quality measure is affected by hardware equipment such as microphone quality ([0001]) and sound producing equipment (e.g. low quality loudspeaker) ([0038]). Therefore Seetharaman teaches quality changes of equipment over time can be identified based on the sound quality measure, which is now incorporated with Freedman’s capture and analysis of various data associated with the interaction with the client.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to incorporate audio performance monitoring as taught by Seetharaman with the customer interactions analysis system with the speech noise reduction function of Freedman in view of Jiang, because doing so would have facilitated changes to the recording setup that improve sound quality. ([0020] of Seetharaman).
Claim 2
Freedman of the combination further teaches the method of claim 1, the method further comprising: determining a third set of frames from the sequence of frames, wherein the third set of frames includes a noise, wherein the noise includes standard noise and non-standard noise ([0051], for example, lengthy humming segments vs music on hold); determining a fourth set of frames from the third set of frames, wherein the fourth set of frames includes the standard noise ([0051] for example, music on hold); and determining, based on a difference between the third set of frames and the fourth set of frames, a second set of frames, wherein the second set of frames includes the non-standard noise ([0051] Still referring to FIG. 4 the audio filtering gate module 91 decides which audio segments are eligible for further analysis and which are non-eligible….However, lengthy "humming" segments will be eligible for analysis for quality control and management purposes. Other parts such as music on hold and radio on hold and the like can also be removed as not suited for further analysis. Based on the audio classification module 90 results the system automatically predicts which audio segments will be suitable for analysis. As a result only specifically selected audio segments are fed to the analysis stage 84. Examiner notes the audio filter gate for example, determines the second set of frames (lengthy humming segments) for further analysis after determining removal of the fourth set of frames (music on hold)).
Claim 3
Freedman of the combination further teaches the method of claim 1, wherein the non-standard noise originates from at least one of: a mechanical source, a street, an ambient humming sound, a human as a source, or an animal as a source (See different speakers (Speaker A, Speaker B, etc. in Fig. 4 of Freedman; [0050] The humming sounds elimination algorithm is also a part of the noise reduction function. "Humming" sounds usually resemble time-domain impulse trains, reflected in frequency-domain impulse trains (possibly widened) that are stationary over relatively long periods (500 mS). Such noises are usually typical to HF environments, or to acoustic environments that are subjected to mechanical periodic sources such as low RPM engines, propellers etc. They are frequently accompanied by white or slightly colored noises. ).
Claim 4
Freedman of the combination further teaches the method of claim 2, wherein the standard noise includes at least one of: a section of hold music, interactive voice response (IVR) system noise, robotic process automation (RPA) system noise, a dial tone, or a beep sound before a voicemail recording starts (See at least Music&Tones 96 linked to audio classification module 90 in Fig. 4 and [0045] of Freedman, The audio classification module 90 utilizes a speech detection function in order to enable the system to identify and distinguish speech signal scenario from several inherently integrated speech elements, such as music and tones 96; [0051] Other parts such as music on hold and radio on hold and the like can also be removed as not suited for further analysis. ).
Claim 5
The combination further teaches the method of claim 2, wherein the determining the second set of frames uses a waveform analysis of power levels based at least on one of: peak-to-peak amplitude, signal zero-crossing rate, short-term energy of a power spectrum, or a use of filters based on the Mel-frequency cepstrum coefficients ([0067] of Jiang, after the noise data frame is obtained, a frequency cepstrum coefficient of the noise data frame is obtained… A frequency cepstrum coefficient (Mel Frequency Cepstrum Coefficient, MFCC) is a cepstrum coefficient on the Mel frequency, has good identification performance, and is widely applied to a field such as voice identification, voiceprint recognition, and language identification.[0068] S1022: Obtain, according to the frequency cepstrum coefficient of the noise and a pre-established noise type model, the noise type of the voice data.[0069] The frequency cepstrum coefficient is respectively substituted into each pre-established noise type model for calculation, and if a calculation result value of a certain noise type model is a maximum, it is considered that the user is located in an environment of the noise type when inputting the voice data, that is, the noise type of the voice data is obtained. ).
Claim 6
Freedman of the combination further teaches the method of claim 2, wherein the determining the second set of frames uses a frame classification model, wherein the frame classification model predicts the standard noise based on audio waveform ([0051] of Freedman, pre-processing functions performed by the audio classification module 90. See also [0046] and “music&tones 96 linked to the audio classification module 90 in Fig. 4).
Claim 7
The combination further teaches the method of claim 6, wherein the frame classification model includes a speech classification model and a noise classification model, and the method further comprising: training the speech classification model using first ground truth data including an audio waveform of the speech for machine learning ([0046] of Freedman, The system uses speaker segmentation either to obtain more data from a particular speaker or to identify the points in time when each speaker is speaking. System speech recognition performances are improved by adapting the functions to the acoustic models using the data obtained a priori from a particular speaker. [0055] of Freedman, automatic learning mechanism concerning the characteristic voice features of a specific speaker being analyzed. When a speaker is known to the system the reference voice characteristics thereof are learned "on-the-fly" during a real time session. In contrast, when a speaker is known to the system in advance of an initiated call the reference voice characteristics of the unknown speaker are extracted from the database with the activation of the call. The database is updated after each call in accordance with results of the learning process. ); and training the noise classification model using second ground truth data including an audio waveform of at least one of hold music, a dial tone, or a beep sound for machine learning ([0074] of Freedman, Referring back to FIG. 4, using the innovative solution presented by the invention, the playback application uses the output of the content analysis system, utilizing the results of both the pre-processing stage 82 and the analysis stage 84. These results were previously stored in the organization knowledge base 86 or in the ICS 94. The results of the audio classification functions 90, the analysis ASR 116, the audio analysis 114, the call flow and emotion 119 and speaker identification functions 118 are all obtained and further processed by the rule engine 112. [0072] of Jiang, In the embodiment of the present invention, a process of establishing the GMM of a certain noise type may be: inputting multiple groups of noise data of a same type, repeatedly training the GMM model according to the noise data, and finally obtaining the GMM of the noise type.).
Claim 8
Freedman of the combination further teaches the method of claim 1, the method further comprising: generating, based on the quality score associated with the contact including the agent, a quality score associated with the agent, wherein the quality score associated with the agent includes an average of a plurality of quality scores associated with contacts including the agent ([0059] Using the rule engine a plurality of phenomena included in but not-limited to a session can be sensed, recognized, identified, organized and optionally handled: a) multiple occurrences of events in a certain time frame, b) sequenced or concurrent occurrences of events, c) logical relations between events, [0060] The recognized phenomena could include the following non-limiting exemplary conclusions:…d) the average percent of the agent's talking time…h) long or frequent hold periods or long and frequent silence periods, which imply that the interaction of the agent with the system is inefficient; [0037], The performance of agents may be analyzed effectively through the capture and analysis of various data associated with the interaction with the client. The present invention provides for such a system. [0054], The system output is cross-referenced with other system outputs in order to improve the accuracy of the system or in order to yield higher order conclusions. Additional, types of system output or interactions may be associated with analyzed speech components to enhance the accuracy of the system and to better identify the speech segments to analyze or the operations and reactions of the contact center agent.).
Claim 9
The combination further teaches the method of claim 1, wherein the quality score associated with the contact is based on a ratio of: an average power level of speech in the sequence of frames, and an average power level of non-standard noise in the sequence of frames ([0099] Jiang, According to a VAD determination result, respectively calculate an average energy of the voice frame and the noise frame to obtain a voice energy level (speechLev) and a noise energy level (noiseLev), and then obtain, by calculating, a signal-to-noise ratio (SNR). [0100] where, Ln and Ls respectively represent the total number of noise frames and the total number of voice frames, ener[Ni] represents the energy of the i.sup.th noise frame, and ener[Sj] represents the energy of the j.sup.th voice frame.)
Claim 10
Freedman of the combination further teaches the method of claim 1, the method further comprising: generating, based on a ratio of a first number of frames in the first set of frames over a second number of frames in the second set of frames, the quality score associated with the contact ([0052]The function 119 attempts to provide an indication of the call-flow parameters of the call. The calculated parameters include the percentages of the call's length, complete silence; talk over, agent speaking; [0060] d) the average percent of the agent's talking time,).
Claim 11
This claim recites substantially the same limitations as those provided in claim 1 above, and therefore it is rejected for the same reasons.
Claim 12
This claim recites substantially the same limitations as those provided in claim 2 above, and therefore it is rejected for the same reasons.
Claim 13
This claim recites substantially the same limitations as those provided in claims 3 and 4 combined above, and therefore it is rejected for the same reasons.
Claim 14
This claim recites substantially the same limitations as those provided in claim 5 above, and therefore it is rejected for the same reasons.
Claim 15
This claim recites substantially the same limitations as those provided in claim 9 above, and therefore it is rejected for the same reasons.
Claims 16-20
These claims recite substantially the same limitations as those provided in claims 11-15 respectively, and therefore they are rejected for the same reasons.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to THOMAS H MAUNG whose telephone number is (571)270-5690. The examiner can normally be reached Monday-Friday, 9am-6pm, EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Carolyn R. Edwards can be reached on 1-(571) 2707136. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/THOMAS H MAUNG/Primary Examiner, Art Unit 2692