DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 29 July 2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1 and 4-20 are rejected under 35 USC 101 because the claimed invention is directed to an abstract idea without significantly more. The claims recite steps for analyzing audio to determine fraudulent audio. The limitations of claims 1 and 4-20, as drafted, are a computer program product or system that, under their broadest reasonable interpretation, cover performance of the limitations in the mind but for the recitation of generic computer components. That is, other than reciting “computer readable storage medium”, “program instructions” “computer”, “processor”, and “memory” nothing in the claim element precludes the steps from practically being performed in the mind and/or with pen and paper calculations. As to claims 1 and 11, under the BRI, a human could listen to an audio stream and compare it to saved audio stream recordings. The human could also transcribe the audio stream. Using the saved audio stream recordings as reference to the audio stream and its transcript, a human could determine the timing, With respect to the dependent claims specifying that the steps of the independent claims be performed by ML models, there appear to be no technical specifics about how this ML model might be structured or carry out analysis of the data. Accordingly, the steps of claims 1 and 11 are directed to organizing human interactions and/or a mental process. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claims recite an abstract idea. This judicial exception is not integrated into a practical application. In particular, claims 1-20 only recite the additional elements “computer readable storage medium”, “program instructions” “computer”, “processor”, and “memory” to perform the aforementioned steps. The processor and other hardware are recited at a high-level of generality (i.e., as a generic processor performing a generic computer function for transliterating text such that they amount to no more than mere instructions to apply the exception using generic computer components.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional hardware elements to perform both the aforementioned steps amount to no more than mere instructions to apply the exception using generic computer components. Mere instructions to apply an exception using generic computer components cannot provide an inventive concept. The claims are not patent eligible.
With the exception of claims 2-3, a similar analysis applies to the remaining dependent claims. Claims 2-3 specify the steps of claim 1 being performed on real-time audio, which goes beyond what a human could reasonably perform as a mental process.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-5, 7, 9, 11-13, 16-18, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over US 20220335949, hereinafter referred to as LI et al., in view of US 20230262160, hereinafter referred to as Trivedi et al., and further in view of US 20240040035
Regarding claim 1, LI et al. discloses a system for analyzing audio, comprising:
a memory configured to store known digital audio representations, wherein the known digital audio representations comprise two or more portions of fraudulent audio streams (“FIGS. 16A, 16B, 16C, 16D, 16E and 16F, together, illustrate aspects of a framework defining a set of processing operations to be performed to derive insights from the contents of recorded and stored speech audio,” LI et al., para [0048]. And, “Different machine-learning models may be used interchangeably to perform a task. Examples of tasks that can be performed at least partially using machine-learning models include various types of scoring; bioinformatics; cheminformatics; software engineering; fraud detection,” LI et al., para [0197]. The use of machine-learning for fraud detection in audio means that portions of fraudulent audio streams are used to train the fraud detection model.); and
a processor operably coupled to the memory (LI et al., para [0010].) and configured to:
receive a portion of an audio stream from an external device (“…divide the speech data set into multiple data segments that each represent a speech segment of multiple speech segments of the speech audio; use at least an acoustic model with each data segment of the multiple data segments to identify likely speech sounds in the speech audio,” LI et al., para [0011].);
produce a transcript of the portion of the audio stream (“…use at least an acoustic model with each data segment of the multiple data segments to identify likely speech sounds in the speech audio; and generate a transcript of the speech data set based, at least in part, on the identified likely speech sounds, or transmit an indication of the generation of the transcript to the requesting device,” LI et al., para [0011].);
determine a timing score by analyzing a timing of the portion of the audio stream and comparing it to labeled timing of the known digital audio representations, wherein analyzing the timing comprises determining a length of pauses between syllables in the portion of the audio stream (“Performing the preprocessing operations of the first pause detection technique may include the at least one processor being caused to perform operations including: divide the speech data set into multiple data chunks that each represent a chunk of multiple chunks of the speech audio; instantiate an instance of an acoustic model neural network comprising connectionist temporal classification (CTC) output; analyze the speech audio to identify pauses between speech sounds by providing each data chunk of the multiple data chunks to the instance of the acoustic model neural network as an input and monitor the CTC output for at least one corresponding string of blank symbols indicative of a pause; and analyze the lengths of the pauses to identify a first set of likely sentence pauses by comparing a length of each string of blank symbols from the CTC output to a predetermined blank threshold length,” LI et al., para [0018].);
determine an emotional score by analyzing an emotional content of the portion of the audio stream and comparing it to labeled emotional content of the known digital audio representations, wherein the emotional content is determined at least by analyzing the portion of the audio stream to determine which words are emphasized in the portion of the audio stream (“More specifically, using the transcript of the text data set 3700 as an input, one or more terms within the transcript (each including one or more words) may be identified as having one or more quantifiable characteristics (e.g., counts of occurrences of each term and/or aggregate counts of multiple terms, degree of relevance of a term within the transcript, degree of strength of positive or negative sentiment about a term, etc.), and/or relational characteristics (e.g., semantic and/or grammatical relationships among terms, whether detected sentiment about a term is positive or negative, etc.),” LI et al., para [0268]. Here, the sentiment strength is considered a “score”.);
determine a background score by analyzing the audio stream to detect background noise and comparing the detected background noise to known background noise contained in the known digital audio representations (“Performing preprocessing operations of the first pause detection technique may include deriving an audio noise level based on at least one measure of a level of audio noise of the speech audio; in response to the request, the at least one processor may be caused to perform preprocessing operations of a second speaker diarization technique to identify a second set of likely speaker changes,” LI et al., para [0016]. Here, deriving an audio noise level involves comparing the audio to a known speaker. And, audio noise level is considered a “background score”.);
determine a content score using the transcript by comparing the transcript to transcripts produced for the known digital audio representations (“During speech-to-text processing, the derived probability distributions associated with the identification of more likely graphemes (e.g., text characters representing phonemes) and/or pauses by an acoustic model, as well as the probability distributions associated with the identification of more likely n-grams by a language model, are used in identifying the sentences spoken in the speech audio to generate a corresponding transcript. During text analytics post-processing, the corresponding transcript is analyzed to select words that are pertinent to identifying topics or sentiments about topics, and/or analyzed along with other transcripts to identify relationships between different pieces of speech audio,” LI et al., para [0059]. And, “Within the control device 2500, in executing the control routine 2570, various post-processing analyses may be performed of the text within the transcript to identify such features as the one or more topics that were spoken about, the relative importance of each topic, indications of sentiments, etc. More specifically, using the transcript of the text data set 3700 as an input, one or more terms within the transcript (each including one or more words) may be identified as having one or more quantifiable characteristics (e.g., counts of occurrences of each term and/or aggregate counts of multiple terms, degree of relevance of a term within the transcript, degree of strength of positive or negative sentiment about a term, etc.), and/or relational characteristics (e.g., semantic and/or grammatical relationships among terms, whether detected sentiment about a term is positive or negative, etc.),” LI et al., para [0268]. The counts, degree of relevance, etc. are quantifiable terms and may be considered “scores”. It is further noted that “content score” has not been given any description in the claim language and may thus be broadly interpreted, including being given to mean the same as “emotional score”.).
LI et al., though, does not disclose determining if the audio stream is malicious by combining scores to produce a combined score and comparing the combined score to a threshold; and determining if the audio stream is malicious by combining the scores to produce a combined score and comparing the combined score to a threshold.
Trivedi et al. is cited to disclose determining if the audio stream is malicious by combining the timing score, emotional score, background score, and content score to produce a combined score and comparing the combined score to a threshold (“At step 212, the process 200 may include determining a status for the incoming call based on one or more of the first, second, and/or third fraud indicator data, wherein the status is at least one of fraudulent or confirmed. For example, the server device 104 may provide the first fraud indicator data determined based on the number of the incoming call at step 206, the second fraud indicator data determined based on the correspondence of the user account interaction data and the entity and/or interaction at step 208, and/or the third fraud indicator data based on the content and/or voice characteristic identified from the voice data received as output from the trained machine learning system at step 210 to a model for processing to determine the status, as described elsewhere herein. In some examples, each of the first, second, and third fraud indicator data may be used in the status determination,” Trivedi et al., para [0053]. And, “In some examples, a combined value (e.g., of first and second values) indicating a likelihood of the incoming call being fraudulent based on both the content of the voice data and the identified voice characteristics may be output as part of the third fraud indicator data 710. In further examples, the combined value may be a weighted aggregation of the first value and the second value. As a non-limiting example, the first value may be weighted or scaled to contribute more to the combined value given that the presence of certain words or phrases may be more reliable as a fraud indicator than voice characteristics. As described elsewhere herein, the third fraud indicator data 710 may include the first value, second value, and/or combined value as a data point. In other examples, the third fraud indicator data 710 may include one or more labeled voice content and characteristics data points of “true” or “false” based on the first value, second value, and/or combined value,” Trivedi et al., para [0088].); and
notifying a user when the combined score is greater than the threshold (“At step 214, the process 200 may include generating a notification that indicates the status for display on the communication device (e.g., user device 102). For example, the notification may include text, images, animated graphics and/or the like that indicate the status of the incoming call (e.g., fraudulent, confirmed, unknown or undetermined). The notification may be provided over the network 106 to the user device 102 for display. In some examples, the notification may be provided as a push notification through the application 113 for display. In other examples, the notification may be provided as a text message to the user device 102,” Trivedi et al., para [0054].). Trivedi et al. benefits LI et al. by combining multiple measures for determining a malicious audio stream, thereby improving the likelihood of accurately detecting fraud. Therefore, it would be obvious for one skilled in the art to combine the teachings of LI et al. with those of Trivedi et al. to improve the fraud detection capabilities of LI et al.
As to claim 11, method claim 11 and system claim 1 are related as system and method of using same, with each claimed element’s function corresponding to the system step. Accordingly claim 11 is similarly rejected under the same rationale as applied above with respect to system claim.
As to claim 17, CRM claim 17 and system claim 1 are related as system and CRM of using same, with each claimed element’s function corresponding to the system step. Accordingly claim 17 is similarly rejected under the same rationale as applied above with respect to system claim.
Regarding claim 2, Trivedi et al., as modified by LI et al., discloses the system of claim 1, wherein the audio stream is received from the external device and comprises real-time audio (“The status determination and corresponding notification generation may occur in real-time or near-real time as data of the incoming call, including voice data, is received and processed by one or more trained machine learning systems and/or models,” Trivedi et al., para [0023]. The incoming call is received from an external device, as shown in Trivedi et al., fig. 1.).
Regarding claim 3, Trivedi et al., as modified by LI et al., discloses the system of claim 2, wherein the external device is a mobile phone (Trivedi et al., fig. 1(118) and para [0036], show incoming calls being placed from an external mobile phone.), and the audio stream is an unexpected call received by the user (“In some examples, a combined value (e.g., of first and second values) indicating a likelihood of the incoming call being fraudulent based on both the content of the voice data and the identified voice characteristics may be output as part of the third fraud indicator data 710,” Trivedi et al., para [0088]. Also, the fact that the incoming call received by a user is potentially fraudulent implies that the call is unexpected.).
Regarding claim 4, Trivedi et al., as modified by LI et., discloses the system of claim 1, wherein the combined score is a weighted score comprising predetermined weights for each of the timing score, emotional score, background score, and content score and wherein the predetermined weights are determined by analyzing the known digital audio representations using machine learning (“The execution of the machine learning system may include deployment of one or more machine learning techniques, such as linear regression, logistical regression, random forest, gradient boosted machine (GBM), deep learning, and/or a deep neural network,” Trivedi et al., para [0029]. And, Trivedi et al., para [0088]. See also Trivedi et al., fig. 9.).
As to claim 12, method claim 12 and system claim 4 are related as system and method of using same, with each claimed element’s function corresponding to the system step. Accordingly claim 12 is similarly rejected under the same rationale as applied above with respect to system claim.
As to claim 18, CRM claim 18 and system claim 4 are related as system and CRM of using same, with each claimed element’s function corresponding to the system step. Accordingly claim 18 is similarly rejected under the same rationale as applied above with respect to system claim.
Regarding claim 5, LI et al., as modified by Trivedi et al., discloses the system of claim 4, wherein the machine learning utilizes logistic regression to determine a weight to apply to each of the timing score, emotional score, background score, and content score (“The execution of the machine learning system may include deployment of one or more machine learning techniques, such as linear regression, logistical regression, random forest, gradient boosted machine (GBM), deep learning, and/or a deep neural network Trivedi et al., para [0029].).
As to claim 13, method claim 13 and system claim 5 are related as system and method of using same, with each claimed element’s function corresponding to the system step. Accordingly claim 13 is similarly rejected under the same rationale as applied above with respect to system claim.
Regarding claim 7, LI et al., as modified by Trivedi et al., discloses the system of claim 1, wherein the timing score is further determined by identifying a speaker in the portion of the audio stream and comparing the portion of the audio stream to known recordings of the speaker that is similar to the portion of the audio stream (“FIGS. 19A, 19B, 19C and 19D, taken together, illustrate an example of use of a speaker diarization technique based on the use of a speaker diarization neural network 2237 as part of performing pre-processing operations to derive a manner of dividing the same speech audio of the same speech data set 3100 into segments. FIG. 19A illustrates the initial division of the speech data set 3100 into data chunks 3110d that each represent a chunk of the speech audio of the speech data set 3100, and the provision of those data chunks 3110d as an input to a speaker diarization neural network 2237, and the use of that speaker diarization neural network 2237 to generate speaker vectors that are each indicative of characteristics of a speaker who speaks in the speech audio. FIGS. 19B-C, taken together, illustrate aspects of the use of the speaker vectors as points in a performance of clustering within a multi-dimensional space to identify speakers. FIG. 19D illustrates the matching of speaker identities to speaker vectors to identify likely speaker changes for inclusion in a change set 3118 of indications of likely speaker changes within the speech audio of the speech data set 3100,” LI et al., para [0299].).
As to claim 20, CRM claim 20 and system claim 7 are related as system and CRM of using same, with each claimed element’s function corresponding to the system step. Accordingly claim 20 is similarly rejected under the same rationale as applied above with respect to system claim.
Regarding claim 9, LI et al., as modified by Trivedi et al., discloses the system of claim 1, wherein the timing score, emotional score, background score, and content score are determined using machine learning to analyze the portion of the audio stream and the transcript (Li et al., fig. 11.).
As to claim 16, method claim 16 and system claim 9 are related as system and method of using same, with each claimed element’s function corresponding to the system step. Accordingly claim 16 is similarly rejected under the same rationale as applied above with respect to system claim.
Claim(s) 6, 14, and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over US 20220335949, hereinafter referred to as LI et al., in view of US 20230262160, hereinafter referred to as Trivedi et al., and further in view of US 20240040035, hereinafter referred to as Dropuljic et al.
Regarding claim 6, LI et al., as modified by Trivedi et al., discloses the system of claim 1, but not wherein the timing score, emotional score, background score, and content score are indications of a probability that the portion of the audio stream was produced electronically. Dropuljic et al. is cited to disclose wherein the timing score, emotional score, background score, and content score are indications of a probability that the portion of the audio stream was produced electronically (“In various embodiments, the output logic module 210 may generate the output label based on the probability of individual outputs (from text categories, audio categories, or audio characteristics) and historical data, or a combination of these outputs and historical data. For example, if the call is discussing vehicle-related information (e.g., a textual category), but historical data indicates that a synthetic voice caller (e.g., audio category or audio characteristics) statistically increases the likelihood of that call being fraudulent, then the system can modify (by raising or lowering) the certainty level of fraud output depending on whether the call is a synthetic voice or not,” Dropuljic et al., para [0051]. Dropulic et al. notes that audio characteristics are used to determine if the audio stream was produced electronically.). Dropuljic et al. benefits LI et al. by identifying an incoming call as synthetic voice, thereby allowing the system to raise the certainty level of fraud. Therefore, it would be obvious for one skilled in the art to combine the teachings of LI et al. with those of Dropulijic et al. to improve the fraud detection capabilities of LI et al.
As to claim 14, method claim 14 and system claim 6 are related as system and method of using same, with each claimed element’s function corresponding to the system step. Accordingly claim 14 is similarly rejected under the same rationale as applied above with respect to system claim.
As to claim 19, CRM claim 19 and system claim 6 are related as system and CRM of using same, with each claimed element’s function corresponding to the system step. Accordingly claim 19 is similarly rejected under the same rationale as applied above with respect to system claim.
Claim(s) 8 and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over US 20220335949, hereinafter referred to as LI et al., in view of US 20230262160, hereinafter referred to as Trivedi et al., and further in view of US 20250054495, hereinafter referred to as Carbune et al.
Regarding claim 8, LI et al., as modified by Trivedi et al., discloses the system of claim 1, but not wherein the background score is determined by removing speech in the portion of the audio stream, wherein the speech is removed using the transcript to identify the speech. Carbune et al. is cited to disclose wherein the background score is determined by removing speech in the portion of the audio stream, wherein the speech is removed using the transcript to identify the speech (“For example, the transcript and/or other speech recognition hypothesis can include metadata that indicates a context of a spoken utterance (e.g., whether a train was heard in the background), and the score processing engine 218 can indicate a degree of confidence that the background noise has been accurately described by the metadata,” Carbune et al., para [0036].). Carbune et al. benefits LI et al. by using a speech transcript to identify the environment of a caller, thereby helping to determine the likelihood that the call is fraudulent. Therefore, it would be obvious for one skilled in the art to combine the teachings of LI et al. with those of Carbune et al. to improve the fraud detection capabilities of LI et al.
As to claim 15, method claim 15 and system claim 8 are related as system and method of using same, with each claimed element’s function corresponding to the system step. Accordingly claim 15 is similarly rejected under the same rationale as applied above with respect to system claim.
Claim(s) 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over US 20220335949, hereinafter referred to as LI et al., in view of US 20230262160, hereinafter referred to as Trivedi et al., and further in view of US 20230013385, hereinafter referred to as Kitagishi et al.
Regarding claim 10, LI et al., as modified by Trivedi et al., discloses the system of claim 1, wherein the background noise includes saliva noises and the background score is determined at least in part based on a frequency of the saliva noises (“On the other hand, adults other than elderly people, who have sufficient ability to swallow, can swallow saliva appropriately, and produce such water sounds less frequently than elderly people. Thus, if the occurrence frequency of the water sounds can be grasped, the age levels can be estimated accurately for elderly people,” Kitagishi et al., para [0008].). Kitagishi et al. benefits LI et al. by using background noise characteristics, such as saliva sounds, to provide insight into the caller’s age and the likelihood of a fraudulent call. Therefore, it would be obvious for one skilled in the art to combine the teachings of LI et al. with those of Kitagishi et al. to improve the fraud detection capabilities of LI et al.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See attached PTO-892. In particular, the examiner notes Khoury, Altaf, Citzer et al., Rohatgi et al., and Terrell et al. as generally addressing methods for deep fake detection similar to those describe in the claims of the present application.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANNE L THOMAS-HOMESCU whose telephone number is (571)272-0899. The examiner can normally be reached Mon-Fri 8-6.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Bhavesh M Mehta can be reached on 5712727453. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANNE L THOMAS-HOMESCU/Primary Examiner, Art Unit 2656