DETAILED ACTION
This office action is in response to Applicant’s submission filed on 3/19/2025. Claims 1-20 are pending in the application. As such, claims 1-20 have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 3/19/2025. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 5, 8, 11, 15, 18, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Altaf et al. (US 20240355337 A1; hereinafter referred to as Altaf) in view of Tak et al. (Tak, Hemlata, et al. "End-to-end anti-spoofing with rawnet2." ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2021.; hereinafter referred to as Tak).
Regarding claim 1, Altaf discloses: an automatic speaker verification-based system with detection of a voice cloning attack, comprising: a speech receiving device for inputting a raw speech signal ([0160] At step 410, the computer may identify, select, or otherwise detect at least one speech region within the set of speech segments of the inbound audio data);
a voice cloning attack detection circuitry configured to detect the voice cloning attack based on the raw speech signal inputted at the speech receiving device ([0222] Using the acoustic features extracted from the call audio, the computing system may detect presentation attacks, such as replayed audio speech (e.g., selectively replaying recorded speech) or synthetic speech (e.g., deepfakes, TTS software));
an authentication component for authenticating a user ([0064] systems and computer-implemented methods for authenticating users based on speech of audio signals. The computer may obtain, responsive to a first authentication of a user using authentication credentials, an audio speech signal of the user for a second authentication. The computer may extract a plurality of features from the audio speech signal. The computer may generate using the plurality of features, a liveness score indicating a likelihood that the user is a human speaker) based on the raw speech signal inputted at the speech receiving device ([0241] the feature extractor 905 may detect replayed pre-recorded human speech or synthetic (“deepfake”) speech. The feature extractor 905 may take raw audio waveform as input and outputs a liveness score. The spoof embedding extractor 910 may take in the audio features or the raw audio waveform to extract the spoofprint embedding);
and a backend operation server that performs an operation based on the authentication of the user… ([0318] The computer may perform the final authentication to permit the client access to the resource. At step 1540, if the liveness score does not satisfy the threshold, the computer may terminate the final authentication of the caller. When the caller is classified as a machine, the computer may determine that the final authentication is unsuccessful and may terminate the multi-factor authentication process. The computer may perform the final authentication to restrict the client access from the resource)
Altaf does not explicitly, but Tak teaches: wherein the voice cloning attack detection circuitry includes a deep learning framework ([Section 1, paragraph 5] This paper reports what is, to the best of our knowledge, the first application of RawNet2 [7] deep neural network classifiers to anti-spoofing) having: a convolutional block configured to extract features from the speech signal ([2, par. 2] The first convolutional layer is applied directly to the raw speech waveform, with all filter parameters being learned automatically. Among the higher layers are residual blocks [10] which extract frame-level representations. The convolutional layer extracts speech features.), a plurality of interconnected residual blocks ([2, par. 4] The first layer of RawNet2 is essentially the same as that of SincNet, whereas the upper layers consist of the same residual blocks and GRU layer as RawNet1), wherein a negative slope in fixed sinc filters in the residual blocks is increased to prevent dead neurons ([3, Table 1] The negative slope in a leaky ReLU is used in the sinc filter and has an increased value compared to the 0 slope of a standard ReLU.), and a gated recurrent unit (GRU) configured to aggregate a frame-level representation from a last of the residual blocks into an utterance-level representation ([2, par. 2] They use either long short-term memory (LSTM) as in [11] or gated recurrent units (GRUs) as in [9] to aggregate utterance-level representations and either a b-vector classifier [12, 13] as in [11] or a DNN backend with concatenation and multiplication (concat & mul) technique as in [9] for verification).
Altaf and Tak are considered analogous in the field of speech processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Altaf to combine the teachings of Tak because doing so would allow for the use of a RawNet2 architecture to be used for detecting spoofed voice, leading to improved classification of real and fake voice by incorporating SincNet filters and residual blocks to process input speech (Tak [Abstract] This paper reports the first application of RawNet2 to anti-spoofing. RawNet2 ingests raw audio and has potential to learn cues that are not detectable using more traditional countermeasure solutions. We describe modifications made to the original RawNet2 architecture so that it can be applied to anti-spoofing. For A17 attacks, our RawNet2 systems results are the second-best reported, while the fusion of RawNet2 and baseline counter measures gives the second-best results reported for the full ASVspoof 2019 logical access condition).
Regarding claim 5, the combination of Altaf and Tak teaches: the system of claim 1. Altaf further teaches: wherein the speech receiving device is a mobile device ([0063] hardware or software components that performs liveness detection for multi-factor authentication for callers or end-users. For instance, a system may employ voice liveness detection as a second-factor of authenticating a user's device or user-side mobile application), wherein the backend operation is access to phone operations ([0072] When performing the second authentication, the computer may perform the second authentication to permit the user access to a resource, responsive to the liveness score satisfying a threshold).
Regarding claim 8, the combination of Altaf and Tak teaches: the system of claim 1. Altaf further teaches: wherein the voice cloning attack detection circuitry detects that the speech signal is generated by speech synthesis ([0279] the model upgrader 1325 may accept feedback indicating new types of presentation attacks from the agent 1340, to adapt to new situations and speech synthesis techniques. The model upgrader 1325 may use the feedback from the agent 1340 to update the models in the speech repetition detector 1305, the background change detector 1310, the passive liveness detector 1315, or the aggregator 1320, among others).
Regarding claim 11, the claim recites similar limitations as claim 1 and therefore is rejected similarly.
Regarding claim 15, the claim recites similar limitations as claim 5 and therefore is rejected similarly.
Regarding claim 18, the claim recites similar limitations as claim 8 and therefore is rejected similarly.
Regarding claim 20, Altaf discloses: a non-transitory computer-readable storage medium including computer executable instructions, wherein the instructions, when executed by a computer, cause… ([0327] the functions may be stored as one or more instructions or code on a non-transitory computer-readable or processor-readable storage medium. The steps of a method or algorithm disclosed herein may be embodied in a processor-executable software module which may reside on a computer-readable or processor-readable storage medium). The rest of the claim recites similar limitations as claim 1 and therefore is rejected similarly.
Claims 2 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Altaf in view of Tak as applied to claims 1, 5, 8, 11, 15, 18, and 20 above, and further in view of Wang et al. (US 20240242705 A1; hereinafter referred to as Wang).
Regarding claim 2, the combination of Altaf and Tak teaches: the system of claim 1. The combination of Altaf and Tak does not explicitly, but Wang teaches: wherein the voice cloning attack detection circuitry includes a transpose convolution layer in the residual blocks for preserving fine-grained temporal information ([0050] generator of HiFi-GAN may be divided into two main modules: a transposed convolution (ConvTranspose) network and a multi-receptive field fusion (MRF) module. Specifically, the Mel spectrogram may first be upsampled by the transposed convolution network, aiming to align the length of output features with the temporal resolution of an original waveform. The upsampled features can then be input to the MRF module consisting of a plurality of residual blocks).
Altaf, Tak, and Wang are considered analogous in the field of speech processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Altaf and Tak to combine the teachings of Wang because doing so would allow for an improved method of voice cloning using a transpose convolution layer, leading to better detection of different methods of voice cloning to more accurately determine a voice cloning attack (Wang [0018] conventional speech cloning techniques are usually based only on reference speech information and given text information. To generate more accurate and realistic speech data, the present disclosure provides a speech generation technique across multiple modalities. For example, multi-modal information of three modalities, i.e., two-dimensional reference image information (i.e., reference video information), audio information containing reference speech, and given text information, can be used to generate speech data that is more similar to speech of a real person who provides that reference speech).
Regarding claim 12, the claim recites similar limitations as claim 2 and therefore is rejected similarly.
Claims 3 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Altaf in view of Tak as applied to claims 1, 5, 8, 11, 15, 18, and 20 above, and further in view of Bharaj et al. (US 20250245296 A1; hereinafter referred to as Bharaj).
Regarding claim 3, the combination of Altaf and Tak teaches: the system of claim 1. The combination of Altaf and Tak does not explicitly, but Bharaj teaches: wherein the voice cloning attack detection circuitry includes a PreLU activation function in each of the residual blocks ([0060] the type of neural network may include or combine one or more convolutional layers, one or more residual layers and one or more dense or fully connected layers. Moreover, a given node or filter in a given layer in the type of neural network may include an activation function, such as: a rectified linear activation function (ReLU), a leaky ReLU, an exponential linear unit (ELU) activation function, a parametric ReLU).
Altaf, Tak, and Bharaj are considered analogous in the field of speech processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Altaf and Tak to combine the teachings of Bharaj because doing so would allow for a deep learning framework to utilize a parametric ReLU activation function to adaptively learn the optimal negative slope, boosting accuracy in detecting real or fake voice when training a deep neural network using large datasets (Bharaj [0059] computer system 100 may automatically train the artificial neural network to more accurately assess whether audio content is real or fake. Notably, the computational techniques may use cross-attention to aid representation learning. Consequently, the trained artificial neural network may have improved EERs, even for smaller training datasets and/or training datasets that are corrupted by codecs and transmission channel artifacts. Therefore, the trained artificial neural network may improve the user experience when using the trained artificial neural network to assess whether audio content is real or fake. This may significantly reduce the cost and complexity of the compute environment in computer system 100 and/or electronic devices 126).
Regarding claim 13, the claim recites similar limitations as claim 3 and therefore is rejected similarly.
Claims 4 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Altaf in view of Tak as applied to claims 1, 5, 8, 11, 15, 18, and 20 above, and further in view of Xue et al. (Xue, Junxiao, and Hao Zhou. "Physiological-physical feature fusion for automatic voice spoofing detection." Frontiers of Computer Science 17.2 (2023): 172318.; hereinafter referred to as Xue).
Regarding claim 4, the combination of Altaf and Tak teaches: the system of claim 1. Tak further teaches: wherein the voice cloning attack detection circuitry includes a LogSoftmax activation function in a final layer of the deep learning framework for recognizing real and fake audio… ([3, par. 4] A softmax activation function is then applied in order to produce two-class predictions: bona-fide or spoof).
The combination of Altaf and Tak does not explicitly, but Xue teaches: and wherein, the LogSoftmax activation function transforms raw model outputs into logarithmic probabilities of real or fake audio data ([III. D.] The goal of the ASV spoof challenge is to calculate the countermeasure (cm) score for each input audio file. High cm scores represent bonafide speech, while low cm scores represent spoofing attacks. Calculated from the softmax output using the log-likelihood ratio).
Altaf, Tak, and Xue are considered analogous in the field of speech processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Altaf and Tak to combine the teachings of Xue because doing so would allow for the use of a Log Softmax function to output probabilities of real and fake audio, leading to more stable numbers and better efficiency in calculating probabilities in comparison to a regular Softmax function (Xue [III. D.] By multiplying the output of the global average pooling layer with the output of the fully connected layer, the SE layer output is obtained. The output of the global average pooling layer is sent to the full connection layer. Finally, the distribution of two kinds of labels is generated through the softmax layer).
Regarding claim 14, the claim recites similar limitations as claim 4 and therefore is rejected similarly.
Claims 6 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Altaf in view of Tak as applied to claims 1, 5, 8, 11, 15, 18, and 20 above, and further in view of McNutty et al. (US 20240361827 A1; hereinafter referred to as McNutty).
Regarding claim 6, the combination of Altaf and Tak teaches: the system of claim 1. The combination of Altaf and Tak does not explicitly, but McNutty teaches: wherein the speech receiving device is a smart speaker having voice assistant circuitry ([0503] Voice cloning can be used with the IOENGINE protocol. Voice cloning is a process that involves creating a synthetic or cloned voice that imitates the speech patterns and vocal characteristics of an individual. It utilizes IOENGINE technology by combining an application in a portable device to facilitate voice assistants, virtual agents, and other applications where human-like speech is desired and combines with an artificial intelligence responsive system on the backend), wherein the backend operation is a financial operation that requires secure operation ([1793] A voice-activated payment assistant, like a virtual banking assistant, uses voice cloning technology to authenticate users before allowing them to perform tasks like checking account balances, making transfers, or paying bills, ensuring secure access to financial information).
Altaf, Tak, and McNutty are considered analogous in the field of speech processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Altaf and Tak to combine the teachings of McNutty because doing so would allow for further use of a voice cloning detection system to authorize or deny access to sensitive information based on a determination of fake or real voice, leading to improved user flexibility and security in actions involving personal financial information (McNutty [1594] By leveraging voice cloning technology, verification systems can be designed to recognize individual users based on their unique vocal characteristics. This can help ensure that verification processes are not only more secure but also more personalized, providing users with a sense of ownership and control over their digital identities. For instance, a financial institution could use voice cloning technology to create a custom voiceprint for a user, which can then be used to verify their identity during phone banking transactions or other interactions).
Regarding claim 16, the claim recites similar limitations as claim 6 and therefore is rejected similarly.
Claims 7, 10, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Altaf in view of Tak as applied to claims 1, 5, 8, 11, 15, 18, and 20 above, and further in view of Mohapatra (US 20210174807 A1).
Regarding claim 7, the combination of Altaf and Tak teaches: the system of claim 1. The combination of Altaf and Tak does not explicitly, but Mohapatra teaches: wherein the speech receiving device is a smart speaker having voice assistant circuitry ([0026] example devices included in the system 10 is an example primary display device, and in the embodiment shown is an audio video display device (AVDD) 12 such as but not limited to an Internet-enabled TV. Thus, the AVDD 12 alternatively may be an appliance or household item, e.g. computerized Internet enabled refrigerator, washer, or dryer. The AVDD 12 alternatively may also be a computerized Internet enabled (“smart”) telephone), wherein the backend operation is control of a household appliance ([0027] The AVDD 12 may include one or more speakers 16 for outputting audio in accordance with present principles, and at least one additional input device 18 such as a keyboard or keypad or an audio receiver/microphone for e.g. entering audible commands to the AVDD 12 to control the AVDD 12).
Altaf, Tak, and Mohapatra are considered analogous in the field of speech processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Altaf and Tak to combine the teachings of Mohapatra because doing so would allow for the use of a secure automatic voice verification system to interact with household appliances based on a determination of a fake or real voice, improving user flexibility in interacting with appliances in different settings (Mohapatra [0047] step at block 214 and in other blocks may be executed by a server communicating with the sender and recipient devices and having access to the sender's voice model or by the recipient device, but more preferably for security reasons is executed locally by the sender device. In this way the sender's voice model is not disseminated onto a network, further reducing the risk of a data breach that could expose the sender's voice model to unauthorized use by hackers).
Regarding claim 10, the combination of Altaf, Tak, and Mohapatra teaches: the system of claim 7. Mohapatra further teaches: wherein the speech receiving device inputs a control command as the raw speech signal to request control of the household appliance ([0027] The AVDD 12 may include one or more speakers 16 for outputting audio in accordance with present principles, and at least one additional input device 18 such as a keyboard or keypad or an audio receiver/microphone for e.g. entering audible commands to the AVDD 12 to control the AVDD 12).
Regarding claim 17, the claim recites similar limitations as claim 7 and therefore is rejected similarly.
Claims 9 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Altaf in view of Tak as applied to claims 1, 5, 8, 11, 15, 18, and 20 above, and further in view of Jung et al. (US 20260087313 A1; hereinafter referred to as Jung).
Regarding claim 9, the combination of Altaf and Tak teaches: the system of claim 1. The combination of Altaf and Tak does not explicitly, but Jung teaches: wherein the voice cloning attack detection circuitry detects that the speech signal is generated by voice conversion ([0044] A vocal tract feature captures a characteristic related to a shape of the vocal cords and detects unnatural formant change or inconsistence of a voice quality which may occur during the voice conversion).
Altaf, Tak, and Jung are considered analogous in the field of speech processing. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Altaf and Tak to combine the teachings of Jung because doing so would allow for a voice cloning attack detection system to analyze different speech features and determine real or fake voice based on these extracted features, leading to an improved voice cloning detection system that can detect different kinds of voice cloning methods (Jung [0060] The neural vocoder detection model 146 outputs a neural vocoder probability which is a probability of generating a voice included in the audio data from the sound feature through the neural vocoder. The neural vocoder detection model 146 is a CNN model and is trained using labeled learning data including a sound feature of an original voice and a sound feature of a voice generated through the neural vocoder. That is, learning data labeled with the sound feature of an original voice and a sound feature of a voice generated through the neural vocoder is collected and the neural vocoder detection model 146 is trained with the collected learning data).
Regarding claim 19, the claim recites similar limitations as claim 9 and therefore is rejected similarly.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Havdan et al. (US 20230206925 A1) – discloses a system and method for classifying genuine or spoofed voice samples using a deep learning neural network including residual blocks and a gated recurrent layer.
Wei et al. (CN 116524907 A) – discloses a speaker identification method using a multi-scale SincNet and conditional GAN to identify speakers using a small number of training sentences.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Nathan Tengbumroong whose telephone number is (703)756-1725. The examiner can normally be reached Monday - Friday, 11:30 am - 8:00 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hai Phan can be reached at 571-272-6338. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NATHAN TENGBUMROONG/Examiner, Art Unit 2654
/HAI PHAN/Supervisory Patent Examiner, Art Unit 2654
