DETAILED ACTION
This action is in response to the application filed on April 21st, 2025. Claims 1-10 are pending and have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement filed April 21st, 2025 fails to comply with 37 CFR 1.98(a)(3)(i) because it does not include a concise explanation of the relevance, as it is presently understood by the individual designated in 37 CFR 1.56(c) most knowledgeable about the content of the information, of each reference listed that is not in the English language. It has been placed in the application file, but the information referred to therein has not been considered.
Claim Objections
Claim 3 objected to because of the following informalities: the claim reads "the vector is passed through a fully connection layer" but should read "the vector is passed through a fully Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 5-7, and 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Li, Chang-tao (CN Pat. No. 114898773 A hereinafter Li), in view of Zeghidour et al. (US Pat. Pub. No. 2023/0019128 A1 hereinafter Zeghidour).
Regarding claim 1, Li discloses a method for classifying generated speech, comprising: applying a one-dimensional convolution operation to raw speech data to embed the raw speech data into a feature space and extract a feature vector (Li, page 3: "step 3) through one-dimensional convolution neural network the time-frequency characteristic of the to-be-discriminant voice signal is model, so as to reduce the time resolution, and the obtained voice feature is input to the depth self-attention neural network classifier;"); and applying the result to a classifier model comprising a natural language processing model to output a classification label (Li, page 3: "step 4) identifying the input low time resolution speech feature by depth self-attention neural network so as to determine whether the speech signal to be judged is synthesized speech."). However, Li fails to expressly recite quantizing the feature vector by applying the feature vector to a residual vector quantizer.
Zeghidour teaches quantizing the feature vector by applying the feature vector to a residual vector quantizer (Zeghidour, [0032]: "The feature vectors 208 are then processed by a residual (e.g., multi-stage) vector quantizer RVQ 106 to generate coded representations of the feature vectors (CFVs) 210 and corresponding quantized representations of the feature vectors (QFVs) 212. ").
Li and Zeghidour are analogous arts because they both belong to the same field of audio processing. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the synthetic speech detection method of Li to incorporate the teachings of Zeghidour to quantize the feature vector. This allows the system to align the bitrate of the audio feature vectors before they are processed further (Zeghidour, [0032]). As such, the system is able to supply more consistent inputs to the classification model, which improves the consistency of the system’s final outputs.
Regarding claim 5, the rejection of claim 1 is incorporated. Li, in view of Zeghidour, discloses all of the elements of the current invention as stated above. Zeghidour further teaches wherein the residual vector quantizer is configured to quantize the feature vector differently according to a length of the raw speech data (Zeghidour, [0028]-[0029]: "Referring to FIG. 1, the compression system 100 receives an audio waveform 112 to be compressed. The waveform 112 can include an audio sample at each time step, where the time step usually corresponds to a particular sampling rate. Higher sampling rates capture higher frequency components of the audio waveform 112. For example, the standard audio sampling rate used by professional digital equipment is 48 kHz since it can reconstruct sound at frequencies up to 24 kHz (e.g., upper limit of human hearing). Although such sampling rates are ideal for comfortable listening, the compression system 100 can generally be configured to process waveforms 112 at any sampling rate, even waveforms with non-uniform sampling. The audio waveform 112 can originate from any suitable audio source. For example, the waveform 112 can be a recording from an external audio device (e.g., speech from a microphone), a purely digital production (e.g., electronic music), or generic audio such as sound effects and background noise (e.g., white noise, room tone)."). The same motivation for claim 1 applies equally to claim 5.
Regarding claim 6, the rejection of claim 1 is incorporated. Li, in view of Zeghidour, discloses all of the elements of the current invention as stated above. Zeghidour further teaches a non-transitory computer-readable recording medium storing a program code for executing the method of claim 1 (Zeghidour, [0097]: "Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible non-transitory storage medium for execution by, or to control the operation of, data processing apparatus."). The same motivation for claim 1 applies equally to claim 6.
Regarding claim 7, Li discloses an apparatus for classifying generated speech, comprising; a feature extractor configured to apply a one-dimensional convolution operation to raw speech data to embed the raw speech data into a feature space and to extract a feature vector (Li, page 3: "step 3) through one-dimensional convolution neural network the time-frequency characteristic of the to-be-discriminant voice signal is model, so as to reduce the time resolution, and the obtained voice feature is input to the depth self-attention neural network classifier;"); and a classifier model configured to receive an output of the quantizer and to output a speech classification result with contextual awareness (Li, page 3: "step 4) identifying the input low time resolution speech feature by depth self-attention neural network so as to determine whether the speech signal to be judged is synthesized speech."). However, Li fails to expressly recite a quantizer disposed downstream of the feature extractor and configured to quantize the feature vector.
Zeghidour teaches a quantizer disposed downstream of the feature extractor and configured to quantize the feature vector (Zeghidour, [0032]: "The feature vectors 208 are then processed by a residual (e.g., multi-stage) vector quantizer RVQ 106 to generate coded representations of the feature vectors (CFVs) 210 and corresponding quantized representations of the feature vectors (QFVs) 212. ").
Li and Zeghidour are analogous arts because they both belong to the same field of audio processing. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the synthetic speech detection method of Li to incorporate the teachings of Zeghidour to quantize the feature vector. This allows the system to align the bitrate of the audio feature vectors before they are processed further (Zeghidour, [0032]). As such, the system is able to supply more consistent inputs to the classification model, which improves the consistency of the system’s final outputs.
Regarding claim 9, the rejection of claim 7 is incorporated. Li, in view of Zeghidour, discloses all of the elements of the current invention as stated above. Zeghidour further teaches wherein the residual vector quantizer is configured to quantize the feature vector differently according to a length of the raw speech data (Zeghidour, [0028]-[0029]: " Referring to FIG. 1, the compression system 100 receives an audio waveform 112 to be compressed. The waveform 112 can include an audio sample at each time step, where the time step usually corresponds to a particular sampling rate. Higher sampling rates capture higher frequency components of the audio waveform 112. For example, the standard audio sampling rate used by professional digital equipment is 48 kHz since it can reconstruct sound at frequencies up to 24 kHz (e.g., upper limit of human hearing). Although such sampling rates are ideal for comfortable listening, the compression system 100 can generally be configured to process waveforms 112 at any sampling rate, even waveforms with non-uniform sampling. The audio waveform 112 can originate from any suitable audio source. For example, the waveform 112 can be a recording from an external audio device (e.g., speech from a microphone), a purely digital production (e.g., electronic music), or generic audio such as sound effects and background noise (e.g., white noise, room tone)."). The same motivation for claim 7 applies equally to claim 9.
Claim(s) 2 is/are rejected under 35 U.S.C. 103 as being unpatentable over Li, in view of Zeghidour, as applied to claims 1, 5-7, and 9 above, and further in view of Almutairi et al. (Z. M. Almutairi and H. Elgibreen, "Detecting Fake Audio of Arabic Speakers Using Self-Supervised Deep Learning," in IEEE Access, vol. 11, pp. 72134-72147, 2023, doi: 10.1109/ACCESS.2023.3286864. hereinafter Almutairi).
Regarding claim 2, the rejection of claim 1 is incorporated. Li, in view of Zeghidour, discloses all of the elements of the current invention as stated above. However, Li, in view of Zeghidour, fails to expressly recite wherein the natural language processing model comprises a BERT (Bidirectional Encoder Representations from Transformers) language model, and wherein the classifier model is configured to output the classification label indicating one of generated speech and real speech based on an output of the BERT language model.
Almutairi teaches wherein the natural language processing model comprises a BERT (Bidirectional Encoder Representations from Transformers) language model (Almutairi, page 8, left column: "we utilize the pre-trained BASE HuBERT model (facebook/hubert-basels960), which was trained on unlabeled speech with ∼95M parameters. The main reason for choosing the BASE version of HuBERT is due to its computational limitations; where it needed the fewest resources compared to the other versions."), and wherein the classifier model is configured to output the classification label indicating one of generated speech and real speech based on an output of the BERT language model (Almutairi, page 8, right column, second to last paragraph: "The main goal of our method is to classify the given waveform audio X and detect if it is fake or real.").
Li, Zeghidour, and Almutairi are analogous arts because they each belong to the same field of audio processing. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the synthetic speech detection method of Li, as modified by the vector quantizers of Zeghidour, to incorporate the teachings of Almutairi to use a BERT language model as the classifier model. BERT models are able to extract useful information for speech recognition tasks (Almutairi, page 7, paragraph 3). As such the model is able to accurately and efficiently classify audio.
Claim(s) 3 is/are rejected under 35 U.S.C. 103 as being unpatentable over Li, in view of Zeghidour and Almutairi, as applied to claim 2 above, and further in view of Rosenberg et al. (US Pat. Pub. No. 2024/0153484 A1 hereinafter Rosenberg) and Tian et al. (CN Pat. No. 116705063 A hereinafter Tian).
Regarding claim 3, the rejection of claim 2 is incorporated. Li, in view of Zeghidour and Almutairi, discloses all of the elements of the current invention as stated above. However, Li, in view of Zeghidour and Almutairi, fails to expressly recite wherein the quantized result is represented as a vector reflecting overall contextual structure through the BERT language model, and the vector is passed through a fully connection layer and a softmax activation function of the classifier model to output the classification label indicating one of generated speech and real speech.
Rosenberg teaches wherein the quantized result is represented as a vector reflecting overall contextual structure through the BERT language model (Rosenberg, [0044]: "a quantizer 217 receives the encoded features 211, 213 as input, and generates quantized vectors (i.e., target context vectors) 219 as output.").
Li, Zeghidour, Almutairi, and Rosenberg are analogous arts because they each belong to the same field of audio processing. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the synthetic speech detection method of Li, as modified by the vector quantizers of Zeghidour and the fake audio detection system of Almutairi, to incorporate the teachings of Rosenberg to have the quantized vectors reflect overall contextual structure. This ensures that context is preserved during processing (Rosenberg, [0044]). This in turn allows the classification model to account for context during processing. However, Li, in view of Zeghidour, Almutairi, and Rosenberg, fails to expressly recite the vector is passed through a fully connection layer and a softmax activation function of the classifier model to output the classification label indicating one of generated speech and real speech.
Tian teaches the vector is passed through a fully connection layer and a softmax activation function of the classifier model to output the classification label indicating one of generated speech and real speech (Tian, page 3, lines 17-18: "the identification feature is sequentially input to the full connection layer and the softmax layer of the (256, C), and the identified counterfeit means label is output"; page 2, last paragraph: "In one embodiment, the at least two discrimination models specifically include two types: The invention uses LFCC characteristic as input, SE-Res2Net50 as pseudo-identification model of learning network, and uses original signal as input, pre-training model HuBERT as pseudo-identification model of learning network.").
Li, Zeghidour, Almutairi, Rosenberg, and Tian are analogous arts because they each belong to the same field of audio processing. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the synthetic speech detection method of Li, as modified by the vector quantizers of Zeghidour, the fake audio detection system of Almutairi, and the speech to text system of Rosenberg, to incorporate the teachings of Tian to include a fully connected layer and a softmax activation function. This helps to detect forged speech despite improvements in voice forging techniques (Tian, Page 2, Background). As such, the system is able to provide an improved output even with more challenging inputs.
Claim(s) 4 and 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Li, in view of Zeghidour, as applied to claims 1, 5-7, and 9 above, and further in view of Lee et al. (US Pat. Pub. No. 2007/0250308 A1 hereinafter Lee).
Regarding claim 4, the rejection of claim 1 is incorporated. Li, in view of Zeghidour, discloses all of the elements of the current invention as stated above. However, Li, in view of Zeghidour, fails to expressly recite wherein the residual vector quantizer is configured to quantize the feature vector, which is a one-dimension array of real values, into positive integer values.
Lee teaches wherein the residual vector quantizer is configured to quantize the feature vector, which is a one-dimension array of real values, into positive integer values (Lee, [0048]: "S.sub.q.sup.b is the vector of quantized spectral data in scalefactor band b, and takes on positive integer values from 0 to 8206.").
Li, Zeghidour, and Lee are analogous arts because they each belong to the same field of audio processing. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the synthetic speech detection method of Li, as modified by the vector quantizers of Zeghidour, to incorporate the teachings of Lee to quantize the feature vector into positive integer values. This allows the system to ensure the vectors are in the desired format and bitrate prior to further processing (Lee, [0002]). As such, the system is able to supply more consistent inputs to the classification model, which improves the consistency of the system’s final outputs.
Regarding claim 8, the rejection of claim 7 is incorporated. Li, in view of Zeghidour, discloses all of the elements of the current invention as stated above. Zeghidour further teaches wherein the quantizer is a residual vector quantizer (Zeghidour, [0032]: "The feature vectors 208 are then processed by a residual (e.g., multi-stage) vector quantizer RVQ 106 to generate coded representations of the feature vectors (CFVs) 210 and corresponding quantized representations of the feature vectors (QFVs) 212. "). The same motivation for claim 7 applies equally to claim 8. However, Li, in view of Zeghidour, fails to expressly recite wherein the quantizer is configured to quantize the feature vector, which is a one-dimension array of real values, into positive integer values.
Lee teaches wherein the quantizer is configured to quantize the feature vector, which is a one-dimension array of real values, into positive integer values (Lee, [0048]: "S.sub.q.sup.b is the vector of quantized spectral data in scalefactor band b, and takes on positive integer values from 0 to 8206.").
Li, Zeghidour, and Lee are analogous arts because they each belong to the same field of audio processing. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the synthetic speech detection method of Li, as modified by the vector quantizers of Zeghidour, to incorporate the teachings of Lee to quantize the feature vector into positive integer values. This allows the system to ensure the vectors are in the desired format and bitrate prior to further processing (Lee, [0002]). As such, the system is able to supply more consistent inputs to the classification model, which improves the consistency of the system’s final outputs.
Claim(s) 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Li, in view of Zeghidour, as applied to claims 1, 5-7, and 9 above, and further in view of Almutairi, Rosenberg, and Tian.
Regarding claim 10, the rejection of claim 7 is incorporated. Li, in view of Zeghidour, discloses all of the elements of the current invention as stated above. However, Li, in view of Zeghidour, fails to expressly recite wherein the classifier model comprises a natural language processing model including a BERT (Bidirectional Encoder Representations from Transformers) language model at a front end, wherein a fully connected layer and a softmax activation layer are disposed at a rear end of the BERT language model, and wherein the output of the quantizer is represented as a vector reflecting overall contextual structure through the BERT language model, and is passed through the fully connected layer and the softmax activation layer to output a classification label indicating one of generated speech and real speech.
Almutairi teaches wherein the classifier model comprises a natural language processing model including a BERT (Bidirectional Encoder Representations from Transformers) language model at a front end (Almutairi, page 8: "we utilize the pre-trained BASE HuBERT model (facebook/hubert-basels960), which was trained on unlabeled speech with ∼95M parameters. The main reason for choosing the BASE version of HuBERT is due to its computational limitations; where it needed the fewest resources compared to the other versions.")
Li, Zeghidour, and Almutairi are analogous arts because they each belong to the same field of audio processing. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the synthetic speech detection method of Li, as modified by the vector quantizers of Zeghidour, to incorporate the teachings of Almutairi to use a BERT language model as the classifier model. BERT models are able to extract useful information for speech recognition tasks (Almutairi, page 7, paragraph 3). As such the model is able to accurately and efficiently classify audio. However, Li, in view of Zeghidour and Almutairi, fails to expressly recite wherein a fully connected layer and a softmax activation layer are disposed at a rear end of the BERT language model, wherein the output of the quantizer is represented as a vector reflecting overall contextual structure through the BERT language model, and is passed through the fully connected layer and the softmax activation layer to output a classification label indicating one of generated speech and real speech.
Rosenberg teaches wherein the output of the quantizer is represented as a vector reflecting overall contextual structure through the BERT language model (Rosenberg, [0044]: "a quantizer 217 receives the encoded features 211, 213 as input, and generates quantized vectors (i.e., target context vectors) 219 as output.").
Li, Zeghidour, Almutairi, and Rosenberg are analogous arts because they each belong to the same field of audio processing. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the synthetic speech detection method of Li, as modified by the vector quantizers of Zeghidour and the fake audio detection system of Almutairi, to incorporate the teachings of Rosenberg to have the quantized vectors reflect overall contextual structure. This ensures that context is preserved during processing (Rosenberg, [0044]). This in turn allows the classification model to account for context during processing. However, Li, in view of Zeghidour, Almutairi, and Rosenberg, fails to expressly recite wherein a fully connected layer and a softmax activation layer are disposed at a rear end of the BERT language model, and the output is passed through the fully connected layer and the softmax activation layer to output a classification label indicating one of generated speech and real speech.
Tian teaches wherein a fully connected layer and a softmax activation layer are disposed at a rear end of the BERT language model, and the output is passed through the fully connected layer and the softmax activation layer to output a classification label indicating one of generated speech and real speech (Tian, page 3, lines 17-18: "the identification feature is sequentially input to the full connection layer and the softmax layer of the (256, C), and the identified counterfeit means label is output"; page 2, paragraph 6: "In one embodiment, the at least two discrimination models specifically include two types: The invention uses LFCC characteristic as input, SE-Res2Net50 as pseudo-identification model of learning network, and uses original signal as input, pre-training model HuBERT as pseudo-identification model of learning network.").
Li, Zeghidour, Almutairi, Rosenberg, and Tian are analogous arts because they each belong to the same field of audio processing. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the synthetic speech detection method of Li, as modified by the vector quantizers of Zeghidour, the fake audio detection system of Almutairi, and the speech to text system of Rosenberg, to incorporate the teachings of Tian to include a fully connected layer and a softmax activation function. This helps to detect forged speech despite improvements in voice forging techniques (Tian, Page 2, Background). As such, the system is able to provide an improved output even with more challenging inputs.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TYLER J BECKER whose telephone number is (703)756-1271. The examiner can normally be reached M-Th, 7:15am-5:45pm PT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Washburn can be reached at (571) 272-5551. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TYLER BECKER/ Examiner, Art Unit 2657
/DANIEL C WASHBURN/ Supervisory Patent Examiner, Art Unit 2657
