DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendment filed June 25th, 2026, has been entered. Claims 1-3, 6-10, 13-17, 19, and 20 have been amended. Claims 1-20 are pending and have been examined.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1, 8, and 15 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
While Applebaum is no longer being relied upon for the new ground of rejection, the motivation to combine Gopala and Lesso, as further detailed in the rejection below, is similar to the motivation to combine Gopala and Applebaum as presented in the previous rejection. As such, the examiner has included the following response to the applicant’s arguments against the motivation to combine Gopala and Applebaum.
In response to applicant's argument that Applebaum is nonanalogous art, it has been held that a prior art reference must either be in the field of the inventor’s endeavor or, if not, then be reasonably pertinent to the particular problem with which the inventor was concerned, in order to be relied upon as a basis for rejection of the claimed invention. See In re Oetiker, 977 F.2d 1443, 24 USPQ2d 1443 (Fed. Cir. 1992). In this case, Applebaum is analogous art because the technical problem of deepfake As stated by the applicant, speaker authentication pertains to “verifying whether a voice belongs to a particular individual” and deepfake detection involves “classification of audio as computer-generated versus organic human speech”. As such, the examiner would argue that computer-generated speech is not the voice belonging to a particular individual, and being able to distinguish between synthetic and organic voices is pertinent to speaker authentication to prevent attacks involving synthesized speech.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-5, 8-12, and 15-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gopala et al. (US Pat. Pub. No. 2021/0074305 A1 hereinafter Gopala), in view of Lesso, John Paul (US Pat. Pub. No. 2020/0082830 A1 hereinafter Lesso).
Regarding claim 1, Gopala discloses a method for detecting audio deepfakes through acoustic prosodic modeling, comprising: extracting, using a feature extraction technique to enable machine learning via a machine learning model that comprises multiple machine learning layers (Gopala, [0114]: “the audio analysis techniques may use one or more convolutional neural networks. A large convolutional neural network may include, e.g., 60 M parameters and 650,000 neurons. The convolutional neural network may include, e.g., eight learned layers with weights, including, e.g., five convolutional layers and three fully connected layers with a final 1000-way softmax or normalized exponential function that produces a distribution over the 1000 class labels. Some of the convolution layers may be followed by max-pooling layers.”), a prosodic feature set from an audio sample based on prosodic features indicative of one or more prosodic characteristics associated with human speech (Gopala, [0071]: " identification engine 124 may perform analysis and classification of the audio content associated with the video. Notably, identification engine 124 may determine a representation of the audio content by performing a transformation on the audio content."; [0071]: "other transformations and/or representations may be used, such as audio feature-extraction techniques, including: pitch detection, tonality, harmonicity, spectral centroid, pitch contour, prosody analysis (e.g., pauses, disfluences), syntax analysis, lexicography analysis, principal component analysis, or another feature extraction technique that determines a group of basis features, at least a subset of which allow discrimination of fake or real audio content."); and classifying the audio sample as a deepfake audio sample or an organic audio sample by applying the machine learning model to the prosodic feature set, wherein the machine learning model is configured as a classification-based detector for audio deepfakes (Gopala, [0073]: "identification engine 124 may classify, based at least in part on an output of the predetermined neural network, the audio content as being fake or real, where the fake audio content is, at least in part, computer-generated."). However, Gopala fails to expressly recite wherein the prosodic feature set comprises at least (i) a jitter feature indicative of vocal jitter associated with a frequency variation in the audio sample, and (ii) a shimmer feature indicative of vocal shimmer associated with amplitude variation in the audio sample; and classifying the audio sample as a deepfake audio sample or an organic audio sample by applying the machine learning model to at least (i) the jitter feature and (ii) the shimmer feature of the prosodic feature set, wherein the machine learning model is configured as a classification-based detector for audio deepfakes.
Lesso teaches wherein the prosodic feature set comprises at least (i) a jitter feature indicative of vocal jitter associated with a frequency variation in the audio sample, and (ii) a shimmer feature indicative of vocal shimmer associated with amplitude variation in the audio sample (Lesso, [0100]: “the at least one alternative feature of the fundamental frequency of the speech of the enrolled speaker may comprise at least one of: the jitter of the speech, i.e. the variability or perturbation of the fundamental frequency; the shimmer of the speech, i.e. the perturbation of the amplitude of the sound”); and classifying the audio sample as a deepfake audio sample or an organic audio sample by applying the machine learning model to at least (i) the jitter feature and (ii) the shimmer feature of the prosodic feature set, wherein the machine learning model is configured as a classification-based detector for audio deepfakes (Lesso, Fig. 10; [0116]: “FIG. 10 is a block diagram, illustrating a system for performing this method of detection of a replay attack, also referred to as spoof detection. The system shown in FIG. 10 is the same as that shown in FIG. 3, and the description of FIG. 3 also applies to FIG. 10, except as specified below.”; [0099]: “As mentioned above, more than one feature relating to the fundamental frequency may be obtained in block 82 of the system of FIG. 3. However, one or more alternative biometric feature may also be obtained, in addition to the cumulative distribution functions of the fundamental frequency of the speech. Specifically, at least one alternative feature of the fundamental frequency of the speech of the enrolled speaker may be obtained.”; [0117]: “As will be apparent from the description of FIG. 3, during enrollment, a biometric is formed for an enrolled speaker by: obtaining a sample of speech of the enrolled speaker; obtaining a measure of a fundamental frequency of the speech of the enrolled speaker in each of a plurality of speech frames; and forming a first distribution function of the fundamental frequency of the speech of the enrolled speaker.”; [0121]-[0124]: “when new input speech is received, a new second distribution function of the fundamental frequency of the input speech is formed from a measure of the fundamental frequency in each of a plurality of speech frames. The comparison block 84 can then compare the new second distribution function with the original first distribution function that was generated from the speech provided at enrollment, and with all of the additional first distribution functions generated from the enrolled user's subsequent speech. The method of comparison between the new second distribution function and each of the multiple first distribution functions may comprise calculating a value of a statistical distance between some or all of the second distribution function and a corresponding part of the respective first distribution function, for example as described with reference to FIG. 6, or may use a machine learning technique, for example as described with reference to FIG. 7. If the new second distribution function is considered to be sufficiently similar to any one of the multiple first distribution functions, that is, with such a degree of similarity that it is identical or nearly identical, then it may be determined that the new input speech is not live speech, but is a recording of a previous utterance of the enrolled user.”).
Gopala and Lesso are analogous arts because they both belong to the same field of speaker authentication. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the fake audio content identification method of Gopala to incorporate the teachings of Lesso to use jitter and shimmer as features when detecting a synthetic voice. Since human voice typically displays significant amounts of jitter and shimmer but synthetic speech typically has very little random jitter and shimmer (Lesso, [0154]), it can be used to differentiate between synthetic and organic speech. As such, using jitter and shimmer as features can improve the system’s ability to differentiate between synthetic and organic speech.
Regarding claim 2, the rejection of claim 1 is incorporated. Gopala, in view of Lesso, discloses all of the elements of the current invention as stated above. Gopala further discloses wherein the classifying the audio sample comprises: identifying the audio sample as the deepfake audio sample in response to the prosodic feature set failing to correspond to a predefined organic audio classification measure as determined by the machine learning model (Gopala, [0073]: "in some embodiments ‘classification’ may involve the use of a threshold (such as a value between 0 and 1, e.g., 0.5) that the output of the predetermined neural network is compared to in order to decide whether given audio content associated with a video is real or fake. Note that the audio content may be allegedly associated with the given individual and the threshold may correspond to the given individual"; [0078]: "the classification may be performed using a classifier or a regression model that was trained using a supervised learning technique (such as a support vector machine, a classification and regression tree, logistic regression, LASSO, linear regression and/or another linear or nonlinear supervised-learning technique) and a training dataset with additional (real and/or synthetic) audio content.").
Regarding claim 3, the rejection of claim 1 is incorporated. Gopala, in view of Lesso, discloses all of the elements of the current invention as stated above. Gopala further discloses wherein the prosodic feature set comprises a pitch feature, an intonation feature, a fundamental frequency feature, a rhythm feature, a stress feature, a harmonic-to-noise ratio feature, or one or more metrics features related to the audio sample (Gopala, [0071]: "other transformations and/or representations may be used, such as audio feature-extraction techniques, including: pitch detection, tonality, harmonicity, spectral centroid, pitch contour, prosody analysis (e.g., pauses, disfluences), syntax analysis, lexicography analysis, principal component analysis, or another feature extraction technique that determines a group of basis features, at least a subset of which allow discrimination of fake or real audio content.").
Regarding claim 4, the rejection of claim 1 is incorporated. Gopala, in view of Lesso, discloses all of the elements of the current invention as stated above. Gopala further discloses wherein the machine learning model is a neural network model (Gopala, [0072]: "identification engine 124 may analyze the representation using a predetermined neural network (such as a convolutional neural network, a recurrent neural network, one or more multi-layer perceptrons, a combination of the neural networks, or, more generally, a neural network that is trained to discriminate between fake and real audio content).").
Regarding claim 5, the rejection of claim 1 is incorporated. Gopala, in view of Lesso, discloses all of the elements of the current invention as stated above. Gopala further discloses wherein the machine learning model is a multilayer perceptron (MLP) model (Gopala, [0072]: "identification engine 124 may analyze the representation using a predetermined neural network (such as a convolutional neural network, a recurrent neural network, one or more multi-layer perceptrons, a combination of the neural networks, or, more generally, a neural network that is trained to discriminate between fake and real audio content).").
Regarding claim 8, Gopala discloses an apparatus for detecting audio deepfakes through acoustic prosodic modeling, the apparatus comprising at least one processor and at least one memory including program code, the at least one memory and the program code configured to, with the at least one processor, cause the apparatus to at least (Gopala, [0005]: “This computer system may include: a computation device (such as a processor); and memory that stores program instructions that are executed by the computation device.”): extract, using a feature extraction technique to enable machine learning via a machine learning model that comprises multiple machine learning layers (Gopala, [0114]: “the audio analysis techniques may use one or more convolutional neural networks. A large convolutional neural network may include, e.g., 60 M parameters and 650,000 neurons. The convolutional neural network may include, e.g., eight learned layers with weights, including, e.g., five convolutional layers and three fully connected layers with a final 1000-way softmax or normalized exponential function that produces a distribution over the 1000 class labels. Some of the convolution layers may be followed by max-pooling layers.”), a prosodic feature set from an audio sample based on prosodic characteristics associated with human speech (Gopala, [0071]: " identification engine 124 may perform analysis and classification of the audio content associated with the video. Notably, identification engine 124 may determine a representation of the audio content by performing a transformation on the audio content."; [0071]: "other transformations and/or representations may be used, such as audio feature-extraction techniques, including: pitch detection, tonality, harmonicity, spectral centroid, pitch contour, prosody analysis (e.g., pauses, disfluences), syntax analysis, lexicography analysis, principal component analysis, or another feature extraction technique that determines a group of basis features, at least a subset of which allow discrimination of fake or real audio content."); and classify the audio sample as a deepfake audio sample or an organic audio sample by applying the machine learning model to the prosodic feature set, wherein the machine learning model is configured as a classification-based detector for audio deepfakes (Gopala, [0073]: "identification engine 124 may classify, based at least in part on an output of the predetermined neural network, the audio content as being fake or real, where the fake audio content is, at least in part, computer-generated."). However, Gopala fails to expressly recite wherein the prosodic feature set comprises at least (i) a jitter feature indicative of vocal jitter associated with a frequency variation in the audio sample, and ii) a shimmer feature indicative of vocal shimmer associated with amplitude variation in the audio sample; and classify the audio sample as a deepfake audio sample or an organic audio sample by applying the machine learning model to at least (i) the jitter feature and (ii) the shimmer feature of the prosodic feature set, wherein the machine learning model is configured as a classification-based detector for audio deepfakes.
Lesso teaches wherein the prosodic feature set comprises at least (i) a jitter feature indicative of vocal jitter associated with a frequency variation in the audio sample, and ii) a shimmer feature indicative of vocal shimmer associated with amplitude variation in the audio sample (Lesso, [0100]: “the at least one alternative feature of the fundamental frequency of the speech of the enrolled speaker may comprise at least one of: the jitter of the speech, i.e. the variability or perturbation of the fundamental frequency; the shimmer of the speech, i.e. the perturbation of the amplitude of the sound”); and classify the audio sample as a deepfake audio sample or an organic audio sample by applying the machine learning model to at least (i) the jitter feature and (ii) the shimmer feature of the prosodic feature set, wherein the machine learning model is configured as a classification-based detector for audio deepfakes (Lesso, Fig. 10; [0116]: “FIG. 10 is a block diagram, illustrating a system for performing this method of detection of a replay attack, also referred to as spoof detection. The system shown in FIG. 10 is the same as that shown in FIG. 3, and the description of FIG. 3 also applies to FIG. 10, except as specified below.”; [0099]: “As mentioned above, more than one feature relating to the fundamental frequency may be obtained in block 82 of the system of FIG. 3. However, one or more alternative biometric feature may also be obtained, in addition to the cumulative distribution functions of the fundamental frequency of the speech. Specifically, at least one alternative feature of the fundamental frequency of the speech of the enrolled speaker may be obtained.”; [0117]: “As will be apparent from the description of FIG. 3, during enrollment, a biometric is formed for an enrolled speaker by: obtaining a sample of speech of the enrolled speaker; obtaining a measure of a fundamental frequency of the speech of the enrolled speaker in each of a plurality of speech frames; and forming a first distribution function of the fundamental frequency of the speech of the enrolled speaker.”; [0121]-[0124]: “when new input speech is received, a new second distribution function of the fundamental frequency of the input speech is formed from a measure of the fundamental frequency in each of a plurality of speech frames. The comparison block 84 can then compare the new second distribution function with the original first distribution function that was generated from the speech provided at enrollment, and with all of the additional first distribution functions generated from the enrolled user's subsequent speech. The method of comparison between the new second distribution function and each of the multiple first distribution functions may comprise calculating a value of a statistical distance between some or all of the second distribution function and a corresponding part of the respective first distribution function, for example as described with reference to FIG. 6, or may use a machine learning technique, for example as described with reference to FIG. 7. If the new second distribution function is considered to be sufficiently similar to any one of the multiple first distribution functions, that is, with such a degree of similarity that it is identical or nearly identical, then it may be determined that the new input speech is not live speech, but is a recording of a previous utterance of the enrolled user.”).
Gopala and Lesso are analogous arts because they both belong to the same field of speaker authentication. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the fake audio content identification method of Gopala to incorporate the teachings of Lesso to use jitter and shimmer as features when detecting a synthetic voice. Since human voice typically displays significant amounts of jitter and shimmer but synthetic speech typically has very little random jitter and shimmer (Lesso, [0154]), it can be used to differentiate between synthetic and organic speech. As such, using jitter and shimmer as features can improve the system’s ability to differentiate between synthetic and organic speech.
Regarding claim 9, the rejection of claim 8 is incorporated. Gopala, in view of Lesso, discloses all of the elements of the current invention as stated above. Gopala further discloses wherein the at least one memory and the program code are configured to, with the at least one processor, further cause the apparatus to at least: identify the audio sample as the deepfake audio sample in response to the prosodic feature set failing to correspond to a predefined organic audio classification measure as determined by the machine learning model (Gopala, [0073]: "in some embodiments ‘classification’ may involve the use of a threshold (such as a value between 0 and 1, e.g., 0.5) that the output of the predetermined neural network is compared to in order to decide whether given audio content associated with a video is real or fake. Note that the audio content may be allegedly associated with the given individual and the threshold may correspond to the given individual"; [0078]: "the classification may be performed using a classifier or a regression model that was trained using a supervised learning technique (such as a support vector machine, a classification and regression tree, logistic regression, LASSO, linear regression and/or another linear or nonlinear supervised-learning technique) and a training dataset with additional (real and/or synthetic) audio content.").
Regarding claim 10, the rejection of claim 8 is incorporated. Gopala, in view of Lesso, discloses all of the elements of the current invention as stated above. Gopala further discloses wherein the prosodic feature set comprises a pitch feature, an intonation feature, a fundamental frequency feature, a rhythm feature, a stress feature, a harmonic-to-noise ratio feature, or one or more metrics features related to the audio sample (Gopala, [0071]: "other transformations and/or representations may be used, such as audio feature-extraction techniques, including: pitch detection, tonality, harmonicity, spectral centroid, pitch contour, prosody analysis (e.g., pauses, disfluences), syntax analysis, lexicography analysis, principal component analysis, or another feature extraction technique that determines a group of basis features, at least a subset of which allow discrimination of fake or real audio content.").
Regarding claim 11, the rejection of claim 8 is incorporated. Gopala, in view of Lesso, discloses all of the elements of the current invention as stated above. Gopala further discloses wherein the machine learning model is a neural network model (Gopala, [0072]: "identification engine 124 may analyze the representation using a predetermined neural network (such as a convolutional neural network, a recurrent neural network, one or more multi-layer perceptrons, a combination of the neural networks, or, more generally, a neural network that is trained to discriminate between fake and real audio content).").
Regarding claim 12, the rejection of claim 8 is incorporated. Gopala, in view of Lesso, discloses all of the elements of the current invention as stated above. Gopala further discloses wherein the machine learning model is a multilayer perceptron (MLP) model (Gopala, [0072]: "identification engine 124 may analyze the representation using a predetermined neural network (such as a convolutional neural network, a recurrent neural network, one or more multi-layer perceptrons, a combination of the neural networks, or, more generally, a neural network that is trained to discriminate between fake and real audio content).").
Regarding claim 15, Gopala discloses a non-transitory computer storage medium comprising instructions for detecting audio deepfakes through acoustic prosodic modeling, the instructions being configured to cause one or more processors to at least perform operations configured to (Gopala, [0005]: “This computer system may include: a computation device (such as a processor); and memory that stores program instructions that are executed by the computation device.”): extract, using a feature extraction technique to enable machine learning via a machine learning model that comprises multiple machine learning layers (Gopala, [0114]: “the audio analysis techniques may use one or more convolutional neural networks. A large convolutional neural network may include, e.g., 60 M parameters and 650,000 neurons. The convolutional neural network may include, e.g., eight learned layers with weights, including, e.g., five convolutional layers and three fully connected layers with a final 1000-way softmax or normalized exponential function that produces a distribution over the 1000 class labels. Some of the convolution layers may be followed by max-pooling layers.”), a prosodic feature set from an audio sample based on prosodic characteristics associated with human speech (Gopala, [0071]: " identification engine 124 may perform analysis and classification of the audio content associated with the video. Notably, identification engine 124 may determine a representation of the audio content by performing a transformation on the audio content."; [0071]: "other transformations and/or representations may be used, such as audio feature-extraction techniques, including: pitch detection, tonality, harmonicity, spectral centroid, pitch contour, prosody analysis (e.g., pauses, disfluences), syntax analysis, lexicography analysis, principal component analysis, or another feature extraction technique that determines a group of basis features, at least a subset of which allow discrimination of fake or real audio content."); and classify the audio sample as a deepfake audio sample or an organic audio sample by applying the machine learning model to the prosodic feature set, wherein the machine learning model is configured as a classification-based detector for audio deepfakes (Gopala, [0073]: "identification engine 124 may classify, based at least in part on an output of the predetermined neural network, the audio content as being fake or real, where the fake audio content is, at least in part, computer-generated."). However, Gopala fails to expressly recite wherein the prosodic feature set comprises at least (i) a jitter feature indicative of vocal jitter associated with a frequency variation in the audio sample, and (ii) a shimmer feature indicative of vocal shimmer associated with amplitude variation in the audio sample; and classify the audio sample as a deepfake audio sample or an organic audio sample by applying the machine learning model to at least (i) the jitter feature and (ii) the shimmer feature of the prosodic feature set, wherein the machine learning model is configured as a classification-based detector for audio deepfakes.
Lesso teaches wherein the prosodic feature set comprises at least (i) a jitter feature indicative of vocal jitter associated with a frequency variation in the audio sample, and (ii) a shimmer feature indicative of vocal shimmer associated with amplitude variation in the audio sample (Lesso, [0100]: “the at least one alternative feature of the fundamental frequency of the speech of the enrolled speaker may comprise at least one of: the jitter of the speech, i.e. the variability or perturbation of the fundamental frequency; the shimmer of the speech, i.e. the perturbation of the amplitude of the sound”); and classify the audio sample as a deepfake audio sample or an organic audio sample by applying the machine learning model to at least (i) the jitter feature and (ii) the shimmer feature of the prosodic feature set, wherein the machine learning model is configured as a classification-based detector for audio deepfakes (Lesso, Fig. 10; [0116]: “FIG. 10 is a block diagram, illustrating a system for performing this method of detection of a replay attack, also referred to as spoof detection. The system shown in FIG. 10 is the same as that shown in FIG. 3, and the description of FIG. 3 also applies to FIG. 10, except as specified below.”; [0099]: “As mentioned above, more than one feature relating to the fundamental frequency may be obtained in block 82 of the system of FIG. 3. However, one or more alternative biometric feature may also be obtained, in addition to the cumulative distribution functions of the fundamental frequency of the speech. Specifically, at least one alternative feature of the fundamental frequency of the speech of the enrolled speaker may be obtained.”; [0117]: “As will be apparent from the description of FIG. 3, during enrollment, a biometric is formed for an enrolled speaker by: obtaining a sample of speech of the enrolled speaker; obtaining a measure of a fundamental frequency of the speech of the enrolled speaker in each of a plurality of speech frames; and forming a first distribution function of the fundamental frequency of the speech of the enrolled speaker.”; [0121]-[0124]: “when new input speech is received, a new second distribution function of the fundamental frequency of the input speech is formed from a measure of the fundamental frequency in each of a plurality of speech frames. The comparison block 84 can then compare the new second distribution function with the original first distribution function that was generated from the speech provided at enrollment, and with all of the additional first distribution functions generated from the enrolled user's subsequent speech. The method of comparison between the new second distribution function and each of the multiple first distribution functions may comprise calculating a value of a statistical distance between some or all of the second distribution function and a corresponding part of the respective first distribution function, for example as described with reference to FIG. 6, or may use a machine learning technique, for example as described with reference to FIG. 7. If the new second distribution function is considered to be sufficiently similar to any one of the multiple first distribution functions, that is, with such a degree of similarity that it is identical or nearly identical, then it may be determined that the new input speech is not live speech, but is a recording of a previous utterance of the enrolled user.”).
Gopala and Lesso are analogous arts because they both belong to the same field of speaker authentication. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the fake audio content identification method of Gopala to incorporate the teachings of Lesso to use jitter and shimmer as features when detecting a synthetic voice. Since human voice typically displays significant amounts of jitter and shimmer but synthetic speech typically has very little random jitter and shimmer (Lesso, [0154]), it can be used to differentiate between synthetic and organic speech. As such, using jitter and shimmer as features can improve the system’s ability to differentiate between synthetic and organic speech.
Regarding claim 16, the rejection of claim 15 is incorporated. Gopala, in view of Lesso, discloses all of the elements of the current invention as stated above. Gopala further discloses wherein the operations are further configured to: identify the audio sample as the deepfake audio sample in response to the prosodic feature set failing to correspond to a predefined organic audio classification measure as determined by the machine learning model (Gopala, [0073]: "in some embodiments ‘classification’ may involve the use of a threshold (such as a value between 0 and 1, e.g., 0.5) that the output of the predetermined neural network is compared to in order to decide whether given audio content associated with a video is real or fake. Note that the audio content may be allegedly associated with the given individual and the threshold may correspond to the given individual"; [0078]: "the classification may be performed using a classifier or a regression model that was trained using a supervised learning technique (such as a support vector machine, a classification and regression tree, logistic regression, LASSO, linear regression and/or another linear or nonlinear supervised-learning technique) and a training dataset with additional (real and/or synthetic) audio content.").
Regarding 17, the rejection of claim 15 is incorporated. Gopala, in view of Lesso, discloses all of the elements of the current invention as stated above. Gopala further discloses wherein the prosodic feature set comprises a pitch feature, an intonation feature, a fundamental frequency feature, a shimmer feature, a rhythm feature, a stress feature, a harmonic-to-noise ratio feature, or one or more metrics features related to the audio sample (Gopala, [0071]: "other transformations and/or representations may be used, such as audio feature-extraction techniques, including: pitch detection, tonality, harmonicity, spectral centroid, pitch contour, prosody analysis (e.g., pauses, disfluences), syntax analysis, lexicography analysis, principal component analysis, or another feature extraction technique that determines a group of basis features, at least a subset of which allow discrimination of fake or real audio content.").
Regarding claim 18, the rejection of claim 15 is incorporated. Gopala, in view of Lesso, discloses all of the elements of the current invention as stated above. Gopala further discloses wherein the machine learning model is a multilayer perceptron (MLP) model (Gopala, [0072]: "identification engine 124 may analyze the representation using a predetermined neural network (such as a convolutional neural network, a recurrent neural network, one or more multi-layer perceptrons, a combination of the neural networks, or, more generally, a neural network that is trained to discriminate between fake and real audio content).").
Claim(s) 6-7, 13-14, and 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gopala, in view of Lesso, as applied to claims 1-5, 8-12, and 15-18 above, and further in view of Wang et al. (US Pat. Pub. No. 2024/0005947 A1 hereinafter Wang).
Regarding claim 6, the rejection of claim 1 is incorporated. Gopala, in view of Lesso, discloses all of the elements of the current invention as stated above. However, Gopala, in view of Lesso, fails to expressly recite scaling the prosodic feature set for processing by the machine learning model.
Wang teaches scaling the prosodic feature set for processing by the machine learning model (Wang, [0045]: "a plurality of extracted acoustic features from the speech data 304 is passed through one or more DNNs 306."; [0051]: "the pooling/gradient reversal layers 308 are configured to perform attention pooling that gives each of a plurality of feature vectors a weight and generates an average vector, wherein the weighting determines the corresponding accuracy.").
Gopala, Lesso, and Wang are analogous arts because they all belong to the same field of synthetic audio detection. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the fake audio content identification method of Gopala, as modified by the speaker recognition techniques of Lesso, to incorporate the teachings of Wang to scale the extracted prosodic features. This allows the features to be effectively averaged (Wang, [0051]). This allows the features to be combined without disproportionately affecting the result.
Regarding claim 7, the rejection of claim 1 is incorporated. Gopala, in view of Lesso, discloses all of the elements of the current invention as stated above. However, Gopala, in view of Lesso, fails to expressly recite applying one or more hidden layers of the machine learning model to the prosodic feature set.
Wang teaches applying one or more hidden layers of the machine learning model to the prosodic feature set (Wang, [0034]: "As such, the DNN 204 can include a bottom input layer 222(1) and a top layer 222(L) (integer L>1), as well as multiple hidden layers, such as the multiple layers 222(2)-222(3).").
Gopala, Lesso, and Wang are analogous arts because they all belong to the same field of synthetic audio detection. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the fake audio content identification method of Gopala, as modified by the speaker recognition techniques of Lesso, to incorporate the teachings of Wang to apply one or more hidden layers to the prosodic features. The hidden layers can help improve the results of a deep neural network (Wang, [0019]). This improves the performance of the overall system.
Regarding claim 13, the rejection of claim 8 is incorporated. Gopala, in view of Lesso, discloses all of the elements of the current invention as stated above. However, Gopala, in view of Lesso, fails to expressly recite wherein the at least one memory and the program code are configured to, with the at least one processor, further cause the apparatus to at least: scale the prosodic feature set for processing by the machine learning model.
Wang teaches wherein the at least one memory and the program code are configured to, with the at least one processor, further cause the apparatus to at least: scale the prosodic feature set for processing by the machine learning model (Wang, [0045]: "a plurality of extracted acoustic features from the speech data 304 is passed through one or more DNNs 306."; [0051]: "the pooling/gradient reversal layers 308 are configured to perform attention pooling that gives each of a plurality of feature vectors a weight and generates an average vector, wherein the weighting determines the corresponding accuracy.").
Gopala, Lesso, and Wang are analogous arts because they all belong to the same field of synthetic audio detection. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the fake audio content identification method of Gopala, as modified by the speaker recognition techniques of Lesso, to incorporate the teachings of Wang to scale the extracted prosodic features. This allows the features to be effectively averaged (Wang, [0051]). This allows the features to be combined without disproportionately affecting the result.
Regarding claim 14, the rejection of claim 8 is incorporated. Gopala, in view of Lesso, discloses all of the elements of the current invention as stated above. However, Gopala, in view of Lesso, fails to expressly recite wherein the at least one memory and the program code are configured to, with the at least one processor, further cause the apparatus to at least: apply one or more hidden layers of the machine learning model to the prosodic feature set.
Wang teaches wherein the at least one memory and the program code are configured to, with the at least one processor, further cause the apparatus to at least: apply one or more hidden layers of the machine learning model to the prosodic feature set (Wang, [0034]: "As such, the DNN 204 can include a bottom input layer 222(1) and a top layer 222(L) (integer L>1), as well as multiple hidden layers, such as the multiple layers 222(2)-222(3).").
Gopala, Lesso, and Wang are analogous arts because they all belong to the same field of synthetic audio detection. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the fake audio content identification method of Gopala, as modified by the speaker recognition techniques of Lesso, to incorporate the teachings of Wang to apply one or more hidden layers to the prosodic features. The hidden layers can help improve the results of a deep neural network (Wang, [0019]). This improves the performance of the overall system.
Regarding claim 19, the rejection of claim 15 is incorporated. Gopala, in view of Lesso, discloses all of the elements of the current invention as stated above. However, Gopala, in view of Lesso, fails to expressly recite wherein the operations are further configured to: scale the prosodic feature set for processing by the machine learning model.
Wang teaches wherein the operations are further configured to: scale the prosodic feature set for processing by the machine learning model (Wang, [0045]: "a plurality of extracted acoustic features from the speech data 304 is passed through one or more DNNs 306."; [0051]: "the pooling/gradient reversal layers 308 are configured to perform attention pooling that gives each of a plurality of feature vectors a weight and generates an average vector, wherein the weighting determines the corresponding accuracy.").
Gopala, Lesso, and Wang are analogous arts because they all belong to the same field of synthetic audio detection. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the fake audio content identification method of Gopala, as modified by the speaker recognition techniques of Lesso, to incorporate the teachings of Wang to scale the extracted prosodic features. This allows the features to be effectively averaged (Wang, [0051]). This allows the features to be combined without disproportionately affecting the result.
Regarding claim 20, the rejection of claim 15 is incorporated. Gopala, in view of Lesso, discloses all of the elements of the current invention as stated above. However, Gopala, in view of Lesso, fails to expressly recite wherein the operations are further configured to: apply one or more hidden layers of the machine learning model to the prosodic feature set.
Wang teaches wherein the operations are further configured to: apply one or more hidden layers of the machine learning model to the prosodic feature set (Wang, [0034]: "As such, the DNN 204 can include a bottom input layer 222(1) and a top layer 222(L) (integer L>1), as well as multiple hidden layers, such as the multiple layers 222(2)-222(3).").
Gopala, Lesso, and Wang are analogous arts because they all belong to the same field of synthetic audio detection. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the fake audio content identification method of Gopala, as modified by the speaker recognition techniques of Lesso, to incorporate the teachings of Wang to apply one or more hidden layers to the prosodic features. The hidden layers can help improve the results of a deep neural network (Wang, [0019]). This improves the performance of the overall system.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TYLER J BECKER whose telephone number is (703)756-1271. The examiner can normally be reached M-Th, 7:15am-5:45pm PT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Washburn can be reached at (571) 272-5551. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TYLER BECKER/ Examiner, Art Unit 2657
/DANIEL C WASHBURN/ Supervisory Patent Examiner, Art Unit 2657
Prosecution Insights