DETAILED ACTION
Claims 1,3-12,15-19,25 and 28 are pending
Claims 1, 19 and 25 are independent
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Examiner’s Remarks
Claims stating “and/or” examiner is interpreting as the limitation as being an “or” statement. Meaning, one or the other of the limitations in the claim can happen.
Response to Arguments
Applicant's arguments filed 03/17/2026 have been fully considered but they are not persuasive.
In regard to 35 USC 103, see Applicant’s remarks pgs. 11-14, Applicant argues “The Office Action refers to paragraph [0013] of Han as allegedly teaching line spectral frequencies (LSFs). However, this passage merely lists LSFs as one possible example among several alternative feature representations that may be used when computing feature vectors from an audio signal.” Examiner would like to point out that paragraph 0013 is cited in order for the applicant to understand that the feature vectors discussed in paragraph 0019 are the line spectral frequencies. Fig. 2 of Han shows that the features are processed through neural networks.
Applicant also argues that Han does not teach bandwidth extension. Examiner would like to point out that with the rejection being a 103 rejection, the prior art Nagel teaches the bandwidth extension (see Nagel Col. 9, paragraph 3) in combination of the features of Han.
Applicant argues that Nagel does not teach bandwidth extension as claimed in the application. Examiner would like to point out that, Col. 9, paragraph 3, lines 21-28 of Nagel states “Thus, the procedure illustrated in FIG. 11 has the advantage that the additionally transmitted selection side information 114 supports a decoder-side (phonem) classification in order to provide a decoder-side adaption of the SBR or BWE (bandwidth extension) parameters [configured to generate the speech output signal such that the speech output signal is bandwidth extended]. Thus, in contrast to FIG. 10, the FIG. 11 embodiment provides, in addition to the selection side information the legacy SBR side information.” Nagel explains that the decoder side allows for a bandwidth extension of the input.
Applicant lastly argues, that Khoury does not teach providing a pointer for separating envelope and excitation components of the speech signal. As cited below, paragraph 0049 of Khoury states “The training system 200A in FIG. 2A includes an input 210, an acoustic channel simulator (also referenced as a channel-compensation device or function) 220, a feed forward convolutional neural network (CNN) 230, a system analyzer 240 for extracting handcrafted features, and a loss function 250. A general overview of the elements of the training system 200A is provided here, followed by details of each element. The input 210 receives a speaker utterance, e.g., a pre-recorded audio signal or an audio signal received from a microphone [receive as input values of the first neural network a plurality of samples of a signal envelope of the narrowband speech input signal,]. The input device 210 may sample the audio signal to produce a recognition speech signal [receive a plurality of samples of an excitation signal of the speech input signal] 212. The recognition speech signal 212 is provided to both the acoustic channel simulator 220 and to the system analyzer 240. The acoustic channel simulator 220 processes the recognition speech signal 212 and provides to the CNN 230 a degraded speech signal 214.” Khoury explains that audio signal is input into a neural network to produce speech signals and analyze the signals, which then splits the excitation signal when processed.
Examiner would like to point out that Han, Nagel and Khoury are used in combination for the rejection. Therefore, the 35 USC 103 rejection is maintained.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or non-obviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 3-4, 12, 17-19, 25 and 28 are rejected under 35 U.S.C. 103 as being unpatentable over Khoury et al (US Published Patent Application No. 20210082439, "Khoury"), and in view of Nagel et al (US Patent No. 10657979, "Nagel") and Han et al (US Published Patent Application No. 20210005182, "Han").
In regard to claim 1 and analogous claims 19 and 25, Khoury teaches a signal envelope extrapolator comprising a first neural network, wherein the first neural network is configured to receive as input values of the first neural network a plurality of samples of a signal envelope of the narrowband speech input signal, and configured to determine as output values of the first neural network a plurality of extrapolated signal envelope samples; an excitation signal extrapolator configured to receive a plurality of samples of an excitation signal of the speech input signal, and (Khoury, paragraph 0049, “The training system 200A in FIG. 2A includes an input 210, an acoustic channel simulator (also referenced as a channel-compensation device or function) 220, a feed forward convolutional neural network (CNN) 230, a system analyzer 240 for extracting handcrafted features, and a loss function 250. A general overview of the elements of the training system 200A is provided here, followed by details of each element. The input 210 receives a speaker utterance, e.g., a pre-recorded audio signal or an audio signal received from a microphone [receive as input values of the first neural network a plurality of samples of a signal envelope of the narrowband speech input signal,]. The input device 210 may sample the audio signal to produce a recognition speech signal [receive a plurality of samples of an excitation signal of the speech input signal] 212. The recognition speech signal 212 is provided to both the acoustic channel simulator 220 and to the system analyzer 240. The acoustic channel simulator 220 processes the recognition speech signal 212 and provides to the CNN 230 a degraded speech signal 214. The CNN 230 is configured to provide features (coefficients) 232 corresponding to the recognition speech signal. In parallel, the signal analyzer 240 extracts handcrafted acoustic features 242 from the recognition speech signal [configured to determine as output values] 212. The loss function 250 utilizes both the features 232 from the CNN 230 and the handcrafted acoustic features 242 from the signal analyzer 240 to produce a loss result 252 and compare the loss result to a predetermined threshold.”)
configured to determine a plurality of extrapolated excitation signal samples; and (Khoury, paragraph 0059, “To achieve the first goal, the trained, channel-compensated CNN 230 takes as input the degraded speech signal described above and generates as output "clean" or channel-compensated features that matches handcrafted features extracted by signal analyzer 240 from a non-degraded recognition speech signal [determine a plurality of extrapolated excitation signal samples].”)
wherein the excitation signal extrapolator comprises a second neural network, wherein the second neural network is configured to receive as input values of the second neural network the plurality of samples of the excitation signal of the speech input signal, and/or is the speech input signal and/or, is a shaped version of the speech input signal, and (Khoury, paragraph 0072, “The second neural network [a second neural network] model 600 includes, in addition to the channel compensated feature generator 610 (such as systems 200A-200C detailed above), a convolutional neural network having an input layer 620, convolutional layers 630, and a max pooling layer 640 that outputs bottleneck features. For training, the second neural network model 600 may additionally include one or more fully connected layers 650 and an output layer 660. An input layer may be two-dimensional, having a first dimension corresponding to an audio sample length [speech input signal] ( e.g., 110 milliseconds) and a second dimension corresponding to the number of acoustic features (i.e. feature vectors) from the channel compensated feature generator 610 (e.g., CNN 230).”)
configured to determine as output values of the second neural network the plurality of extrapolated excitation signal samples. (Khoury, paragraph 0072, “For training, the second neural network [the second neural network] model 600 may additionally include one or more fully connected layers 650 and an output layer 660. An input layer may be two-dimensional, having a first dimension corresponding to an audio sample length ( e.g., 110 milliseconds) and a second dimension corresponding to the number of acoustic features (i.e. feature vectors) from the channel compensated feature generator 610 (e.g., CNN 230)... The output layer 660 may have 3622 output units, each output unit [output values] corresponding to a single particular speaker in training data. Naturally, the system may be scaled to accommodate a different number of speakers [plurality of extrapolated excitation signal samples].”)
However, Khoury does not explicitly teach a combiner configured to generate the speech output signal depending on the plurality of extrapolated signal samples and on the plurality of extrapolated excitation signal samples such that the speech output signal is bandwidth extended with respect to the speech input signal;
wherein the apparatus is configured for reproducing the speech output signal which is bandwidth extended compared to the speech input signal.
wherein the input values of the second neural network are a first plurality of time- domain signal samples of the excitation signal of the speech input signal, and/or is the speech input signal and/or, is a shaped version of the speech input signal, wherein the second neural network is configured to determine the output values of the second neural network such that the plurality of extrapolated excitation signal samples are a second plurality of time- domain signal samples of an extended time-domain excitation signal being bandwidth-extended with respect to the excitation signal of the speech input signal,
wherein the input values of the first neural network are a first plurality of line spectral frequencies of the speech input signal, and wherein the first neural network is configured to determine as the output values of the first neural network a second plurality of line spectral frequencies of the speech output signal;
wherein each of one or more of the second plurality of line spectral frequencies is associated with a frequency being greater than any frequency being associated with any of the first plurality of line spectral frequencies.
Nagel teaches a combiner configured to generate the speech output signal depending on the plurality of extrapolated signal samples and on the plurality of extrapolated excitation signal samples such that the speech output signal is bandwidth extended with respect to the speech input signal; (Nagel, Fig. 11, Col. 9, paragraph 3, lines 21-28, “Thus, the procedure illustrated in FIG. 11 has the advantage that the additionally transmitted selection side information 114 supports a decoder-side (phonem) classification in order to provide a decoder-side adaption of the SBR or BWE (bandwidth extension) parameters [configured to generate the speech output signal such that the speech output signal is bandwidth extended]. Thus, in contrast to FIG. 10, the FIG. 11 embodiment provides, in addition to the selection side information the legacy SBR side information.” and paragraph 4, “The encoded input signal [respect to the speech input signal] consists of subsequent frames 800, 806, 812. Each frame has the encoded core signal. Exemplarily, frame 800 has speech as the encoded core signal. Frame 806 has music as the encoded core signal and frame 812 again has speech as the encoded core signal. Frame 800 has, exemplarily, as the side information only the selection side information but no SBR side information.” And Col. 8, paragraph 6, “Furthermore, it is advantageous that the signal estimator 118 comprises an analysis filter 910, an excitation extension block 112 and a synthesis filter 940. Thus, blocks 910, 912, 914 may correspond to blocks 1600, 1700 and 1800 of FIG. 15. Particularly, the analysis filter 910 is an LPC analysis filter. The envelope estimation block 902 controls the filter coefficients of the analysis filter 910 so that the result of block 910 is the filter excitation signal. This filter excitation signal is extended with respect to frequency in order to 45 obtain an excitation signal at the output of block 912 which not only has the frequency range of the decoder 120 for an output signal but also has the frequency or spectral range not defined by the core coder and/or exceeding spectral range of the core signal. [plurality of extrapolated signal envelope samples and depending on the plurality of extrapolated excitation signal samples., examiner would like to point out that the signals are being extended exceeding the signal range, which is being interpreted as the extrapolated signal as when a signal is extrapolated, the signal is extended passed the known values.] ”)
wherein the apparatus is configured for reproducing the speech output signal which is bandwidth extended compared to the speech input signal. (Nagel, Fig. 11, Col. 9, paragraph 3, “Thus, the procedure illustrated in FIG. 11 has the advantage that the additionally transmitted selection side information 114 supports a decoder-side (phonem) classification in order to provide a decoder-side adaption of the SBR or BWE (bandwidth extension) parameters [bandwidth extended compared to the speech input signal.]. Thus, in contrast to FIG. 10, the FIG. 11 embodiment provides, in addition to the selection side information the legacy SBR side information.” and paragraph 4, “The encoded input signal consists of subsequent frames 800, 806, 812. Each frame has the encoded core signal. Exemplarily, frame 800 has speech as the encoded core signal [reproducing the speech output]. Frame 806 has music as the encoded core signal and frame 812 again has speech as the encoded core signal. Frame 800 has, exemplarily, as the side information only the selection side information but no SBR side information.”)
wherein the input values of the second neural network are a first plurality of time- domain signal samples of the excitation signal of the speech input signal, and/or is the speech input signal and/or, is a shaped version of the speech input signal, wherein the second neural network is configured to determine the output values of the second neural network such that the plurality of extrapolated excitation signal samples are a second plurality of time- domain signal samples of an extended time-domain excitation signal being bandwidth-extended with respect to the excitation signal of the speech input signal. (Nagel, Col. 6, paragraph 4, “Alternatively, however, the feature extractor [the second neural network] can also operate or extract a feature from the encoded core signal. Typically, the encoded core signal comprises a representation of scale factors for frequency bands or any other representation of audio information. Depending on the kind of feature extraction, the encoded representation of the audio signal [and/or is the speech input signal] is representative for the decoded core signal and, therefore features can be extracted. Alternatively or additionally, a feature can be extracted not only from a fully decoded core signal but also from a partly decoded core signal. In frequency domain coding, the encoded signal is representing a frequency domain representation comprising a sequence of spectral frames. The encoded core signal can, therefore, be only partly decoded to obtain a decoded representation of a sequence of spectral frames, before actually performing a spectrum-time conversion [configured to determine the output values of the second neural network such that the plurality of extrapolated excitation signal samples are a second plurality of time- domain signal samples]. Thus, the feature extractor 104 can extract features either from the encoded core signal or a partly decoded core signal or a fully decoded core signal. The feature extractor 104 can be implemented, with respect to its extracted features as known in the art and the feature extractor may, for example, be implemented as in audio fingerprinting or audio ID technologies”)
Khoury and Nagel are related to the same field of endeavor (i.e. signal recognition). In view of the teachings of Nagel, it would have been obvious for a person with ordinary skill in the art to apply the teachings of Nagel to Khoury before the effective filing date of the claimed invention in order to allow for frequencies to be enhanced from the audio signals. (Nagel, Col. 6, paragraph 1, “FIG. 1 illustrates a decoder for generating a frequency enhanced audio signal 120.”)
However, Khoury and Nagel do not teach wherein the input values of the first neural network are a first plurality of line spectral frequencies of the speech input signal, and wherein the first neural network is configured to determine as the output values of the first neural network a second plurality of line spectral frequencies of the speech output signal;
wherein each of one or more of the second plurality of line spectral frequencies is associated with a frequency being greater than any frequency being associated with any of the first plurality of line spectral frequencies.
Han teaches wherein the input values of the first neural network are a first plurality of line spectral frequencies of the speech input signal, and wherein the first neural network is configured to determine as the output values of the first neural network a second plurality of line spectral frequencies of the speech output signal; (Han, paragraph 0013, “The audio signal may be processed by feature computation component 110 to produce a sequence of feature vectors that represent the audio signal. Any appropriate feature vectors may be used, such as Mel-frequency cepstral coefficients, linear prediction coefficients, linear prediction cepstral coefficients, line spectral frequencies [a first plurality of line spectral frequencies of the speech input signal,], discrete wavelet transform coefficients, or perceptual linear prediction coefficients.” and paragraph 0019, “In FIG. 2, the input to system 200 may be a sequence of feature vectors and the output may be a sequence of speech unit score vectors. The processing may be performed using any number of streams, such as first stream 210, second stream 220, and third stream 230 [wherein the first neural network is configured to determine as the output values of the first neural network a second plurality of line spectral]. Each of the streams may process the sequence of feature vectors in parallel with each other. Each stream may process the feature vectors using a different dilation rate. For example, first stream 210 may process the feature vectors with a first dilation rate, second stream 220 may process the feature vectors with a second dilation rate, and third stream 230 may process the feature vectors with a third dilation rate.”)
wherein each of one or more of the second plurality of line spectral frequencies is associated with a frequency being greater than any frequency being associated with any of the first plurality of line spectral frequencies. (Han, paragraph 0026, “The different dilation rates of the streams may improve the performance of the acoustic model. Different aspects of speech may occur over different time frames, and the different dilation rates may allow the processing to better capture information at different time frames. Some aspects of speech may occur over short time frames, such as stop consonants, and smaller dilation rates may allow for improved processing of these aspects [second plurality of line spectral frequencies is associated with a frequency being greater than any frequency being associated with any of the first plurality of line spectral frequencies., examiner would like to point out that the smaller dilation rates allow for a greater frequency of speech signals]. Some aspects of speech may occur over longer time frames, such as diphthongs, and longer dilation rates may allow for improved processing of these aspects. The combination of different dilation rates may thus provide improved performance over acoustic models that don't include multiple streams with different dilation rates in different streams.”)
Khoury and Han are related to the same field of endeavor (i.e. signal recognition). In view of the teachings of Han, it would have been obvious for a person with ordinary skill in the art to apply the teachings of Han to Khoury before the effective filing date of the claimed invention in order to allow for more accurate results. (Han, paragraph 0088, “The proposed architecture seemingly helps the self-attention mechanism better process embeddings in each stream and lead to more accurate results overall.”)
In regard to claim 3, Khoury, Nagel and Han teach the apparatus of claim 1.
Nagel further teaches wherein, when the first neural network is trained, the signal envelope extrapolator is configured to transform a plurality of linear predictive coding coefficients, being derived from an original speech signal, into Finite impulse response filter coefficients by calculating an impulse response and by truncating the impulse response. (Nagel, Col. 8, paragraph 6, “Furthermore, it is advantageous that the signal estimator 118 comprises an analysis filter 910, an excitation extension block 112 and a synthesis filter 940. Thus, blocks 910, 912, 914 may correspond to blocks 1600, 1700 and 1800 of FIG. 40 15. Particularly, the analysis filter 910 is an LPC analysis filter. The envelope estimation block 902 controls the filter coefficients of the analysis filter 910 so that the result of block 910 is the filter excitation signal [transform a plurality of linear predictive coding coefficients, being derived from an original speech signal, into Finite impulse response filter coefficients]. This filter excitation signal is extended with respect to frequency in order to 45 obtain an excitation signal at the output of block 912 which not only has the frequency range of the decoder 120 for an output signal but also has the frequency or spectral range not defined by the core coder and/or exceeding spectral range of the core signal. Thus, the audio signal 909 at the output of 50 the decoder is upsampled and interpolated by an interpolator 900 and, then, the interpolated signal is subjected to the process in the signal estimator [impulse response and by truncating the impulse response] 118. Thus, the interpolator 900 in FIG. 9 may correspond to the interpolator 1500 of FIG. 15. Advantageously, however, in contrast to FIG. 15, 55 the feature extraction 104 is performed using the noninterpolated signal rather than on the interpolated signal as illustrated in FIG. 15.”)
Khoury and Nagel are combinable for the same rationale as set forth above with respect to claim 1.
In regard to claim 4, Khoury, Nagel and Han teach the apparatus of claim 3.
Nagel further teaches wherein, when the first neural network is trained, the signal envelope extrapolator is configured to feed back an error or a gradient of the error between the wideband speech output signal and the original wideband speech signal. (Nagel, Col. 11, paragraph 1, “The selection side information generator 1202 is additionally configured to set the selection side information 1210 so that the selection side information uniquely defines the parametric representation alternative resulting in a frequency enhanced audio signal best matching with the original signal under an optimization criterion. The optimization criterion may be an MMSE (minimum means squared error) [configured to feed back an error] based criterion, a criterion minimizing the sample-wise difference or advantageously a psychoacoustic criterion 20 minimizing the perceived distortion or any other optimization criterion known to those skilled in the art.”)
Khoury and Nagel are combinable for the same rationale as set forth above with respect to claim 1.
In regard to claim 12, Khoury, Nagel and Han teach the apparatus of claim 4.
Khoury further teaches wherein the first discriminator neural network has been trained using recorded speech. (Khoury, paragraph 0053, “As noted above speaker recognition subsystem 20 may receive recognition speech signals 212 from various telephones, computers, and microphones 10.”)
In regard to claim 16, Khoury, Nagel and Han teach the apparatus of claim 1.
Nagel further teaches wherein the apparatus comprises a signal analyser configured to generate the plurality of samples of the signal envelope of the speech input signal and the plurality of samples of the excitation signal of the speech input signal from the speech input signal. (Nagel, Col. 8, paragraph 6, “Particularly, the analysis filter 910 is an LPC analysis filter. The envelope estimation block 902 controls the filter coefficients of the analysis filter [a signal analyser] 910 so that the result of block 910 is the filter excitation signal. This filter excitation signal is extended with respect to frequency in order to obtain an excitation signal at the output of block [generate the plurality of samples of the signal envelope of the speech input signal] 912 which not only has the frequency range of the decoder 120 for an output signal but also has the frequency or spectral range not defined by the core coder and/or exceeding spectral range of the core signal. Thus, the audio signal 909 at the output of the decoder is upsampled and interpolated by an interpolator 900 and, then, the interpolated signal is subjected to the process in the signal estimator 118.”)
Khoury and Nagel are combinable for the same rationale as set forth above with respect to claim 1.
In regard to claim 17, Khoury and Nagel teach the apparatus of claim 1.
Khoury further teaches wherein the first neural network comprises one or more convolutional neural networks. (Khoury, paragraph 0026, “…derivation of channel-compensated features using a CNN,…”)
In regard to claim 18, Khoury and Nagel teach the apparatus of claim 1.
Khoury further teaches wherein the first neural network comprises one or more deep neural networks. (Khoury, paragraph 0026, “…use of a multi-input DNN for increased accuracy.”)
In regard to claim 28, Khoury and Nagel teach the apparatus of claim 1.
Nagel further teaches wherein the speech input signal is a narrowband speech input signal, and/or wherein the speech output signal is a wideband speech output signal. (Nagel, Col. 2, paragraph 1, “After an interpolation of the narrowband signal [narrowband speech input signal] to a wideband sample rate [wideband speech output signal.], a feature vector is computed. Then, by means of a pre-trained statistical hidden Markov model (HMM), an estimate for the wideband spectral envelope is determined in terms of linear prediction (LP) coefficients. These wideband coefficients are used for analysis filtering of the interpolated narrowband signal.”)
Khoury and Nagel are combinable for the same rationale as set forth above with respect to claim 1.
Claims 5-11 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Khoury, in view of Nagel, in further view of Han and in further view of Kavalerov et al (A Multi-Class Hinge Loss for Conditional GANs, "Kavalerov").
In regard to claim 5 and analogous claim 15, Khoury and Nagel teach the apparatus of claim 1.
Khoury teaches wherein, during training of the first neural network, the first discriminator neural network is arranged to receive, as input values of the first discriminator neural network, the output values of the first neural network or is arranged to receive, as the input values of the first discriminator network, derived values being derived from the output values of the first neural network; (Khoury, paragraph 0072, “For training, the second neural network model 600 may additionally include one or more fully connected layers 650 and an output layer 660. An input layer may be two-dimensional, having a first dimension corresponding to an audio sample length ( e.g., 110 milliseconds) and a second dimension corresponding to the number of acoustic features [receive, as the input values of the first discriminator network, derived values being derived from the output values of the first neural ] (i.e. feature vectors) from the channel compensated feature generator 610 (e.g., CNN 230).”)
However, Khoury does not explicitly teach wherein the first neural network is to be trained using a first discriminator neural network; wherein, when the first neural network is trained, the first neural network and the first discriminator neural network are arranged to operate as a generative adversarial network;
wherein, on receiving the input values of the first discriminator neural network, the first discriminator neural network is configured to determine, as output of the first discriminator neural network, a first quality indication for the input values of the first discriminator neural network; and
wherein the first neural network is configured to be trained depending on the first quality indication.
Han teaches wherein, on receiving the input values of the first discriminator neural network, the first discriminator neural network is configured to determine, as output of the first discriminator neural network, a first quality indication for the input values of the first discriminator neural network; and (Han, paragraph 0039, “In some implementations, step 520 may compute a sequence of first stream vectors, step 530 may compute a sequence of second stream vectors, and step 540 may compute a sequence of speech unit score vectors. The sequence of speech unit score vectors may then be used in a speech application.”)
wherein the first neural network is configured to be trained depending on the first quality indication. (Han, paragraph 0040, “…an error may be computed using a label from the training data, and back propagation may be performed with stochastic gradient descent to update the model parameters [trained depending on the first quality indication.]. This process may be performed iteratively over a corpus of training data until the model parameters have converged.”)
Khoury and Han are combinable for the same rationale as set forth above with respect to claim 2.
Kavalerov teaches wherein the first neural network is to be trained using a first discriminator neural network; wherein, when the first neural network is trained, the first neural network and the first discriminator neural network are arranged to operate as a generative adversarial network; (Karalerov, 1.1, Background, “A GAN [second generative adversarial network] [15] is a framework to train a generative model that maps random vectors z 2 Z into data example space x 2 X concurrently with a discriminative network [a second discriminator neural network,]that evaluates its success by judging examples from the dataset and generator as real or fake.”)
Khoury and Karalerov are related to the same field of endeavor (i.e. neural networks). In view of the teachings of Karalerov, it would have been obvious for a person with ordinary skill in the art to apply the teachings of Karalerov to Khoury before the effective filing date of the claimed invention in order to make sure updates are class specific. (Karalerov, 1. Introduction, paragraph 2, “We describe an algorithm that uses both a projection discriminator and an auxiliary classifier with a loss that ensures generator updates are always class specific.”)
In regard to claim 6, Khoury, Nagel, Han and Kavalerov teach the apparatus of claim 5.
Han further teaches wherein, on receiving the input values of the first discriminator neural network, the first discriminator neural network is configured to determine the quality indication such that the quality indication indicates a probability for that the input values of the first discriminator neural network relate to a recorded speech signal instead of an artificially generated speech signal, or indicates an estimation whether the output values of the first discriminator neural network relate to a recorded signal or to an artificially generated signal. (Han, paragraph 0037, “At step 540, a speech unit score vector is computed by processing the first stream vector and the second stream vector (and optionally other stream vectors). Any appropriate techniques may be used, such as any of the techniques described herein. Each element of the speech unit score vector may indicate a likelihood or probability of a corresponding speech unit [that the quality indication indicates a probability for that the input values] being present in a portion of the audio signal. [relate to a recorded speech signal]”)
Khoury and Han are combinable for the same rationale as set forth above with respect to claim 2.
In regard to claim 7, Khoury, Nagel, Han and Kavalerov teach the apparatus of claim 5.
Khoury further teaches wherein the first neural network or the second neural network has been trained using a loss function depending on the quality indication determined by the first discriminator neural network. (Khoury, paragraph 0049, “The loss function 250 utilizes both the features 232 from the CNN 230 and the handcrafted acoustic features 242 from the signal analyzer 240 to produce a loss result 252 and compare the loss result to a predetermined threshold. If the loss result is greater than the predetermined threshold T, the loss result is used to modify connections within the CNN 230, and another recognition speech signal or utterance is processed to further train the CNN [using a loss function depending on the quality indication] 230. Otherwise, if the loss result is less than or equal to the predetermined threshold T, the CNN 230 is considered trained, and the CNN 230 may then be used for providing channel-compensated features to the speaker recognition subsystem 20. (See FIG. 2B, discussed in detail below.)”)
In regard to claim 8, Khoury, Nagel, Han and Kavalerov teach the apparatus of claim 7.
Khoury further teaches wherein the loss function depends on a Hinge loss or depends on a Wasserstein distance or depends on an entropy-based loss. (Khoury, paragraph 0074, “When the bottleneck features are applied in classifying a particular speech signal under test against models (e.g., Gaussian Mixture Model), the loss function to minimize for classification is categorical Cross-Entropy.”)
In regard to claim 9, Khoury, Nagel, Han and Kavalerov teach the apparatus of claim 8.
Kavalerov further teaches wherein the loss function depends on a Hinge loss Lhinge being defined as:
PNG
media_image1.png
25
271
media_image1.png
Greyscale
wherein D() indicates the output of the first discriminator neural network. (Karalerov, pg. 2, Col. 2,
PNG
media_image2.png
36
316
media_image2.png
Greyscale
)
Khoury and Karalerov are combinable for the same rationale as set forth above with respect to claim 5.
In regard to claim 10, Khoury, Nagel, Han and Kavalerov teach the apparatus of claim 7.
Khoury further teaches wherein the loss function depends on an additional L-loss. (Khoury, paragraph 0064, “The loss function processor 250 receives the channel- compensated low-level features 232 and the handcrafted acoustic features 242 and calculates a loss result 252. The loss function employed by the loss function processor 250 may include a mean squared error function.”)
In regard to claim 11, Khoury, Nagel, Han and Kavalerov teach the apparatus of claim 9.
Karalerov further teaches wherein the loss function is defined according to:
PNG
media_image3.png
32
405
media_image3.png
Greyscale
(Karalerov, pg. 5, Col. 1, paragraph 1, “For SAGAN, MHGAN, and MHGAN-SSL we optimize with size 1024 batches, learning rates of 1e -4 and 4e -4 for G and D, and 1 D step per G step. For ACGAN and ACGAN-SSL, we found training to be unsuccessful with only 1 D step per G step, so we train with 2 D steps per G step (training with more than 2 D steps did not improve results). For ACGANSSL we also used a learning rate of 5e 4 for D and a z dimension of 120 instead of 128 in our other experiments [20]. The generator’s auxiliary classifier loss weight is fixed at lambda = 0.1 [examiner interprets this as the lambda of the equation] for all experiments.”)
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SKYLAR K VANWORMER whose telephone number is (703)756-1571. The examiner can normally be reached M-F 6:00am to 3:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Usmaan Saeed can be reached on (571) 272-4046. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/S.K.V./Examiner, Art Unit 2146 /USMAAN SAEED/Supervisory Patent Examiner, Art Unit 2146
Prosecution Insights