DETAILED ACTION
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 02/03/2026 has been entered.. Claims 1-20 are pending in the application and have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The response filed on 02/03/2026 has been correspondingly accepted and considered in this Office Action. Claims 1-20 have been examined.
Applicant’s amendments to claim 1, 9 and 17, indicating the codebook of discrete vectors encodes information corresponding to a speaking style of the speaker within the audio sequence with the support for improved implementation as indicated in the Applicant Response filed 02/03/2026, page 9, overcome the 35 U.S.C 112 (a) rejections previously set forth in Office Action mailed 11/21/2025. The dependent claims 2-8, 10-16 and 18-20 overcome the 35 U.S.C 112(a) rejections previously set forth in the Office Action mailed 11/21/2025 based on their dependency to the amended claim 1, 9 and 17 respectively. Therefore, the above referenced rejections under 35 U.S.C. 112(a) are withdrawn.
Response to Arguments
Applicant's arguments filed 02/03/2026 have been fully considered as follows:
Applicant’s arguments with respect to claim 1 (also representative of claims 9 and 17) state that
“In this manner, because Garbacea's objective is to embed pitch information in the encoded information, Garbacea, in fact, teaches away from Applicant's pitch and encoded vector representation separation. In a non-limiting example, in one or more embodiments, "[b]y determining the pitch data for speech audio sequences and providing the pitch data generated for to the decoder, while using vector quantization to encode the symbolic representation for speech audio sequences, embodiments train a neural networks to reproduce input speech audio sequences in high quality while using less computing and storage resources. Separating the pitch of the speech from the symbolic representation of the speech allows the encoder to encode the symbolic representation of the speech using a smaller vector codebook." Specification, [0017]”
Applicant’s arguments above with respect to claim 1 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Applicant’s further arguments with respect to claim 1 state that
“Cernak does not solve the deficiencies of Garbacea. For example, Cernak does not
describe extracting pitch using a pitch detection algorithm, and further describes using a syllable- based DLOP codebooks (i.e., through encoding).”
The examiner respectfully disagrees, Cernak teaches in Fig. 2, continuous pitch extraction which is in turn processed by parallel Prosodic Analysis Spiking NN detecting syllable boundaries of the continuous F0 which produces the stylization based on 3-bit quantization of second order DLOP parameters as taught in Cernak sect II C (Fundamental frequency F0 is the lowest rate of vocal fold vibration during voiced speech, typically measured in Hertz (Hz), cycles per second). It is the physical, acoustic correlate of pitch, which is the auditory perception of how high or low a voice sounds). That is, the pitch is extracted in a continuous method which is further encoded by the Spiking NN. Therefore, Cernak teaches extracting, using a pitch detection algorithm, pitch data representing detected pitch of a speaker within the audio sequence and therefore, the rejections of Claims 1 , 9 and 17 are rejected under 35 U.S.C. 103 are sustained and further updated accordingly. To further compact prosecution, Zhao, Yi, et al. "Improved prosody from learned f0 codebook representations for vq-vae speech waveform reconstruction." arXiv preprint arXiv:2005.07884 (2020)is further used to teach extracting, using a pitch detection algorithm, pitch data representing detected pitch of a speaker within the audio sequence as indicated in this Office Action.
In response to the art rejection(s) of the remainder of dependent claims are rejected under 35 U.S.C 103, in case said claims are correspondingly discussed and/or argued for at least the same rationale presented in Remarks filed 02/03/2026, Examiner respectfully notes as follows. For completeness, should the mentioned claims be likewise traversed for similar reasons to independent claims 1, 9 and 17 correspondingly, Examiner respectfully directs Applicant to the same previous supra reasons provided in the response directed towards claims 1, 9 and 17 correspondingly discussed above. For at least the same supra provided reasons, Examiner likewise respectfully disagrees, and Applicant's arguments have been fully considered but they are not persuasive.
Claim Rejections - 35 USC § 103
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claims 1- 4, 6-12, 14-18 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over M. Cernak, et. al., "Composition of Deep and Spiking Neural Networks for Very Low Bit Rate Speech Coding," in IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 24, no. 12, pp. 2301-2312, Dec. 2016 in view of Zhao, Yi, et al. "Improved prosody from learned f0 codebook representations for vq-vae speech waveform reconstruction." arXiv preprint arXiv:2005.07884 (2020).
Regarding claim 1, Cernak teaches a computer-implemented method, comprising: receiving an audio sequence, the audio sequence including speech audio(see Cernak, Fig. 2, speech signal
PNG
media_image1.png
630
820
media_image1.png
Greyscale
); extracting, using a pitch detection algorithm, pitch data representing detected pitch of a speaker within the audio sequence (see Cernak, sect III, Fig. 2 Continuous pitch (extraction pitch algorithm) to determine the pitch of input speech signal; Cernak, sect II C, Effective encoding of the F0 signal can be realized by curve fitting done on a syllable level. We thus propose to encode the continuous F0 signal using the discrete (Legendre) orthogonal polynomial (DLOP), as in [7] (Fundamental frequency F0 is the lowest rate of vocal fold vibration during voiced speech, typically measured in Hertz (Hz), cycles per second). It is the physical, acoustic correlate of pitch, which is the auditory perception of how high or low a voice sounds.)); processing the audio sequence through a convolutional neural network to generate a vector representation of the audio sequence (see Cernak, Section III, bank of DNNs(CNNs) performing segmental speech analysis of conventional acoustic features, and a parallel spiking NN detecting syllable boundaries of the continuous F0 signal and the convolutions of neural network ); generating, by a vector quantizer, an encoded vector representation of wherein the codebook of discrete vectors encodes information corresponding to a speaking style of the speaker within the audio sequence, and wherein the encoded vector representation of the audio sequence is generated independently from the pitch data extracted using the pitch detection algorithm (see Cernak, Fig. 2, sect III A, The outputs of segmental speech analysis are phonological posteriors(independent of pitch) where all unique patterns of the training data create the segmental codebook(codebook of discrete vectors). The number of unique binary patterns (the size of the segmental codebook) is a small fraction of the whole permissible patterns (for example, for the eSPE phonological system, it is about 0.5%). The binary patterns are often repeated frame by frame. The segmental code thus consists of an index of the codebook, along with the duration of the code( vector codebook as indicated in specifications [0016]). The output of the spiking NN syllable analysis is stylized(speaking style) using 3-bit quantization of second order DLOP parameters (continuous F0 signal using the discrete (Legendre) orthogonal polynomial (DLOP) represents pitch from pitch detection algorithm). All stylized F0 mean and F0 slope values create the prosodic codebooks, and the prosodic code consists of the two indexes of the F0 mean and slope codebooks(codebooks of discrete vectors according to speaking style), along with the duration of the transmitted syllable); receiving, by a decoder, the pitch data extracted by the pitch detection algorithm and the encoded vector representation of the audio sequence generated by the vector quantizer (see Cernak, The decoder shown in Fig. 2(b) is based on a DNN performing synthesis of speech cepstral parameters from transmitted segmental and prosodic information); and reconstructing the audio sequence using the pitch data and the encoded vector representation of the audio sequence ( see Cernak, Fig. 2 (b) (iii) a synthesis DNN [46] that decodes the segmental code to the speech parameters for speech re-synthesis ).
While Cernak teaches extracting, using a pitch detection algorithm, pitch data representing detected pitch of a speaker within the audio sequence using continuous pitch extraction and processing the audio sequence through a convolutional neural network to generate a vector representation of the audio sequence using DNNs ( CNNs are a type of DNN), to further compact prosecution Zhao further teaches extracting, using a pitch detection algorithm, pitch data representing detected pitch of a speaker within the audio sequence( see Zhao, sect 4.1, discusses extraction of F0 to obtain the pitch of the speech); processing the audio sequence through a convolutional neural network to generate a vector representation of the audio sequence(see Zhao, Fig. 2, encoder, sect. 4.2 Each block consists of a 1D convolution layer, followed by another 1D convolution layer and a gated activation layer using tanh and sigmoid functions); generating, by a vector quantizer, an encoded vector representation of the audio sequence using the vector representation of the audio sequence and a codebook of discrete vectors, wherein the codebook of discrete vectors encodes information corresponding to a speaking style of the speaker within the audio sequence ( see Zhao, sect 4.3 discusses vector quantization of discrete vectors of F0 for the speaking style within the audio sequence ) .
Cernak and Zhao are considered to be analogous to the claimed invention because they relate to audio coding methods. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Cernak on using DNN for speech compression using speech coders operating at very low bit rate with the pitch extraction teachings of Zhao to separately learn the pitch patterns as well as other F0-related supra-segmental information ( see Zhao, sect 1).
Regarding claim 2, Cernak in view of Zhao teaches the computer-implemented method of claim 1. Zhao further teaches wherein processing the audio sequence through the convolutional neural network to generate the vector representation of the audio sequence comprises: providing the audio sequence in a raw form to convolutional neural network (see Zhao, sect 4.2 discusses the encoder for raw waveforms processed by convolution networks). The motivation to combine as claim 1 applies here.
Regarding claim 3, Cernak in view of Zhao teaches the computer-implemented method of claim 1. Cernak teaches generating audio feature vectors representing the audio sequence using a mel extractor (see Cernak, sect III C, Fig. 3 (b) shows the MFCC extraction ( mel extractor)for Pitch analysis which are 39-order MFCC features input to the Spiking NN). Zhao further teaches providing the audio feature vectors representing the audio sequence to convolutional neural network (see Zhao, sect 4.2 discusses the encoder for raw waveforms processed by convolution networks ). The motivation to combine as claim 1 applies here.
Regarding claim 4, Cernak in view of Zhao teaches the computer-implemented method of claim 1. Cernak further teaches downsampling the vector representation of the audio sequence (see Cernak, III C, 4 discusses down sampling of speech signals ).
Regarding claim 6, Cernak in view of Zhao teaches the computer-implemented method of claim 1. Cernak further teaches extracting initial pitch data from the audio sequence (see Cernak, sect III, Fig. 2 Continuous pitch extraction(pitch algorithm) to determine the pitch of input speech signal). Zhao further teaches downsampling the initial pitch data to generate the pitch data representing the detected pitch within the audio sequence (see Zhao, sect 4.2 discusses down sampling blocks for F0 ( pitch data)). The same motivation to combine as claim 1 applies here.
Regarding claim 7, Cernak in view of Zhao teaches the computer-implemented method of claim 1. Zhao further teaches wherein generating the encoded vector representation of the audio sequence using the vector representation of the audio sequence and the codebook of discrete vectors comprises: for each audio feature vector of the vector representation of the audio sequence, selecting a coding vector from the codebook of discrete vectors (see Zhao, sect 4.3 discusses the encoder output at time step n is quantized using the closest vector included in the VQ codebook/F0 codebook, as computed by Euclidean distance. The closest code vector n for each time step n is repeated for the entire sequence to obtain code vector). The motivation to combine as claim 1 applies here.
Regarding claim 8, Cernak in view of Zhao teaches the computer-implemented method of claim 1. Cernak teaches wherein each vector of the codebook of discrete vectors represents a phoneme (see Cernak, sect II B, describes the acoustic features being coded on phonetic/sub-phonetic level for different phoneme classes ).
Regarding claim 9, is directed to a non-transitory computer-readable storage medium claim corresponding to the computer-implemented method claim presented in claim 1 and is rejected under the same grounds stated above regarding claim 1.
Regarding claim 10, is directed to a non-transitory computer-readable storage medium claim corresponding to the computer-implemented method claim presented in claim 2 and is rejected under the same grounds stated above regarding claim 2.
Regarding claim 11, is directed to a non-transitory computer-readable storage medium claim corresponding to the computer-implemented method claim presented in claim 3and is rejected under the same grounds stated above regarding claim 3.
Regarding claim 12, is directed to a non-transitory computer-readable storage medium claim corresponding to the computer-implemented method claim presented in claim 4 and is rejected under the same grounds stated above regarding claim 4.
Regarding claim 14, is directed to a non-transitory computer-readable storage medium claim corresponding to the computer-implemented method claim presented in claim 6 and is rejected under the same grounds stated above regarding claim 6.
Regarding claim 15, is directed to a non-transitory computer-readable storage medium claim corresponding to the computer-implemented method claim presented in claim 7 and is rejected under the same grounds stated above regarding claim 7.
Regarding claim 16, is directed to a non-transitory computer-readable storage medium claim corresponding to the computer-implemented method claim presented in claim 8 and is rejected under the same grounds stated above regarding claim 8.
Regarding claim 17, is directed to a system claim corresponding to the computer-implemented method claim presented in claim 1 and is rejected under the same grounds stated above regarding claim 1.
Regarding claim 18, is directed to a system claim corresponding to the computer-implemented method claim presented in claim 3 and is rejected under the same grounds stated above regarding claim 3.
Regarding claim 20, is directed to a system claim corresponding to the computer-implemented method claim presented in claim 6 and is rejected under the same grounds stated above regarding claim 6.
Claims 5, 13, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over M. Cernak, et. al., "Composition of Deep and Spiking Neural Networks for Very Low Bit Rate Speech Coding," in IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 24, no. 12, pp. 2301-2312, Dec. 2016 in view of Zhao, Yi, et al. "Improved prosody from learned f0 codebook representations for vq-vae speech waveform reconstruction." arXiv preprint arXiv:2005.07884 (2020) further in view of Zeghidour et. al, US Patent 11,600,282.
Regarding claim 5, Cernak in view of Zhao teaches the computer-implemented method of claim 1. Zhao further teaches receiving, by an audio decoder, the pitch data, wherein the audio decoder is a recurrent neural network (see Zhao, sect 4.4 WaveRNN module );generating reconstructed audio feature vectors representing the audio sequence using the pitch data and the encoded vector representation of the audio sequence (see Zhao, sect 4.4 as shown in Fig. 3 the decoder equation 10 is implemented using the was implemented using a combination of upsampling blocks, downsampling blocks, and a WaveRNN module).
However, Cernak in view of Zhao fail to teach generating the reconstructed audio sequence by passing the reconstructed audio feature vectors through a neural network.
However, Zeghidour generating the reconstructed audio sequence by passing the reconstructed audio feature vectors through a neural network (see Zeghidour, col 10 lines 57-67 discusses the reconstructed output waveform Fig. 3, 206 compared with the training example waveform 204 to determine the reconstruction loss).
Cernak, Zhao and Zeghidour are considered to be analogous to the claimed invention because they relate to audio coding methods. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Cernak in view of Zhao on using a joint training of encoder and decoder networks along with the codebooks of the vector quantizers teachings of Zeghidour to achieve more efficient audio compression ( see Zeghidour, col 2 lines 10-28).
Regarding claim 13, is directed to a non-transitory computer-readable storage medium claim corresponding to the computer-implemented method claim presented in claim 5 and is rejected under the same grounds stated above regarding claim 5.
Regarding claim 19, is directed to a system claim corresponding to the computer-implemented method claim presented in claim 5 and is rejected under the same grounds stated above regarding claim 5.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
O. Cífka, et. al. , "Self-Supervised VQ-VAE for One-Shot Music Style Transfer," ICASSP 2021 - 2021 IEEE International Conference
PNG
media_image2.png
354
1012
media_image2.png
Greyscale
on Acoustics, Speech and Signal Processing (ICASSP), Toronto, ON, Canada, 2021, pp. 96-100 teaches extraction and separation of content and style using disentangled pitch and timbre representations, learned in a self-supervised manner without the need for annotations (see Cifka, Fig. 1)
Engel et. al US Patent 10,068,557 teaches a method methods that include or otherwise leverage a machine-learned neural synthesizer model that can provide intuitive control over timbre and dynamics and enable exploration of new sounds that would be difficult or impossible to produce with a hand-tuned synthesizer (see Engel, Abstract).
Dewasurendra et. al. US PgPub. 2024/0428814 uses The voice encoder based on neural networks to compress the speech signal in an attempt to reduce the bit-rate of the speech signal using linear prediction coding algorithm (e.g., Code-excited linear prediction (CELP), algebraic-CELP (ACELP), or other linear prediction technique) or other voice coding algorithm (see Dewasurendra, Fig. 2).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NANDINI SUBRAMANI whose telephone number is (571)272-3916. The examiner can normally be reached Monday - Friday 12:00pm - 5:00 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Bhavesh M Mehta can be reached at (571)272-7453. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NANDINI SUBRAMANI/ Examiner, Art Unit 2656
Prosecution Insights