DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This Office Action is in response to correspondence filed 25 June 2026 in reference to application 18/840,066. Claims 1-20 are pending and have been examined.
Response to Amendment
The amendment filed 25 June 2026 has been accepted and considered in this office action. Claims 11 and 18 have been amended.
Response to Arguments
Applicant’s arguments, see Remarks, filed 25 June 2025, with respect to the rejection(s) of claim(s) 1-20 under 35 USC 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Chatlani et al. (US Patent 11,854,564).
Claim Rejections - 35 USC § 103
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claim(s) 1-4, 7-15, 17, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zezario et al. (Self-Supervised Autoencoder with Linear-Regression Decoder for Speech Enhancement) in view of Chatlani et al. (US Patent 11,854,564) in view of Biswas et al. (Audio Codec Enhancement with Generative Adversarial Networks).
Consider claim 1, Zezario teaches a method of restoring clean speech from coded audio data (section "2. The proposed DAELD system", lines 1-2, "speech enhancement"), comprising:
obtaining audio data comprising a first set of frames (section "2. The proposed DAELD system", line 12, "noisy speech signal";
extracting a set of feature vectors from the audio data using a self-supervised deep learning model including a neural network (p. 6670, right column, lines 8-9, "extract representative features through the self-supervised learning process"), the set of feature vectors being respectively extracted from the first set of frames (section "2. The proposed DAELD system", lines 12-14, "the noisy speech signals are first processed by the encoder to obtain high dimensional feature representations"); and
generating enhanced speech data comprising a second set of frames from the set of feature vectors ( section "2. The proposed DAELD system", lines 14-15, "representations, which are then transformed to obtain enhanced speech signals"; equ. (4)), the enhanced speech data corresponding to clean speech in the coded audio data (abstract, lines 13-15, "noise could be removed during the reconstruction from the hidden representations").
Zezario does not specifically teach generating enhanced speech data comprising a second set of frames from the set of feature vectors using a generative deep learning model including a neural network.
In the same field of denoising signals with an autoencoder, Chatlani teaches generating enhanced speech data comprising a second set of frames from the set of feature vectors using a generative deep learning model including a neural network (figure 5, 7A, col 16 line 52-col 17 line 12, using a decoder network to generate a clean speech representation).
It would have been obvious to one of ordinary skill in the art at the time of effective filing to use a neural network as the decoder portion of the autoencoder as taught by Chatlani in the system of Zezario in order to make use of the standard autoencoder architecture and to provide a decoder that can be accurately trained to generate clean speech (Chatlani col 17 lines 13-34).
Zezario and Chatlani do not specifically teach that the audio data is coded audio data, and extracting and filtering the coded audio data.
In the same field of generative audio enhancement, Biswas teaches that the audio data is coded audio data, and extracting and filtering the coded audio data (figure 1, section 2, GAN maps coded audio data to enhanced speech data).
It would have been obvious to one of ordinary skill in the art at the time of effective filing to enhance coded audio data as taught by Biswas in the system of Zezario and Chatlani in order to improve the sound quality of standard speech encoding systems (Biswas Abstract).
Consider claim 2, Zezario and Biswas teach the method of claim 1, further comprising:
receiving original coded data (Zezario page 6670, left column, line 24, "TIMIT" database. Biswas section 3.1, dataset),
the obtaining comprising down-sampling the original coded data (Biswas section 3.1, down sampling from 48kHz to 16kHz).
Consider claim 3, Biswas teaches The method of claim 2, wherein the original coded data corresponds to a sampling rate of 48 kHz, and wherein the coded audio data corresponds to a sampling rate of 16 kHz (Biswas section 3.1, down sampling from 48kHz to 16kHz).
Consider claim 4, Zezario teaches The method of claim 1, the coded audio data containing noise or reverbs (page 6671, left column, lines 5-6, "six types of noise" ).
Consider claim 7, Zezario and Biswas teach The method of claim 1, further comprising:
obtaining a training set of distorted speech signals of a specific sampling rate lower than a predetermined sampling rate (Zezario page 6670, TMIT database, page 6671, left column, lines 5-6, "six types of noise" Biswas section 3.1, down sampling from 48kHz to 16kHz); and
building the self-supervised deep learning model using the training set of distorted speech signals (Zezario section 2.1, training the encoder).
Consider claim 8, Zezario Biswas teaches The method of claim 1, further comprising:
obtaining a dataset of down-sampled speech signals relative to a predetermined sampling rate (Biswas section 3.1, down sampling from 48kHz to 16kHz);
generating a training set of sets of feature vectors from the dataset using the self-supervised deep learning model (Zezario Section 3.1, generating training set); and
building the generative deep learning model using the training set of sets of feature vectors (section 2.1, training the learning model).
Consider claim 9, Zezario and Biswas teaches The method of claim 1 further comprising:
obtaining a dataset of distorted speech signals of a specific sampling rate lower than a predetermined sampling rate (Zezario page 6670, TMIT database, page 6671, left column, lines 5-6, "six types of noise" Biswas section 3.1, down sampling from 48kHz to 16kHz); and
training a combined model comprising the self-supervised deep learning model connected with the generative deep learning model using the dataset (Zezario section 2.1, training the encoder Biswas, training encoder and generator section 2).
Consider claim 10, Zezario and Chatlani and Biswas teach a system for restoring clean speech from coded audio data configured to perform the method of claim 1 (see rejection of claim 1 above).
Chatlani further teaches a memory (col 22 lines 63-67 memory, figure 10); and
one or more processors coupled to the memory (col 22 lines 63-67 processors, figure 10).
It would have been obvious to one of ordinary skill in the art at the time of effective filing to use a processor and memory as taught by Matsukawa in the system of Zezario and Biswas in order to allow the system to be implemented with off the shelf and widely available computer components.
Consider claim 11, Zezario teaches a method of restoring clean speech from coded audio data (section "2. The proposed DAELD system", lines 1-2, "speech enhancement"), comprising:
obtaining audio data comprising a first set of frames (section "2. The proposed DAELD system", line 12, "noisy speech signal";
generating a training set of sets of feature vectors from the dataset using a self- supervised deep learning model (Zezario Section 3.1, generating training set);
building a generative deep learning model using the training set of sets of feature vectors (section 2.1, training the learning model);
extracting a set of feature vectors from the audio data using a self-supervised deep learning model including a neural network (p. 6670, right column, lines 8-9, "extract representative features through the self-supervised learning process"), the set of feature vectors being respectively extracted from the first set of frames (section "2. The proposed DAELD system", lines 12-14, "the noisy speech signals are first processed by the encoder to obtain high dimensional feature representations"); and
generating enhanced speech data comprising a second set of frames from the set of feature vectors using a generative deep learning model including a neural network ( section "2. The proposed DAELD system", lines 14-15, "representations, which are then transformed to obtain enhanced speech signals"; equ. (4)), the enhanced speech data corresponding to clean speech in the coded audio data (abstract, lines 13-15, "noise could be removed during the reconstruction from the hidden representations").
Zezario does not specifically teach obtaining a dataset of coded, down-sampled speech signals relative to a predetermined sampling rate, and extracting and filtering the coded audio data.
In the same field of generative audio enhancement, Biswas teaches obtaining a dataset of coded, down-sampled speech signals relative to a predetermined sampling rate (Biswas section 3.1, down sampling from 48kHz to 16kHz), and extracting and filtering the coded audio data (figure 1, section 2, GAN maps coded audio data to enhanced speech data).
It would have been obvious to one of ordinary skill in the art at the time of effective filing to enhance coded audio data as taught by Biswas in the system of Zezario in order to improve the sound quality of standard speech encoding systems (Biswas Abstract).
Zezario and Biswas do not specifically teach implementing the method using a computer-readable, non-transitory storage medium storing computer-executable instructions and
wherein the generative deep learning model includes a recurrent network;
In the same field of generative speech enhancement, Chatlani teaches implementing the method using a computer-readable, non-transitory storage medium storing computer-executable instructions (col 22 lines 63-67 memory, figure 10) and wherein the generative deep learning model includes a recurrent network (figure 5 col 16 lines 23- 51, Recurrent layers included between encoder and decoder).
It would have been obvious to one of ordinary skill in the art at the time of effective filing to use a memory and an RNN as taught by Chatlani in the system of Zezario and Biswas in order to allow the system to be implemented with off the shelf and widely available computer components as well as to allow the system to remember encoded information from previous timesteps to increase decoding accuracy (Chatlani col 22 lines 45-50).
Consider claim 12, Zezario and Biswas teach the computer-readable, non-transitory storage medium of claim 11, further comprising:
obtaining a first training set of coded speech signals of a specific sampling rate lower than the predetermined sampling rate (Zezario page 6670, TMIT database, page 6671, left column, lines 5-6, section 3.1, clean utterances. Biswas section 3.1, down sampling from 48kHz to 16kHz); and
creating the self-supervised deep learning model from the first training set (Zezario section 2.1, training the encoder).
Consider claim 13, Zezario teaches The computer-readable, non-transitory storage medium of claim 12, the method further comprising:
obtaining a second dataset of clean speech signals of the specific sampling rate corresponding to the coded speech signals (Zezario page 6670, TMIT database, page 6671, left column, lines 5-6, section 3.1, clean utterances); and
obtaining the first training set comprising distorting a copy of the second dataset with one or more artifacts caused by a recording environment, a recording equipment, or a coding algorithm (Zezario section 3.1, inserting environmental noise),
the creating being performed further using the second dataset (section 3.1, using noisy speech for training).
Consider claim 14, Zezario and Biswas the computer-readable, non-transitory storage medium of claim 11, further comprising:
receiving original coded data of the predetermined sampling rate (Zezario page 6670, left column, line 24, "TIMIT" database. Biswas section 3.1, dataset),
the extracting comprising down-sampling the original coded data (Biswas section 3.1, down sampling from 48kHz to 16kHz).
Consider claim 15, Zezario and Biswas the computer-readable, non-transitory storage medium of claim 11, the dataset of coded, down-sampled speech signals containing noise or reverbs, and the coded audio data also containing noise or reverbs (page 6671, left column, lines 5-6, "six types of noise" added to signals ).
Consider claim 17, Zezario and Biswas teach The computer-readable, non-transitory storage medium of claim 11,the coded audio data comprising a first set of frames, the set of feature vectors being respectively extracted from the first set of frames, and the enhanced speech data comprising a second set of frames (zezario figure 1, input and output, section 3.1, system built on using Mel frequency power spectrums, which requires frames to be extracted).
Consider claim 20, Zezario and Biswas teach The computer-readable, non-transitory storage medium of claim 11, the method further comprising:
obtaining a second dataset of clean speech signals of the predetermined sampling rate corresponding to the coded, down-sampled speech signals (Zezario page 6670, TMIT database, page 6671, left column, lines 5-6, section 3.1, clean utterances, Biswas section 3.1, down sampling from 48kHz to 16kHz); and
the creating being performed further using the second dataset (section 3.1, using speech for training).
Claim(s) 5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zezario and Chatlani and Biswas as applied to claims 1 above, and further in view of Ravanelli et al. (Multi-Task Self-Supervised Learning for Robust Speech Recognition).
Consider claim 5, Zezario and Chatlani and Biswas teaches The method of claim 1, but does not specifically teach
the self-supervised deep learning model including an encoder and a plurality of workers,
each worker of the plurality of workers performing a self-supervised task related to a distinct speech property, and
a worker of the plurality of workers performing the self-supervised task related to a pre-defined sampling strategy that draws anchor, positive, and negative samples from a pool of representations generated by the encoder.
In the same field of self-supervised learning, Ravanelli teaches
the self-supervised deep learning model including an encoder and a plurality of workers (section 2.2, encoder, section 2.3, workers),
each worker of the plurality of workers performing a self-supervised task related to a distinct speech property (section 2.3, each worker performing a self-supervised task, figure 1, different properties for each worker), and
a worker of the plurality of workers performing the self-supervised task related to a pre-defined sampling strategy that draws anchor, positive, and negative samples from a pool of representations generated by the encoder (section 2.3.2, draws on anchor, positive, and negative samples from a pool).
It would have been obvious to one of ordinary skill in the art at the time of effective filing to use the encoder and worker architecture as taught by Ravanelli in the system of Zezario and Chatlani and Biswas in order to better learn acoustic features (Ravanelli abstract, section 1).
Claim(s) 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zezario and Biswas and Chatlani as applied to claims 11 above, and further in view of Ravanelli et al. (Multi-Task Self-Supervised Learning for Robust Speech Recognition).
Consider claim 16, Zezario and Biswas and Chatlani teach The computer-readable, non-transitory storage medium of claim 11, but does not specifically teach
the self-supervised deep learning model including an encoder and a plurality of workers,
each worker of the plurality of workers performing a self-supervised task related to a distinct speech property, and
a worker of the plurality of workers performing the self-supervised task related to a pre-defined sampling strategy that draws anchor, positive, and negative samples from a pool of representations generated by the encoder.
In the same field of self-supervised learning, Ravanelli teaches
the self-supervised deep learning model including an encoder and a plurality of workers (section 2.2, encoder, section 2.3, workers),
each worker of the plurality of workers performing a self-supervised task related to a distinct speech property (section 2.3, each worker performing a self-supervised task, figure 1, different properties for each worker), and
a worker of the plurality of workers performing the self-supervised task related to a pre-defined sampling strategy that draws anchor, positive, and negative samples from a pool of representations generated by the encoder (section 2.3.2, draws on anchor, positive, and negative samples from a pool).
It would have been obvious to one of ordinary skill in the art at the time of effective filing to use the encoder and worker architecture as taught by Ravanelli in the system of Zezario and Biswas and Chatlani in order to better learn acoustic features (Ravanelli abstract, section 1).
Allowable Subject Matter
Claims 6,18, and 19 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. The following is a statement of reasons for the indication of allowable subject matter:
Consider Claim 6, Zezario and Biswas teach the method of claim 1, the generative deep learning model including a conditional network (Biswas section 2,) but do not specifically teach “the generative deep learning model including a conditional network and a recurrent network, the conditional network converting the set of feature vectors into a set of output feature vectors by considering multiple frames each time, and the recurrent network generating the enhanced speech data from the set of output feature vectors one sample at a time, wherein each frame of the second set of frames comprises a plurality of samples” when combined with each and every other limitation of the claim and base claim. Rather the prior art does not rely on an RNN for reconstruction. Therefore claim 6 contains allowable subject matter.
Claim 18 contains similar limitations as claim 6 and therefore contains allowable subject matter as well.
Claim 19 depends on and further limits claim 18 and therefore contains allowable subject matter as well.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DOUGLAS C GODBOLD whose telephone number is (571)270-1451. The examiner can normally be reached 6:30am-5pm Monday-Thursday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Flanders can be reached at (571)272-7516. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
DOUGLAS GODBOLD
Examiner
Art Unit 2655
/DOUGLAS GODBOLD/Primary Examiner, Art Unit 2655
Prosecution Insights