DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This Office Action is in response to correspondence filed 19 November 2024 in reference to application 18/952,607. Claims 1-20 are pending and have been examined.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-11, 14, and 17-20 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Kuleshov et al. (Audio Super-Resolution Using Neural Nets).
Consider claim 1, Kuleshov teaches a method performed by one or more computers (abstract), the method comprising:
receiving an input audio waveform that comprises a respective input audio sample for each of a plurality of input time steps (section 2, audio processing, time series of audio signal samples);
processing the input audio waveform using an encoder neural network to generate a set of feature vectors representing the input audio waveform, wherein the encoder neural network comprises a sequence of encoder blocks ( section 3.2, and figure 1, down sampling blocks) that are each configured to:
process a respective set of input feature vectors in accordance with a set of encoder block parameters to generate a set of output feature vectors, comprising down-sampling the set of input feature vectors (section 3.2, down sampling the features in the down sampling blocks); and
processing the set of feature vectors representing the input audio waveform using a decoder neural network to generate an output audio waveform that comprises a respective output audio sample for each of a plurality of output time steps (section 3.2 and figure 1, up sampling blocks), wherein the decoder neural network comprises a sequence of decoder blocks that are each configured to:
process a respective set of input feature vectors in accordance with a set of decoder block parameters to generate a set of output feature vectors, comprising up-sampling the set of input feature vectors (section 3.2 and figure 1, up sampling blocks up samples the features which were down sampled in the encoder side);
wherein the output audio waveform represents a version of the input audio waveform captured at a higher sampling frequency than the input audio waveform (section 3.1, figure 2, recovering the high resolution signal from a low resolution signal).
Consider claim 2, Kuleshov teaches the method of claim 1, wherein the encoder neural network has been trained on a set of training examples that each comprise: (i) a training input audio waveform, and (ii) a target output audio waveform that should be generated by the audio processing neural network by processing the training input audio waveform, wherein the target output audio waveform represents a version of the training input audio waveform with a higher sampling frequency than the training input audio waveform (introduction, training on pairs of high and low resolution samples, section 4 datasets, generating training data by low pass filtering high resolution data, figure 2 as well).
Consider claim 3, Kuleshov teaches the method of claim 2, wherein for one or more of the training examples, the training input audio waveform has a sampling frequency that is less than or equal to 4KHz and the target output audio waveform has a sampling frequency that is greater than or equal to 8KHz (section 3.1, and section 4, target resolution is 16k, low resolution may be 4k).
Consider claim 4, Kuleshov teaches the method of claim 2, wherein the set of training examples comprises a plurality of training examples with training input audio waveforms that have a range of different sampling frequencies (section 3.1, different up sampling ratios used of 2, 4, and 6).
Consider claim 5, Kuleshov teaches the method of claim 4, wherein the range of sampling frequencies comprises at least four distinct sampling frequencies (section 3.1, different up sampling ratios used of 2, 4, and 6. The 4th frequency would be the target frequency of 16k).
Consider claim 6, Kuleshov teaches the method of claim 5, wherein the four distinct sampling frequencies comprise one or more of: 2KHz, 3KHz, 4KHz, or 5KHz (section 3.1, figure 2, 4k sampling frequency)
Consider claim 7, Kuleshov teaches the method claim 4, wherein the plurality of training examples with training input audio waveforms that have the range of different sampling frequencies each include a target output audio waveform with a same target sampling frequency (section 3.1, different ratios all based on target rate of 16k).
Consider claim 8, Kuleshov teaches The method of claim 7, wherein the target sampling frequency is 16 KHz (section 3.1, d target rate of 16k).
Consider claim 9, Kuleshov teaches the method of claim 1, wherein the audio processing neural network implements causal convolutions (section 3.2, figure 1, each block consists of convolutions. Introduction, network run in real time, so convolutions must be causal as future data would not be available).
Consider claim 10, Kuleshov teaches the method of claim 1, wherein the audio processing neural network is implemented on a mobile device (section 5, audio applications, performing super resolution in handsets. ).
Consider claim 11, Kuleshov teaches The method of claim 1, wherein each encoder block in the sequence of encoder blocks down-samples the set of input feature vectors to the encoder block using a respective strided convolution operation (section 3.2, figure 1, strided convolution used in each block).
Consider claim 14, Kuleshov teaches The method of claim 1, wherein each decoder block in the sequence of decoder blocks up-samples the set of input feature vectors to the decoder block using a respective strided transposed convolution operation (section 3.2 and figure 1, up sampling with strided convolution layers).
Consider claim 17, Kuleshov teaches The method of claim 1, wherein for each encoder block that is after a first encoder block in the sequence of encoder blocks, the set of input feature vectors to the encoder block comprises a set of output feature vectors generated by a preceding encoder block in the sequence of encoder blocks (figure 1, section 3.2, each down sample block feeds into each other).
Consider claim 18, Kuleshov teaches the method of claim 1, wherein for each decoder block that is after a first decoder block in the sequence of decoder blocks, the set of input feature vectors to the decoder block comprises: (i) a set of output feature vectors of a corresponding encoder block, and (ii) a set of output feature vectors generated by a preceding decoder block in the sequence of decoder blocks (figure 1, section 3.2, up sample block feeds into next up sample block, along with skip connections from corresponding down sample block).
Consider claim 19, Kuleshov teaches a system (abstract) comprising:
one or more computers (section 4, page 6, implemented on GPU, which is a computer); and
one or more storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers (section 4, page 6, implemented on GPU, which requires memory and instructions), cause the one or more computers to perform operations comprising:
receiving an input audio waveform that comprises a respective input audio sample for each of a plurality of input time steps (section 2, audio processing, time series of audio signal samples);
processing the input audio waveform using an encoder neural network to generate a set of feature vectors representing the input audio waveform, wherein the encoder neural network comprises a sequence of encoder blocks ( section 3.2, and figure 1, down sampling blocks) that are each configured to:
process a respective set of input feature vectors in accordance with a set of encoder block parameters to generate a set of output feature vectors, comprising down-sampling the set of input feature vectors (section 3.2, down sampling the features in the down sampling blocks); and
processing the set of feature vectors representing the input audio waveform using a decoder neural network to generate an output audio waveform that comprises a respective output audio sample for each of a plurality of output time steps (section 3.2 and figure 1, up sampling blocks), wherein the decoder neural network comprises a sequence of decoder blocks that are each configured to:
process a respective set of input feature vectors in accordance with a set of decoder block parameters to generate a set of output feature vectors, comprising up-sampling the set of input feature vectors (section 3.2 and figure 1, up sampling blocks up samples the features which were down sampled in the encoder side);
wherein the output audio waveform represents a version of the input audio waveform captured at a higher sampling frequency than the input audio waveform (section 3.1, figure 2, recovering the high resolution signal from a low resolution signal).
Consider claim 20, Kuleshov teaches One or more non-transitory computer storage media storing instructions that when executed by one or more computers (section 4, page 6, implemented on GPU, which requires memory and instructions), cause the one or more computers to perform operations comprising:
receiving an input audio waveform that comprises a respective input audio sample for each of a plurality of input time steps (section 2, audio processing, time series of audio signal samples);
processing the input audio waveform using an encoder neural network to generate a set of feature vectors representing the input audio waveform, wherein the encoder neural network comprises a sequence of encoder blocks ( section 3.2, and figure 1, down sampling blocks) that are each configured to:
process a respective set of input feature vectors in accordance with a set of encoder block parameters to generate a set of output feature vectors, comprising down-sampling the set of input feature vectors (section 3.2, down sampling the features in the down sampling blocks); and
processing the set of feature vectors representing the input audio waveform using a decoder neural network to generate an output audio waveform that comprises a respective output audio sample for each of a plurality of output time steps (section 3.2 and figure 1, up sampling blocks), wherein the decoder neural network comprises a sequence of decoder blocks that are each configured to:
process a respective set of input feature vectors in accordance with a set of decoder block parameters to generate a set of output feature vectors, comprising up-sampling the set of input feature vectors (section 3.2 and figure 1, up sampling blocks up samples the features which were down sampled in the encoder side);
wherein the output audio waveform represents a version of the input audio waveform captured at a higher sampling frequency than the input audio waveform (section 3.1, figure 2, recovering the high resolution signal from a low resolution signal).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 12, 13, 15, 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kuleshov in view of Liu et al. (US Patent 11,727,926).
Consider claim 12, Kuleshov teaches The method of claim 11, but does not specifically teach wherein for each encoder block in the sequence of encoder blocks, the strided convolution operation is a one-dimensional or two-dimensional strided convolution operation.
In the same field of audio enhancement using encoder-decoder bottleneck architectures, Liu teaches wherein for each encoder block in the sequence of encoder blocks, the strided convolution operation is a one-dimensional or two-dimensional strided convolution operation (col 18 lines 43-50, down sampling with 2D convolutional function with stride greater than 1) .
It would have been obvious to one of ordinary skill in the art at the time of effective filing to use 2d convolutional functions as taught by Liu in the system of Kuleshov in order to more accurately enhance the audio signal (Liu col 1 lines 45 col 2 line 44).
Consider claim 13, Kuleshov teaches the method of claim 1, but does not specifically teach wherein for each encoder block in the sequence of encoder blocks, a dimensionality of the output feature vectors generated by the encoder block is higher than a dimensionality of the input feature vectors processed by the encoder block.
In the same field of audio enhancement wherein for each encoder block in the sequence of encoder blocks, a dimensionality of the output feature vectors generated by the encoder block is higher than a dimensionality of the input feature vectors processed by the encoder block (col 18 lines 35-42, growth rates, i.e. number of outputs for each input of 32) .
It would have been obvious to one of ordinary skill in the art at the time of effective filing to use convolutional functions that increase dimensionality as taught by Liu in the system of Kuleshov in order to more accurately enhance the audio signal (Liu col 1 lines 45 col 2 line 44).
Consider claim 15, Kuleshov teaches The method of claim 14, but does not specifically teach wherein for each decoder block in the sequence of decoder blocks, the strided transposed convolution operation is a one-dimensional or two-dimensional strided transposed convolution operation.
In the same field of audio enhancement using encoder-decoder bottleneck architectures, Liu teaches wherein for each decoder block in the sequence of decoder blocks, the strided transposed convolution operation is a one-dimensional or two-dimensional strided transposed convolution operation (col 19 lines 20-30, up sampling using transposed convolution corresponding to col 18 lines 43-50, down sampling with 2D convolutional function with stride greater than 1) .
It would have been obvious to one of ordinary skill in the art at the time of effective filing to use 2d convolutional functions as taught by Liu in the system of Kuleshov in order to more accurately enhance the audio signal (Liu col 1 lines 45 col 2 line 44).
Consider claim 16, Kuleshov teaches the method of claim 1, but does not specifically teach wherein for each decoder block in the sequence of decoder blocks, a dimensionality of the output feature vectors generated by the decoder block is lower than a dimensionality of the input feature vectors processed by the decoder block.
In the same field of audio enhancement wherein for each encoder block in the sequence of encoder blocks, a dimensionality of the output feature vectors generated by the encoder block is higher than a dimensionality of the input feature vectors processed by the encoder block (col 19 lines 20-30, up sampling using transposed convolution corresponding to col 18 lines 43-50, down sampling with 2D convolutional function with stride greater than) .
It would have been obvious to one of ordinary skill in the art at the time of effective filing to use convolutional functions that increase dimensionality as taught by Liu in the system of Kuleshov in order to more accurately enhance the audio signal (Liu col 1 lines 45 col 2 line 44).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Ahn et al. (US 2023/0274754) teaches an audio enhancement system using a similar auto-encoder architecture.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DOUGLAS C GODBOLD whose telephone number is (571)270-1451. The examiner can normally be reached 6:30am-5pm Monday-Thursday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Flanders can be reached at (571)272-7516. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
DOUGLAS GODBOLD
Examiner
Art Unit 2655
/DOUGLAS GODBOLD/Primary Examiner, Art Unit 2655
Prosecution Insights