Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103 is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Priority
Acknowledgment is made of applicant's claim for domestic priority based on US provisional applications 63/325108, 63/417273, and 63/482949 filed on 03/29/2022, 10/18/2022, and 02/02/2023 respectively, and PCT application PCT/US2023/015507 filed on 03/17/2023.
Claim Rejections - 35 USC § 103
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 103 that form the basis for the rejections under this section made in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 37-40, 48-49, and 53-56 are rejected under 35 USC 103 for being unpatentable over Chen et al. (US 2019/0139563 A1) in view of Warner (US 2019/0349704 A1).
Regarding Claims 37 and 54, according to MPEP 2181I, examiners will apply 35 USC 112(f) to a claim limitation if it meets the following 3-prong analysis:
(A) the claim limitation uses the term "means" or "step" or a term used as a substitute for "means" that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term "means" or "step" or the generic placeholder is modified by functional language, typically, but not always linked by the transition word "for" (e.g., "means for") or another linking word or phrase, such as "configured to" or "so that"; and
(C) the term "means" or "step" or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-ATA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f or pre-AJA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”’) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
In particular, claims 37 and 54 recited “spatial cue based separation module to obtain an intermediate audio signal, the spatial cue based separation module being configured to…” and “a source cue based separation module to generate an output audio signal, the source cue based separation module configured to…”.
Here, “module” is a nonce word substitute for “means”. Further, “module” is modified by functional terms like “configured to”. Finally, the claimed functions of the respective “spatial cue based separation module”, “source cue based separation module” contained no structure, material, or acts for performing the respective function.
Therefore, interpretation under 35 USC 112(f) in view of the specification at Fig. 1 is applicable to claims 37 and 54:
Regarding Claims 37 and 54, Chen discloses an audio processing system for source separation (Fig. 3, audio separator 208), comprising
a spatial cue based separation module (¶21, a beamforming network to perform spatial filtering; ¶31, beamformer being part of audio separator 208), configured to obtain an input audio signal comprising at least two channels (¶31, audio separator 208 takes a plurality of audio channels created by microphone array 206) and process the input audio signal to obtain an intermediate audio signal (¶31 and ¶69, beamformer as part of audio separator 208 takes audio channels created by microphone array 206 to produce spatially filtered audio channels as input to a trained machine learning model, each channel corresponds to a directional beam), and
a source cue based separation module (Fig. 10, source separator 1002) configured to process the intermediate audio signal to generate an output audio signal by implementing a neural network trained to predict a noise reduced output audio signal given samples of the intermediate audio signal (¶69, feed each of the audio channels out of the beamformer into a source separator, each source separator produces spectrally filtered outputs using a machine learning model to separate out sources; ¶72 and ¶75, feed a beam from the beamformer into source separator comprising a trained neural network model producing one or more masks and a residual; per ¶22 and Fig. 9, audio separator includes a post selection process to select the top C channels with the best quality audio corresponding to the audio sources).
Chen does not disclose the spatial cue based separation module being configured to determine a mixing parameter of the at least two channels of the input audio signal and modify the at least two channels, based on the mixing parameter, to obtain the intermediate audio signal.
Warner discloses a spatial cue based separation module processing input audio signal to obtain an intermediate audio signal (¶22 and ¶26, processor 102 receives a multi-channel audio signal, analyzes the multi-channel audio signal to extract location / position representing time varying position of the sound; ¶¶37-38, estimate a position of a sound source by generating a time varying position of the sound; per ¶24, calculate a directional vector representing the direction of sound origin), the spatial cue based separation module being configured to determine a mixing parameter of the at least two channels of the input audio signal (¶82, the multi-channel audio signal includes a left channel and a right channel; ¶87, apply a stereo localization model to determine a time varying position in the sound stage) and modify the at least two channels, based on the mixing parameter, to obtain the intermediate audio signal (¶90, when up-mixing stereo to multi-channel, and down-mixing multi-channel signals to stereo, the stereo localization model uses inter-channel level differences to determine lateral panning location and uses correlation between left and right channels to determine front / back localization; ¶91 and ¶118, determine a time varying position of a sound in a stereo audio signal per Fig. 15).
It would’ve been obvious to one ordinarily skilled in the art before the effective filing date of the invention to configure the spatial cue based separation module / trained neural network of Chen to determine a mixing parameter of at least two channels of the input audio signal to modify the at least two channels to obtain the intermediate audio signal (Chen, ¶31, ¶72, and ¶75, separator neural network model produces two masks that separate out four channel audio sources; compare Warner, ¶90, down-mixing multichannel signals (i.e., the four channel audio sources) to two channel stereo to obtain time-vary position of a sound (i.e., spatially separated position of an audio source) in the two channel audio signal) in order to perform spatial separation of audio and extract time varying spatial locations of sound in a sound-stage (Warner, ¶26 and ¶37).
Regarding Claims 38 and 55, Chen as modified by Warner discloses wherein the input audio signal is divided into a plurality of consecutive frames (Warner, ¶36, processor processes one or more frames of audio to estimate channel envelope to determine time-varying volume level; see further ¶¶111-12, use STFT to transform received time domain signal into time-frequency representation to calculate time-frequency envelope for each complex frequency band over time), and wherein the mixing parameter indicates at least one of:
a distribution of the panning of the at least two channels over a plurality of frames in at least one frequency band (Warner, ¶90, using inter-channel level differences to determine lateral panning location; ¶109, audio signals are filtered to produce multiple frequency bands and processor applies analysis to each frequency band in time-frequency representation individually; i.e., determine lateral panning location based on inter-channel level differences for each frequency band), and
a distribution of the inter-channel phase difference of the at least two channels in at least one frequency band over a plurality of frames (Warner, ¶90, left and right channels correlation is positive when the channels are in phase or negative when the channels are out of phase; ¶109, audio signals are filtered to produce multiple frequency bands and processor applies analysis to each frequency band in time-frequency representation individually; i.e., determine channel correlation based on whether the channels are in phase or out of phase for each frequency band).
Regarding Claims 39 and 56, Chen as modified by Warner discloses the mixing parameter is determined for a plurality of frequency bands (Warner, ¶109, audio signals are filtered to produce multiple frequency bands and processor applies analysis to each frequency band in time-frequency representation individually; ¶110, generate a time-varying position for each sound at each frequency range).
Regarding Claim 40, Chen as modified by Warner discloses wherein the spatial cue based separation module operates at a first time and/or frequency resolution (Chen, ¶¶77-79, neural network model takes input signal from a beam and projects the time-frequency bins of each source in the input signal onto a high dimensional embedding space; compare Warner, ¶112, use short-time Fourier transform (STFT) to transform a received time domain signal for each overlapping period of time into time-frequency representation), the method further comprising:
providing, by the spatial cue based separation module, metadata to the source cue based separation module, the metadata indicating the time and/or frequency resolution of the spatial cue based separation module (Warner, ¶67, processor 102 examines an estimated position vector / time-varying position to determine sound as significant for reporting in an event queue; Warner, ¶78 and ¶111, the event queue being an event localization vector including data corresponding to angle, magnitude, loudness, priority, class, time stamp, and duration calculated for each frequency band; Warner, ¶79, processor directs the event indication queue to one or more downstream systems; compare Chen, ¶69, provide spatially filtered audio channels to a source separator); and
generating, by the source cue based separation module, the output audio signal based on the intermediate audio signal and the metadata (Chen, ¶72 and ¶¶78-79, each source separator comprises a trained neural network model that takes the input signal and projects the time-frequency bins of the input signal onto the high-dimensional embedding space:
PNG
media_image1.png
28
92
media_image1.png
Greyscale
where T, F, K denotes time, frequency, and embedding axis; i.e., source separator of Chen uses at least the time stamp in Warner’s event data / metadata corresponding to significant sound event to project corresponding time-frequency bins of (Xt, f) onto the high-dimensional embedding space).
Regarding Claim 48, Chen as modified by Warner discloses mixing the output audio signal with the input audio signal to generate a mixed output audio signal (Chen, ¶31, audio separator takes plurality of audio channels created by beamformer (i.e., spatially filtered channels to produce separated audio sources comprising four channels with separated audio); compare Warner, ¶90, up-mixing stereo (i.e., two channel input audio signal) to multi-channel (e.g., four channels with separated audio as required by Chen)).
Regarding Claim 49, Chen as modified by Warner discloses mixing the output audio signal with the intermediate audio signal to generate a mixed output audio signal (Chen, ¶¶30-31, perform spatial filtering of the audio sources to create a plurality of audio channels as input for audio separator to produce four channels output; Warner, ¶90, up-mixing stereo (2-channel input) to multi-channel).
Regarding Claim 53, Chen discloses wherein the source cue based separation module is configured remove at least one of stationary noise, non-stationary-noise, background audio content (¶39, separating out movie dialog from background sounds), and reverberation.
Allowable Subject Matter
Claims 41-47 and 50-52 are objected to as being dependent upon a rejected base claim, but would be allowable upon being rewritten in independent form including all of the limitations of the base claim and any intervening claims for the following reason:
Prior arts of record do not disclose or render obvious:
41. The method according to claim 40, wherein the intermediate audio signal is divided into a plurality of consecutive frames and each frame is divided into a plurality of frequency bands, and wherein generating the output audio signal comprises:
predicting, by the neural network, a source gain mask, the source gain mask indicating a gain for applying to each frequency band of each frame of the intermediate audio signal; and
smoothing the source gain mask based on the metadata.
45. The method according to claim 37, wherein the spatial cue based separation module determines the mixing parameter with a time resolution which is lower than the time resolution of the source cue based separation module, preferably at least two times lower, more preferably at least four times lower, most preferably at least six times lower.
46. The method according to claim 37, wherein the spatial cue based separation module determines the mixing parameter with a frequency resolution which is lower than the frequency resolution of the source cue based separation module, preferably at least two times lower, more preferably at least five times lower, most preferably at least ten times lower.
47. The method according to claim 37, further comprising: mixing the intermediate audio signal with the input audio signal to generate a mixed intermediate audio signal; and providing the mixed intermediate audio signal to the source cue based separation module.
50. The method according to claim 37, wherein the input audio signal is divided into a plurality of consecutive frames and each frame is divided into a plurality of frequency bands, wherein spatial cue based separation module is further configured to determine a spatial gain mask based on the mixing parameter, the spatial gain mask indicating a gain for applying to each frequency band of each frame of the input audio signal and modifying the at least two channels by applying said spatial gain mask.
51. The method according to claim 50, wherein the intermediate audio signal is divided into a plurality of consecutive frames and each frame is divided into a plurality of frequency bands, and wherein generating the output audio signal comprises: predicting, by the neural network, a source gain mask, the source gain mask indicating a gain for applying to each frequency band of each frame of the intermediate audio signal; combining the source gain mask and the spatial gain mask to form an aggregate gain mask; and applying the aggregate gain mask to the input audio signal.
52. The method according to claim 37, further comprising: providing at least one of the input audio signal, the intermediate audio signal and the output audio signal to a classifier; determining, with the classifier, a probability metric indicating a likelihood that at least one of the input audio signal, the intermediate audio signal and the output audio signal comprises a target audio source; and controlling a gain of the output audio signal based on the probability metric.
Conclusion
Prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
US 2016/0005407 A1 discloses multi-channel audio coding comprising audio encoding system configured to generate a downmix signal bitstream and spatial metadata for generating a multi-channel upmix signal from the downmix signal with spatial metadata defining frequency resolution setting and time resolution setting for mixing parameters (¶100).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to examiner Richard Z. Zhu whose telephone number is 571-270-1587 or examiner’s supervisor Hai Phan whose telephone number is 571-272-6338. Examiner Richard Zhu can normally be reached on M-Th, 0730:1700.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RICHARD Z ZHU/Primary Examiner, Art Unit 2654 05/28/2026
Prosecution Insights