Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-22 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Li et al (20180025721) in view of Taghizadeh (20220150661).
As per claim 1, Li et al (20180025721) teaches a multichannel neural frontend speech enhancement model for speech recognition (as front end – para 0103, for speech recognition – abstract), the speech enhancement model comprising:
a speech cleaner configured to: receive, as input, a multichannel noisy input signal and a multichannel contextual noise signal (as using LSTM for multichannel speech processing – para 0075, wherein the network is trained to handle noisy speech signals – para 0051, wherein the LSTM is designed to recognized speech – para 0051, from which context is derived – para 0019);
and generate, as output, a single channel cleaned input signal; an encoder configured to: receive, as input, an input comprising the single channel cleaned input signal output from the speech cleaner and a single channel noisy input signal (see Li et al (20180025721) , computing both the results for a clean input speech signal and a noisy input signal – see para 0071, training of clean utterances, and clean utterances corrupted with varying levels of noise);
As to the claim language toward “and generate, as output, an un-masked output; and a masking layer configured to: receive, as input, the single channel noisy input signal and the un-masked output generated as output from the encoder; and generate, as output, enhanced input speech features corresponding to a target utterance.”; Li et al (20180025721) teaches masking/filterbanks, to detect a noisy speech signal and generating a clean version (para 0076- 0077), and joint calculations (para 0088), and filterbanks to model noisy/clean speech (para 0005, 0031, 0048); however, Li et al (20180025721) does not explicitly applying a filterbank/mask on a mix channel of noisy/clean speech, to enhance the speech spectrum; Taghizadeh (20220150661) teaches inputting multichannel noisy speech waveforms (Fig. 4, top), denoising/cleaning the noisy channel mixed signal – fig 4, output of subblock 405 into 407, then, applying mask 409 with a mixed noisy input and clean blocks, to output an enhanced speech spectrum – output of subblock 409). Therefore, it would have been obvious to one of ordinary skill in the art of acoustic processing of speech signal to modify the processing of the T-F signals of Li et al (20180025721) with a masking of a combination of cleaned/denoised block with the noisy speech and outputting and cleaned enhanced speech spectrum, as taught by Taghizadeh (20220150661) above, because it would advantageously improve the WER (word error rate) – see Fig. 7 – the complete model WER is better than using each section alone).
As per claim 2,3, the combination of Li et al (20180025721) in view of Taghizadeh (20220150661) teaches self attention blocks processing the sub-arrays – para 0040, wherein the size contains the claimed 4 blocks, and the output layers are designed to be similar/conformed to other models – para 0043).
As per claim 4, the combination of Li et al (20180025721) in view of Taghizadeh (20220150661) teaches the speech enhancement model of claim 1, wherein the speech enhancement model executes on data processing hardware residing on a user device, the user device configured to capture the target utterance and the multichannel contextual noise signal via an array of microphones of the user device (see Li et al (20180025721), as, capturing the speech from the client device, recorded from multiple microphones – para 0034).
As per claim 5, the combination of Li et al (20180025721) in view of Taghizadeh (20220150661) teaches the speech enhancement model of claim 4, wherein the speech enhancement model is agnostic to a number of microphones in the array of microphones (see Li et al (20180025721), as, recording over multiple microphones – para 0034, but processing over single/multi channels – para 0022, 0005).
As per claim 6, the combination of Li et al (20180025721) in view of Taghizadeh (20220150661) teaches the speech enhancement model of claim 1, wherein the speech cleaner executes an adaptive noise cancelation algorithm to generate the single channel cleaned input signal by:
applying a finite impulse response (FIR) filter on all channels of the multichannel noisy input signal except for a first channel of the multichannel noisy input signal to generate a summed output (see Taghizadeh (20220150661), para 0041-0043 – see single channel denoising – and in para 0066, the denoising mask); and summing the processing blocks to obtain the enhance spectrum – para 0065);
and subtracting the summed output from the first channel of the multichannel noisy input signal (this claim scope is equivalent to isolating the enhance clean signal – see Taghizadeh (20220150661) , para 0066, 0067, 0059).
As per claim 7, the combination of Li et al (20180025721) in view of Taghizadeh (20220150661) teaches the speech enhancement model of claim 1, wherein a backend speech system is configured to process the enhanced input speech features corresponding to the target utterance (see Li et al (20180025721), as, improving the ASR system, and operating on pre-defined features over a multichannel model – para 0053, in view of para 0004) .
As per claim 8, the combination of Li et al (20180025721) in view of Taghizadeh (20220150661) teaches the speech enhancement model of claim 7, wherein the backend speech system comprises at least one of an automatic speech recognition (ASR) model or an audio or audio-video calling application (examiner notes that the claim language in the alternate format, and, Li et al (20180025721) meets the claim scope by discussing the models for ASR – para 0002).
As per claim 9, the combination of Li et al (20180025721) in view of Taghizadeh (20220150661) teaches ASR/spectral errors/loss (see Li et al (20180025721) – figure 7, WER for the ASR), and Taghizadeh (20220150661) teaches spectral loss – see para 0018, L2 loss
As per claim 10, the combination of Li et al (20180025721) in view of Taghizadeh (20220150661) teaches the speech enhancement model of claim 9, wherein the spectral loss is based on an L1 loss function and L2 loss function distance between an estimated ratio mask and an ideal ratio mask, the ideal ratio mask computed using reverberant speech and reverberant noise (see Li et al (20180025721) teaches SNR loss based on reverberation – para 0071; and Taghizadeh (20220150661) teaching L2 loss –see Fig. 4, subblock 411 and paragraph 0018 calculating a difference measure, in the form of a L2 loss between spectral representations).
As per claim 11, the combination of Li et al (20180025721) in view of Taghizadeh (20220150661) teaches the speech enhancement model of claim 9, wherein the ASR loss is computed by generating, using an ASR encoder of the ASR model configured to receive enhanced speech features predicted by the speech enhancement model for a training utterance as input, predicted outputs of the ASR encoder for the enhanced speech features (see Li et al (20180025721) and mapping and explanation in claims 1,4,7,8 above; and Taghizadeh (20220150661) teaches the enhance speech spectrum – fig. 4, output of subblock 409, mapping to the claimed enhanced speech features);
generating, using the ASR encoder configured to receive target speech features for the training utterance as input, target outputs of the ASR encoder for the target speech features; and computing the ASR loss based on the predicted outputs of the ASR encoder for the enhanced speech features and the target outputs of the ASR encoder for the target speech features (see Li et al (20180025721), figure 7, showing the word-error-rate as the measured ASR loss, for each component during the recognition process, and collectively – see ‘whole model’) .
Claims 12-22 are method claims whose steps are performed by the implemented speech enhance model claims of claims 1-11 above and as such, claims 12-22 are similar in scope and content to claims 1-11 above; therefore, claims 12-22 are rejected under similar rationale as presented against claims 1-11 above.
Allowable Subject Matter
The following is a statement of reasons for the indication of allowable subject matter: The combination of disclosed elements in the specification, toward cleaning a multichannel noise input speech/audio signal and a multichannel contextual noise signal, then applying self attention blocks to the cleaned multichannel noisy audio/speech signal, the cleaned multichannel context signal, and processing with the single channel noisy audio/speech signal, and masking and unmasking using the self-attention blocks, with certain aspects of conformer blocks and a combination of loss, are not explicitly taught by the prior art of record. In terms of the well known technique of attention encoder/decoders, examiner notes, as an example, the Li et al (20180025721) and Taghizadeh (20220150661) references, alone and in combination, as presented above. Furthermore, Watanabe et al (20180261225) teaches a masked based beam former to improve upon end-to-end speech recognition (para 0059), operating on speech and noise signals – Fig. 4, see also, figure 9. The differential between the claim scope, and the Watanabe reference, is the claim scope that encapsulates applicants self attention block processing disclosed in para 0052, 0053 and the equation between, reproduced below:
PNG
media_image1.png
750
472
media_image1.png
Greyscale
Compared to Watanabe’s (20180261225):
PNG
media_image2.png
628
474
media_image2.png
Greyscale
PNG
media_image3.png
774
462
media_image3.png
Greyscale
However, none of the prior art of record, fairly teaches or suggests, the combination of elements found in the specification, as outline above.
Response to Arguments
Applicants amended title is accepted, and the objection to the title has been withdrawn. Applicant's arguments filed 09/17/2025 have been fully considered but they are not persuasive. As per applicants arguments, starting on the bottom of page 3 of the response, examiner disagrees and argues that applicant is reading elements of the specification into the claim scope of “multichannel contextual noise signal”. Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). Examiner reminds applicants, that the Broadest Reasonable Interpretation (BRI) is used to interpret claim limitations (see MPEP 2111 Claim Interpretation; Broadest Reasonable Interpretation). Looking at applicants specification, para 0009 explicitly states “the user device is configured to capture…the multichannel contextual noise signal via an array of microphones…”. In other words, a signal that is captured by the microphones, is labeled as “multichannel contextual noise signal”. There is absolutely no mention of a “separate signal representing noise context from a period before the utterance begins”, nor is this feature claimed tied to “multichannel contextual noise signal”. The same rationale applies to applicants arguments against the Taghizadeh reference, located on page 4 of the response; furthermore, examiner notes that the inputs provided in the Li et al reference are speech-recognized to provide context – hence, the initial signal contains ‘context’, and contains noisy speech signals. As to applicants arguments against the Li and Taghizadeh references, starting on pp 4 of the response, examiner disagrees and notes the mapping in the office action, using Li showing the operations on noisy signals and clean speech signals, and again, reiterating, that context is derived from the speech signals; and applicants claimed “contextual noise signal” is a signal captured by microphone arrays of the user devices, without further defining the signal itself. See commentary above regarding Broadest Reasonable Interpretation, and applicants specification para 0009 stating as such. As per applicants arguments on middle of pp 4 of the response, toward “The combination of Li and Taghizadeh fails to disclose the claimed encoder”, applicants present the argument that Li conflates the process of generating data used for the training with the model’s runtime architecture, examiner argues that although Li contains extra processing steps, the disclosure of Li still meets the current claim scope – in other words, to overcome the Li/Li in view of Taghizadeh reference, examiner suggests adding claim language to overcome this combination. Furthermore, examiner has presented allowable subject matter, in the current and previous office actions, and the claim scope has not been modified to capture these comprehensive elements. As to applicants arguments, on pp 5-6 of the response, toward the combination of the Li/Taghizadeh references, and “improper hindsight reasoning”, examiner notes that the motivation to combine the references comes from the Taghizadeh reference itself, as an improvement over the Li reference, denoising/cleaning the noisy channel mixed signal, and applying a mask with a mixed noisy input and clean blocks, so as to improve the word error rate. Furthermore, it is the Li reference that teaches generating a clean version of the noisy speech signal, and filterbnakds to model both noisy and clean speech; the purpose of the Taghizadeh reference is to teach applying a modified mask to a mixed noisy input and speech blocks, as detailed in the referred Fig. 4 – after processing using mixing weight, the result is an enhanced noise audio signal (enhanced from the original), and then the residual blocks are processed in parallel with a cleaned denoised block and parallel uncleaned block, which is then masked into an enhanced speech spectrum; and hence, Taghizadeh operates on a “cleaned input signal” and a “noisy input signal”.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Regarding the concept to Conformer models:
Koizumi et al., “Df-conformer: Integrated architecture of conv-tasnet and conformer using linear complexity self-attention for speech enhancement”, In 2021 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), Oct. 17-20, 2021, arXiv:2106. 15813v2 [eess.AS], 5 pages.
Regarding the concept of noisy speech/context channels and clean speech/context channels:
Borgstrom (20230162758), Fig 1b, 2c, para 0048-0051
Song et al (20220108712) Fig. 1,2, para 0026-0030
Ciccarelli et al (20200201435), fig. 4, para 0039-0044
Regarding the concept of noise context/clean context channels:
Variani et al (20180068675) teaches neural networks (Fig. 1), to process noisy-into-clean speech (para. 0029 –0035)
Jarvinen (20150106088) see figure 4, with speaker derived character, para 0027-0031.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Michael Opsasnick, telephone number (571)272-7623, who is available Monday-Friday, 9am-5pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Mr. Richemond Dorvil, can be reached at (571)272-7602. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
/Michael N Opsasnick/Primary Examiner, Art Unit 2658 03/26/2026