Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 01/26/26 has been entered.
This office action is in response to correspondence 01/26/26 regarding application 18/548,750, in which claims 27 and 36 were amended. Claims 21-38 are pending in the application and have been considered.
Response to Arguments
The examiner agrees with Applicant on page 6 that no new matter was introduced via the amendments to claims 27 and 36.
Amended claims 27 and 36 overcome the objections for a minor informality, and so they are withdrawn.
Applicant’s arguments on pages 6-8 regarding the 35 U.S.C. 103 rejections based on Benattar, Huang, Taneda, and Tashev have been considered but are not persuasive.
Applicant argues that Benattar does not disclose “…wherein the audio processing profile includes at least a mixing ratio for mixing the audio recording with the processed audio recording and wherein the mixing ratio is controlled at least in part based on the context…” as recited in independent claim 21, allegedly because there is no disclosure by Benattar that the mixing unit 708 uses a mixing ratio for mixing the audio input signals 706 with the channel outputs 703, and there is no factual support in Benattar for the examiner’s finding that mixing unit 708 combines the channel outputs 703, some of which are original source signal and some of which are noise suppressed according to the audio processing profile.
In response, Figure 7 of Benattar is reproduced below:
PNG
media_image1.png
482
764
media_image1.png
Greyscale
As Benattar describes at para [0145], “The channel outputs 703 of the audio divider 701 are connected to the inputs 706 of an audio processing unit 707. The audio processing unit 707 is responsive to audio controller 704, and contains one or more adaptive filters to combine audio input signals 706. The audio controller 704 dictates which inputs are combined and the manner of combination. The audio processing unit 707 is connected to a mixing unit 708 which combines the channel outputs 703 of the audio processing unit 707 in a manner dictated by audio controller 704.”
In other words, according to Benattar above, the channel outputs 703 are connected to the inputs 706, which the diagram clearly shows being passed to mixing unit 708 to be mixed with outputs 710 of audio processing unit 710:
“[0146] Mixing may be accomplished using a digital signal processor. For example a Cirrus Logic C54700xx Audio-System-on-a-chip (ASOC) processor may be used to mix the outputs 710 of audio processing unit 707.”
Applicant is correct that there is no explicit disclosure of a “mixing ratio” in Benattar, however, as pointed out on page 4 of the Final Rejection 10/28/26, as known in the art, mixing of audio inherent requires a “mixing ratio” of the source signals being mixed. While the examiner respectfully disagrees with Applicant that the claims as drafted are patentable over Benattar-Huang, solely to expedite prosecution, a new grounds for rejection is made based further in part on Soulodre et al. (US 20190108851), which explicitly discloses a mixing ratio and mixing an audio signal and a processed audio signal (See Figure 3).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 21-24, 27-29, 37, and 38 are rejected under 35 U.S.C. 103 as being unpatentable over Benattar et al. (US 20160163303) in view of Huang et al. (US 20210125625), in further view of Soulodre et al. (US 20190108851).
Consider claim 21, Benattar discloses an audio processing method, comprising:
receiving, with one or more sensors of a device, environment information about an audio recording captured by the device (location of the audio recording from a location service, [0082], received via a microphone, [0131]);
detecting, with at least one processor of the device, a context of the audio recording based on the audio recording and the environment information, the context representing recording locations (e.g. determining the location audio was recorded, e.g. a passenger train or city street, [0142], using processor within smartphone housing, [0152]);
determining, with the at least one processor, a model based on the context (a noise profile for the recording location, [0142]);
processing, with the at least one processor, the audio recording based on the model to produce a processed audio recording with suppressed noise (processing the audio according to the profile to suppress undesired environmental noise, [0145], [0142], [0038-0049]);
determining, with the at least one processor, an audio processing profile based on the context, (applying mixing controls to the combined sound environment according to the audio profile, [0037]).
Benattar does not specifically mention the context detected using a scene classifier trained to classify audio content into one or more classes; wherein the model is a deep neural network trained to estimate gains.
Huang discloses the context detected using a scene classifier trained to classify audio content into one or more classes (neural network 240 in classifying section 16a2 is trained to classify the ambient environment into multiple scenarios according to CL scores, [0021], [0029], [0030]); wherein the model is a deep neural network trained to estimate gains (neural network in noise suppression section estimates gains, [0027], [0028]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Benattar such that the context is detected using a scene classifier trained to classify audio content into one or more classes; wherein the model is a deep neural network trained to estimate gains in order to improve operation in high noise environments, as suggested by Huang ([0004]), predictably making devices such as mobile phones more useful in any environments regardless of noise type and power, as suggested by Huang ([0004], [0005]). The references cited are analogous art in the same field of audio processing.
Benettar and Huang do not specifically mention wherein the audio processing profile includes at least a mixing ratio for mixing the audio recording with the processed audio recording and wherein the mixing ratio is controlled at least in part based on the context; and combining, with the at least one processor, the audio recording and the processed audio recording based on the mixing ratio.
Soulodre discloses an audio processing profile includes at least a mixing ratio for mixing the audio recording with the processed audio recording and wherein the mixing ratio is controlled at least in part based on a context (input signal X 104, Fig. 3, which is an audio recording, [0057], is processed via Signal Treatment Module and combined with treated signal at a mixing ratio set by treatment gains g1-g7 and gT at a percentage based on the brickwall frequency according to whether Latch 907 is set, [0098], which is reset when a gap is detected, [0027]); and combining, with the at least one processor, the audio recording and the processed audio recording based on the mixing ratio (see above, [0098], [0027]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Benattar and Huang such that the audio processing profile includes at least a mixing ratio for mixing the audio recording with the processed audio recording and wherein the mixing ratio is controlled at least in part based on the context; and combining, with the at least one processor, the audio recording and the processed audio recording based on the mixing ratio in order to reduce audible artifacts such as pumping and variable sound, as suggested by Soulodre, [0027], predictably improving perceived audio quality, a goal identified by Soulodre ([0021]). The references cited are analogous art in the same field of audio processing.
Consider claim 22, Benattar discloses the context indicates that the audio recording was captured indoors or outdoors (e.g. on a city street, or inside a restaurant, [0142]).
Consider claim 23, Benattar discloses the context is detected using an audio scene classifier (e.g. the main dining in Del Frisco’s restaurant, or inside a 1970 Chevelle SS with a well-tuned 396 cubic inch V8 engine, [0142]).
Consider claim 24, Benattar discloses the context is detected using the audio scene classifier in combination with a physical state of the device determined at least in part by the environment information (the location of the device considered a physical state of the device, e.g. inside the main dining in Del Frisco’s restaurant, [0142]).
Consider claim 27, Benattar discloses the audio recording an binaural recording (two microphones may be used in positions corresponding to the user’s ears, i.e. binaural recording, [0133]).
Consider claim 28, Benattar discloses the context is determined at least in part based on a location of the device as determined by a position system of the device (e.g. inside the main dining in Del Frisco’s restaurant, [0142], the restaurant considered the location, and inside the main dining considered the device position within the location).
Consider claim 29, Benattar discloses the audio processing profile includes at least one of an equalization curve or dynamic range control data (e.q. settings, for which a curve is inherent, [0070], and dynamic control of frequency ranges, [0066]).
Consider claim 37, Benattar-Huang-Soulodre discloses a system of processing audio, comprising: one or more processors (processor within smartphone housing, Benattar [0152]); and a non-transitory computer-readable medium storing instructions that, when executed by the one or more processors (memory within iPhone, Benattar [0152]), cause the one or more processors to perform operations of claim 21 (see claim 21 above).
Consider claim 38, Benattar-Huang-Soulodre discloses non-transitory computer-readable medium storing instructions that, when executed by the one or more processors (processor executes instructions from memory within iPhone, Benattar [0152]), cause the one or more processors to perform operations of claim 21 (see claim 21 above).
Claims 25 and 26 rejected under 35 U.S.C. 103 as being unpatentable over Benattar et al. (US 20160163303) in view of Huang et al. (US 20210125625), in further view of Soulodre et al. (US 20190108851), in further view of Taneda (US 20060224382).
Consider claim 25, Benattar discloses the context is detected using the audio scene classifier in combination with a physical state of the device determined at least in part by the environment information (the location of the device considered a physical state of the device, e.g. inside the main dining in Del Frisco’s restaurant, [0142]).
Benattar, Huang, and Soulodre do not specifically mention visual information obtained by an image capture sensor device of the device.
Taneda discloses visual information obtained by an image capture sensor device of the device (visual features extracted from video, Abstract).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Benattar, Huang, and Soulodre by including visual information obtained by an image capture sensor device of the device in order to improve voice activity detection efficiency, as suggested by Taneda ([0003]), predictably reducing misclassification (Taneda, [0002]). The references cited are analogous art in the same field of audio processing.
Consider claim 26, Benattar discloses the context indicates that the audio recording was captured while being transported (in the cabin of a moving vehicle, [0079]).
Claims 30 and 31 are rejected under 35 U.S.C. 103 as being unpatentable over Benattar et al. (US 20160163303) in view of Huang et al. (US 20210125625), in further view of Soulodre et al. (US 20190108851), in further view of Tashev et al. (US 20190318755).
Consider claim 30, Benattar discloses wherein processing, with the at least one processor, the audio recording based on the model to produce a processed audio recording comprises: obtaining a speech from the audio recording (speech signal, [0059]); computing a frequency spectrum of the speech, the frequency spectrum including a plurality of frequencies (the 10 hz to 20,000 hz frequency spectrum, [0076]).
Benattar does not specifically mention: speech frame; frequency bins; extracting frequency band features from the plurality of frequency bins; estimating gains for each of the plurality of frequency bands based on the frequency band features and the model; adjusting the estimated gains based on the audio processing profile; converting the frequency band gains into frequency bin gains; modifying the frequency bins with the frequency bin gains; reconstructing the speech frame from the modified frequency bins; and converting the reconstructed speech frame into an output speech frame.
Huang discloses speech frame (speech dominant frame, [0034]); frequency bins (frequency bins, [0037]); extracting frequency band features from the plurality of frequency bins (extracting features from the frequency bands, [0029]); estimating gains for each of the plurality of frequency bands based on the frequency band features and the model (estimating a series of frequency band gains, [0029]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Benattar by including speech frame; frequency bins; extracting frequency band features from the plurality of frequency bins; estimating gains for each of the plurality of frequency bands based on the frequency band features and the model for reasons similar to those for claim 21.
Benattar, Huang, and Soulodre do not specifically mention: adjusting the estimated gains based on the audio processing profile; converting the frequency band gains into frequency bin gains; modifying the frequency bins with the frequency bin gains; reconstructing the speech frame from the modified frequency bins; and converting the reconstructed speech frame into an output speech frame.
Tashev discloses adjusting the estimated gains based on the audio processing profile (audio processing for the noise model, using the DNN, [0029-0033]); converting the frequency band gains into frequency bin gains; modifying the frequency bins with the frequency bin gains (applying to suppression gains to each frequency bin, [0028]); reconstructing the speech frame from the modified frequency bins (the output is fed to the waveform reconstruction module, [0055]); and converting the reconstructed speech frame into an output speech frame (synthesis of the output waveform, [0073]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Benattar, Huang, and Soulodre by adjusting the estimated gains based on the audio processing profile; converting the frequency band gains into frequency bin gains; modifying the frequency bins with the frequency bin gains; reconstructing the speech frame from the modified frequency bins; and converting the reconstructed speech frame into an output speech frame in order to better mitigate both stationary and non-stationary noise, predictably resulting in better listening quality, as suggested by Tashev ([0003]). The references cited are analogous art in the same field of audio processing.
Consider claim 31, Benattar, Huang, and Soulodre do not, but Tashev discloses the band features include at least one of Mel Frequency Cepstral Coefficients (MFCC), Bark Frequency Cepstral Coefficients (BFCC), or a band harmonicity feature indicating how much the band is composed of a periodic audio signal (MFCC features, [0105]; the examiner notes the claim language requires “one of”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Benattar, Huang, and Soulodre such that the band features include at least one of Mel Frequency Cepstral Coefficients (MFCC), Bark Frequency Cepstral Coefficients (BFCC), or a band harmonicity feature indicating how much the band is composed of a periodic audio signal for reasons similar to those for claim 30.
Claims 32-34 are rejected under 35 U.S.C. 103 as being unpatentable over Benattar et al. (US 20160163303) in view of Huang et al. (US 20210125625), in further view of Soulodre et al. (US 20190108851), in further view of Tashev et al. (US 20190318755), in further view of Chen et al. (US 20130151244).
Consider claim 32, Benattar, Huang, Soulodre, and Tashev do not, but Chen discloses band features include the harmonicity feature and the harmonicity feature is computed from the frequency bins of the speech frame or calculated by correlation between the speech frame and a previous speech frame (harmonic to non-harmonic ratio, computed from frequency bins of speech signal, [0024-0027]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Benattar, Huang, Soulodre, and Tashev such that band features include the harmonicity feature and the harmonicity feature is computed from the frequency bins of the speech frame or calculated by correlation between the speech frame and a previous speech frame in order to estimate speech quality, with predictable advantages in user hearing perception, as suggested by Chen ([0001-0002]). The references cited are analogous art in the same field of audio processing.
Consider claim 33, Benattar does not, but Huang discloses the model is a deep neural network (DNN) model that is configured to estimate the gains and voice activity detection (VAD) for each frequency band of the speech frame based on the band features of the speech frame (classifying and noise suppression sections use deep neural networks to estimate gains and classify as speech or without speech, [0028-0029], Fig 3A element S3120, S314).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Benattar such that the model is a deep neural network (DNN) model that is configured to estimate the gains and voice activity detection (VAD) for each frequency band of the speech frame based on the band features of the speech frame for reasons similar to those for claim 21.
Benattar, Huang, Soulodre, and Tashev do not specifically mention a fundamental frequency of the speech frame.
Chen discloses a fundamental frequency of the speech frame (estimate the fundamental frequency of the frame, Fig 3 step 302, [0002]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Benattar, Huang, Soulodre, and Tashev by utilizing a fundamental frequency of the speech frame for reasons similar to those for claim 32.
Consider claim 34, Benattar discloses an adaptive filter (adaptive filter, [0043]).
Benattar does not specifically mention a Wiener Filter.
Huang discloses a Wiener suppression rule is combined with the DNN model to compute the estimated gains (Wiener filter in addition to the neural-network based noise suppressor, [0031]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Benattar such that a Wiener filter is combined with the DNN model to compute the estimated gains for reasons similar to those for claim 21.
Claims 35 and 36 are rejected under 35 U.S.C. 103 as being unpatentable over Benattar et al. (US 20160163303) in view of Huang et al. (US 20210125625), in further view of Soulodre et al. (US 20190108851), in further view of Turgut et al. (“Acoustic observations of internal tides and tidal currents in shallow water”. J. Acoust. Soc. Am. 133, 1981-1986, 2013).
Consider claim 35, Benattar does not, but Huang discloses a model trained with audio samples and associated noise (models trained with clean and noisy signals, [0030]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Benattar such that the model is trained with audio samples and associated noise (model trained with clean and noisy spectrograms for reasons similar to those for claim 21.
Benattar, Huang, and Soulodre do not specifically mention the audio recording was captured near a body of water and audio samples of tides and associated noise.
Turgut discloses audio recordings were captured near a body of water and audio samples of tides and associated noise (broadband signal spectrograms measured in the East China Sea during the summer of 2008, which include frequency shifts indicative of tides, Abstract, page 1981).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Benattar, Huang, and Soulodre in order to improve acoustic monitoring capability reliability, as suggested by Turgut (Section 1, page 1981). The references cited are analogous art in the same field of audio processing.
Consider claim 36, Benattar does not, but Huang discloses the training data is separated into two datasets: a first dataset and a second dataset that includes the associated noise samples (ground truth containing clean speech and noisy speech, [0030]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Benattar such that the training data is separated into two datasets: a first dataset and a second dataset that includes the associated noise samples for reasons similar to those for claim 21.
Benattar, Huang, and Soulodre do not specifically mention a dataset that includes the tide samples.
Turgut discloses a dataset that includes the tide samples (broadband signal spectrograms measured in the East China Sea during the summer of 2008, which include frequency shifts indicative of tides, Abstract, page 1981).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Benattar, Huang, and Soulodre by including a dataset that includes the tide samples for reasons similar to those for claim 35.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
US 20170311078 Liu discloses dynamic audio signal enhancement with a AGC module and mixer which mix high frequency signals and processed low frequency signals adaptively, see Fig 3, [0055]
US 11317206 Neerbek discloses analyzing a video scene associated with an audio recording and adaptively selecting original or noise suppressed audio
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Jesse Pullias whose telephone number is 571/270-5135. The examiner can normally be reached on M-F 8:00 AM - 4:30 PM. The examiner’s fax number is 571/270-6135.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Andrew Flanders can be reached on 571/272-7516.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Jesse S Pullias/
Primary Examiner, Art Unit 2655 05/01/25
Prosecution Insights