DETAILED ACTION
Introduction
This office action is in response to Applicant’s submission filed on November 7, 2024.
Claims 1-17 are pending in the application. As such, claims 1-17 have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Drawings
The drawings were received on November 7, 2024. These drawings have been accepted and considered by the Examiner.
Claim Objections
Claims 13-15 are objected to because of the following informalities:
Claims 13-15, line 6 for each, reads “greater than the threshold”. Examiner believes this to be a clerical error and it is intended to read “greater than the predetermined threshold”.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3 and 16-17 are rejected under 35 U.S.C. 103 as being unpatentable over Watanabe et al. (US Patent Pub. No. 20180261225 A1), hereinafter Watanabe, in view of Tang et al. (US Patent Pub. No. 20230298611 A1), hereinafter Tang.
Regarding claims 1, 16 and 17, Watanabe teaches an information processing device, a non-transitory computer-readable storage medium, and a method (Watanabe in [0013] teaches a speech recognition system, in [0005] teaches a method for automatic speech recognition, and in [0108] teaches using hardware, software or a combination thereof. When implemented in software, the software code can be executed on any suitable processor or collection of processors, whether provided in a single computer or distributed among multiple computers)
comprising:
[claim 16 only] non-transitory computer-readable storage medium storing a program causing a computer to execute processing (Watanabe in [0108] teaches using hardware, software or a combination thereof. When implemented in software, the software code can be executed on any suitable processor or collection of processors, whether provided in a single computer or distributed among multiple computers)
processing circuitry (Watanabe in [0013] teaches a speech recognition system with a central processing unit (CPU) and a graphic processing unit (GPU))
to calculate an acoustic component from mixed audio data by using a predetermined function, the mixed audio data including target speech audio and mixed noise, the target speech audio being a target to be enhanced, the mixed noise being noise to be mixed with the target speech audio, the acoustic component being a component of the target speech audio and the mixed noise (Watanabe in [0006] teaches analysis of components of a speech and noise signal, performing speech enhancement, using a function, estimating masks, and estimating filter coefficients, identifying a speech signal, and in [0008] teaches taking into account background noise);
to estimate an acoustic feature by inputting the acoustic component to a feature estimation model trained to estimate an acoustical feature of speech and noise (Watanabe in [0006] teaches analysis of components of a speech and noise signal, computing expected speech and noise statistics regarding power spectral density (PSD) matrices relevant to speech and noise signals, where the speech and noise statistics are time-invariant features);
to calculate a noise component from noise data by using the predetermined function, the noise component being a component of noise, the noise data not including the target speech audio but including noise (Watanabe in [0063, Fig. 4] teaches analysis of the noise signal separate from the speech signal);
to estimate a noise feature by inputting the noise component to a noise estimation model trained to estimate an acoustical feature of the noise (Watanabe in [0103, Fig. 9] teaches determining features of the noise signal using an estimation module);
to estimate a target speech mask by inputting the [integrated feature] to a speech enhancement model trained to estimate a mask for enhancing speech (Watanabe in [0008] teaches using speech enhancement techniques to generate an enhanced speech signal, and in [0006] teaches including a network first estimates time-frequency masks, which are used to compute expected speech and noise statistics regarding power spectral density (PSD) matrices relevant to speech and noise signals, the speech and noise statistics are time-invariant features);
and
to restore speech audio in which the target speech audio is enhanced from the acoustic component and the target speech mask (Watanabe in [0033] teaches producing an enhanced speech signal).
Watanabe teaches the acoustic feature, the noise feature, the acoustical feature of speech and noise, and the acoustical feature of noise.
Watanabe does not teach, however Tang teaches
to estimate a correlation between [the acoustic feature and the noise feature by inputting the acoustic feature and the noise feature] to a correlation estimation model trained to estimate a correlation between [the acoustical feature of speech and noise, and the acoustical feature of noise] (Tang in [0030] teaches determining correlation information between the speech component and the noise component);
to calculate an integrated feature by weighting the acoustic feature with the estimated correlation (Tang in [0060] teaches applying the sample-independent attention mechanism to a certain frequency-domain feature representation corresponds to the process of multiplying a 1×F vector by an F×F weight matrix, the parameter of the fully connected layer indicating the frequency correlation is also referred to as frequency-domain weighting information, which yields the parameter of the fully connected layer (integrated feature)).
Tang is considered to be analogous to the claimed invention because it is in the same field of speech enhancement. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Watanabe further in view of Tang to allow for correlating and weighting based on the speech and noise signals. Motivation to do so would allow for the performance of speech enhancement to be improved, which helps to obtain completely pure speech (Tang [0030]).
Regarding claim 2, Watanabe, as modified above, teaches the information processing device according to claim 1.
Watanabe further teaches
further comprising:
an interface to accept input of data (Watanabe in [0039] teaches using an input interface including multiple microphones to convert sound into multi-channel speech signals);
wherein, the processing circuitry
acquires the mixed audio data via the interface (Watanabe in [0039] teaches using an input interface including multiple microphones to convert sound into multi-channel speech signals),
and
acquires the noise data via the interface (Watanabe in [0039] teaches using an input interface including multiple microphones to convert sound into multi-channel speech signals).
Regarding claim 3, Watanabe, as modified above, teaches the information processing device according to claim 1.
Watanabe further teaches
further comprising:
an interface to accept input of data (Watanabe in [0039] teaches using an input interface including multiple microphones to convert sound into multi-channel speech signals);
wherein, the processing circuitry
acquires, via the interface, acoustic data having a segment including the target speech audio and a segment not including the target speech audio (Watanabe in [0039] teaches using an input interface including multiple microphones to convert sound into multi-channel speech signals [the microphones will receive as input noisy speech as well as noise with no speech]),
generates the mixed audio data from data on the segment including the target speech audio in the acoustic data (Watanabe in [0039] teaches using an input interface including multiple microphones to convert sound into multi-channel speech signals),
and
generates the noise data from data on the segment not including the target speech audio in the acoustic data (Watanabe in [0039] teaches using an input interface including multiple microphones to convert sound into multi-channel speech signals).
Claims 4-6 are rejected under 35 U.S.C. 103 as being unpatentable over Watanabe, in view of Tang, in view of Aran (US Patent Pub. No. 20180359580 A1).
Regarding claims 4, 5 and 6, Watanabe, as modified above, teaches the information processing device according to claims 1, 2 and 3.
Watanabe, as modified above, teaches calculating the acoustic component.
Watanabe, as modified above, does not teach, however Aran teaches
wherein the processing circuitry
divides the mixed audio data into a plurality of blocks (Aran in [0044] teaches reduce noise in the audio data, digitize the audio data, and divide the digitized audio data into frames representing time intervals),
and
[calculates the acoustic component] for the blocks (Aran in [0044] teaches reduce noise in the audio data, digitize the audio data, and divide the digitized audio data into frames representing time intervals).
Aran is considered to be analogous to the claimed invention because it is in the same field of speech enhancement. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Watanabe, as modified above, further in view of Aran to allow for dividing the digitized audio data into frames representing time intervals. Motivation to do so would allow for improving the likelihood that the speech recognition process will output speech results that make sense grammatically (Aran [0044]).
Claims 7-12 are rejected under 35 U.S.C. 103 as being unpatentable over Watanabe, in view of Tang, in view of Aran, in view of Solbach et al. (US Patent Pub. No. 20090323982 A1), hereinafter Solbach.
Regarding claims 7, 8 and 9, Watanabe, as modified above, teaches the information processing device according to claims 4, 5 and 6.
Watanabe further teaches
wherein,
the processing circuitry further
estimates a noise mask for enhancing noise (Watanabe in [0098] teaches using a mask estimation network includes pre-trained mask data sets that have been obtained by training the mask estimation network using predetermined mask estimation ground truth input data),
estimates a restored noise feature by inputting the [restored noise component] to the noise estimation model (Watanabe in [0103, Fig. 9] teaches determining features of the noise signal using an estimation module),
and
generates a combined noise feature by combining the restored noise feature with the noise feature in a time direction (Watanabe in [0063] teaches the mask estimation uses two real-valued BLSTM networks, where the BLSTM network is used for generating a speech mask and the BLSTM network is for generating a noise mask, and each of the BLSTM networks outputs the time-frequency masks related speech signals and noise signals, and concatenating these features using weight matrices).
Watanabe, as modified above, teaches the acoustic feature, the combined noise feature, and the processing circuitry.
Watanabe, as modified above, does not teach, however Tang teaches
when the [combined noise feature] is generated, the [processing circuitry] estimates the correlation from the [acoustic feature] and the [combined noise feature] (Tang in [0030] teaches determining correlation information between the speech component and the noise component).
Tang is considered to be analogous to the claimed invention because it is in the same field of speech enhancement. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Watanabe, as modified above, further in view of Tang to allow for correlating and weighting based on the speech and noise signals. Motivation to do so would allow for the performance of speech enhancement to be improved, which helps to obtain completely pure speech (Tang [0030]).
Watanabe, as modified above, teaches the acoustic component, and the noise mask.
Watanabe, as modified above, does not teach, however Solbach teaches
calculates a restored noise component by emphasizing noise with the [acoustic component] and the [noise mask] (Solbach in [0011] teaches a noise component signal may be determined in each sub-band of signals received by the microphone by subtracting the primary acoustic signal weighted by a complex-valued coefficient sigma from the secondary acoustic signal).
Solbach is considered to be analogous to the claimed invention because it is in the same field of speech enhancement. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Watanabe, as modified above, further in view of Solbach to allow for a noise component signal to be determined. Motivation to do so would allow for improved noise suppression while minimizing speech distortion (Solbach [0024]).
Regarding claims 10, 11 and 12, Watanabe, as modified above, teaches the information processing device according to claims 7, 8 and 9.
Watanabe, as modified above, teaches the restored noise feature.
Watanabe further teaches
wherein the processing circuitry
generates a combined noise feature by (Watanabe in [0063] teaches the mask estimation uses two real-valued BLSTM networks, where the BLSTM network is used for generating a speech mask and the BLSTM network is for generating a noise mask, and each of the BLSTM networks outputs the time-frequency masks related speech signals and noise signals, and concatenating these features using weight matrices)
combining, in a time direction, the restored noise feature with the noise feature [estimated for a block immediately following the block for which the restored noise component has been calculated] (Watanabe in [0103, Fig. 9] teaches determining features of the noise signal using an estimation module, and in [0063] teaches the mask estimation uses two real-valued BLSTM networks, where the BLSTM network is used for generating a speech mask and the BLSTM network is for generating a noise mask, and each of the BLSTM networks outputs the time-frequency masks related speech signals and noise signals, and concatenating these features using weight matrices).
Watanabe, as modified above, does not teach, however Aran teaches
[estimated for] a block immediately following the block (Aran in [0044] teaches reduce noise in the audio data, digitize the audio data, and divide the digitized audio data into frames representing time intervals).
Aran is considered to be analogous to the claimed invention because it is in the same field of speech enhancement. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Watanabe, as modified above, further in view of Aran to allow for dividing the digitized audio data into frames representing time intervals. Motivation to do so would allow for improving the likelihood that the speech recognition process will output speech results that make sense grammatically (Aran [0044]).
Watanabe, as modified above, teaches the acoustic component, and the noise mask.
Watanabe, as modified above, does not teach, however Solbach teaches
the restored noise component (Solbach in [0011] teaches a noise component signal may be determined in each sub-band of signals received by the microphone by subtracting the primary acoustic signal weighted by a complex-valued coefficient sigma from the secondary acoustic signal).
Solbach is considered to be analogous to the claimed invention because it is in the same field of speech enhancement. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Watanabe, as modified above, further in view of Solbach to allow for a noise component signal to be determined. Motivation to do so would allow for improved noise suppression while minimizing speech distortion (Solbach [0024]).
Claims 13-15 are rejected under 35 U.S.C. 103 as being unpatentable over Watanabe, in view of Tang, in view of Aran, in view of Solbach, in view of Shi et al. (US Patent Pub. No. 20140180682 A1), hereinafter Shi.
Regarding claims 13, 14 and 15, Watanabe, as modified above, teaches the information processing device according to claims 7, 8 and 9.
Watanabe, as modified above, teaches the processing circuitry, generating the combined noise feature, and the restored noise component.
Watanabe, as modified above, does not teach, however Shi teaches
wherein the processing circuitry
calculates a noise likelihood (Shi in [0116] teaches a noise detector 108 compares the noise likelihood of the input signal supplied from the likelihood calculator 107 with a preset threshold value to determine whether the n-th frame of the input signal is a non-stationary noise frame. For example, when a noise likelihood threshold value R_th for determination of non-stationary noise is preset, and the noise likelihood R(n) is greater than the noise likelihood threshold value R_th, the n-th frame of the input signal is determined to be a non-stationary noise frame. Conversely, when the noise likelihood R(n) is equal to or less than the noise likelihood threshold value R_th, the n-th frame of the input signal is determined not to be a non-stationary noise frame),
determines whether or not the noise likelihood is equal to or greater than a predetermined threshold (Shi in [0116] teaches a noise detector 108 compares the noise likelihood of the input signal supplied from the likelihood calculator 107 with a preset threshold value to determine whether the n-th frame of the input signal is a non-stationary noise frame. For example, when a noise likelihood threshold value R_th for determination of non-stationary noise is preset, and the noise likelihood R(n) is greater than the noise likelihood threshold value R_th, the n-th frame of the input signal is determined to be a non-stationary noise frame. Conversely, when the noise likelihood R(n) is equal to or less than the noise likelihood threshold value R_th, the n-th frame of the input signal is determined not to be a non-stationary noise frame),
and
[generates the combined noise feature] when the noise likelihood is equal to or greater than the threshold, the noise likelihood being a likelihood of [the restored noise component] (Shi in [0116] teaches a noise detector 108 compares the noise likelihood of the input signal supplied from the likelihood calculator 107 with a preset threshold value to determine whether the n-th frame of the input signal is a non-stationary noise frame. For example, when a noise likelihood threshold value R_th for determination of non-stationary noise is preset, and the noise likelihood R(n) is greater than the noise likelihood threshold value R_th, the n-th frame of the input signal is determined to be a non-stationary noise frame. Conversely, when the noise likelihood R(n) is equal to or less than the noise likelihood threshold value R_th, the n-th frame of the input signal is determined not to be a non-stationary noise frame).
Shi is considered to be analogous to the claimed invention because it is in the same field of speech enhancement. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Watanabe, as modified above, further in view of Shi to allow for comparing the noise likelihood of the input signal. Motivation to do so would allow for detecting various sudden noises without an increase in a processing load of the device (Shi [0014]).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PAUL J. MUELLER whose telephone number is (571)272-1875. The examiner can normally be reached M-F 9:00am-5:00pm (Eastern).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel C. Washburn can be reached at 571-272-5551. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
PAUL MUELLER
Examiner
Art Unit 2657
/PAUL J. MUELLER/Examiner, Art Unit 2657
Prosecution Insights