DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
1. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
2. Claim(s) 1, 2, 8-10, 17, 18, 20, 23, 26, and 29 are rejected under 35 U.S.C. 103 as being unpatentable over Muraoka (U.S. Pat. No. 4,987,557 A) in view of Sharma (U.S. Pub. No. 2024/0144936 A1).
Regarding Claim 1, Muraoka teaches a device (apparatus of Fig. 1, Col. 4, Lns. 10-14) comprising:
a memory configured to store instructions (memory 1, Fig. 1, Col. 4, Lns. 18-21); and
one or more processors (processor array 2 which includes a plurality of processors, Fig. 1, Col. 4, Lns. 25-29) configured to:
detect single-stream data (plurality of processors serving as multipliers for multiplying items from memory 1 individually, Fig. 1, Abstract, Col. 4, Lns. 25-29).
Muraoka fails to explicitly teach generate multi-stream augmented data that includes one or more modified versions of the single-stream data;
process the multi-stream augmented data to generate multiple output channels; and
reduce the multiple output channels to produce single-stream output data.
However Sharma teaches generate multi-stream augmented data that includes one or more modified versions of the single-stream data (single channel speech enhancement process 10 generates 102 a plurality of modified signals from the single microphone signal. For example, single channel speech enhancement process 10 uses speech signal 200 and one or more channel processors (e.g., represented in FIG. 2 as channel processor 202) to generate a plurality of modified speech signals (e.g., modified speech signals 204, 206, 208, 210, 212) from the single microphone signal (e.g., speech signal 200), Figs. 1 and 2, Para. [0013]; see also Paras. [0014]-[0026]);
process the multi-stream augmented data to generate multiple output channels (single channel speech enhancement process 10 exploits the “front end” (i.e., channel processors 202) for single microphone channel data by providing as input the single microphone channel to generate a number of modified single channel signals, Fig. 2, Para. [0027]); and
reduce the multiple output channels to produce single-stream output data (performing 104 speech processing on the plurality of modified signals includes combining 120 the plurality of modified signals, thus defining a combined signal, Para. [0028]; the various modified speech signals received by the channel combining system (e.g., channel combining system 214) are processed at the front end into a single channel representation (e.g., combined signal 216), Fig. 2, Para. [0030]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the device (as taught by Muraoka) to include the multi-stream augmented modified version, generating the multiple output channels and reducing to a single stream output data (as taught by Sharma). Doing so ensures that no information is lost (Sharma Para. [0010]).
Regarding Claim 2, Muraoka in view of Sharma teaches wherein the one or more processors are further configured to perform one or more first operations on the single-stream data to generate the one or more modified versions of the single-stream data (Sharma, single channel speech enhancement process 10 generates 102 a plurality of modified signals from the single microphone signal, Figs. 1 and 2, Para. [0013]; see also Paras. [0014]-[0026]).
Regarding Claim 8, Muraoka in view of Sharma teaches wherein the one or more first operations include applying a gain (Sharma, single channel speech enhancement process 10 generates 102 modified speech signal 212 by performing automatic gain control or signal level enhancements on speech signal 200, Para. [0026]).
Regarding Claim 10, Muraoka in view of Sharma teaches wherein the one or more processors are configured to process the multi-stream augmented data using a recurrent network that processes each stream of the multi-stream augmented data in parallel and that uses the same network weights for each stream of the multi-stream augmented data (Sharma, channel combining system includes a neural network or other machine learning system configured to receive the plurality of modified signals and the original speech signal as inputs and apply weights to each of the modified speech signals to generate a single channel representation of the plurality of weighted modified speech signals, Para. [0029]; a recurrent neural network can be used, Para. [0030]).
Regarding Claim 17, Muraoka in view of Sharma teaches wherein the single-stream data includes dual-channel audio data (Sharma, suppose a speaker verification system is used to segment the audio, single channel speech enhancement process 10 constructs 108 e.g., two new channels by applying a mask to speech signal 200, Para. [0016]).
Regarding Claim 18, it is similarly rejected as Claim 17.
Regarding Claim 20, Muraoka in view of Sharma teaches further comprising one or more microphones configured to provide the single-stream data (Sharma, single channel speech enhancement process 10 receives 100 a signal from a single microphone, Figs. 1 and 4, Para. [0012]).
Regarding Claim 23, Muraoka in view of Sharma teaches wherein the one or more processors are included in a neural processing unit (NPU) (Sharma, performing 104 speech processing on the plurality of modified signals includes combining 120 the plurality of modified signals, Fig. 2, Para. [0028]; channel combining system 214 includes a neural network or other machine learning system, Fig. 2, Para. [0029]; i.e. the combining system 214 will contain a processor to carry out the speech processing on the modified signals).
Regarding Claim 26, it is similarly rejected as Claim 1. The method is found in Sharma (process 10, Figs. 1 and 2).
Regarding Claim 29, it is similarly rejected as Claim 1. The on-transitory computer-readable medium is found in Sharma (Fig. 4, Paras. [0035]-[0042]).
3. Claim(s) 3, 13, 19, and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Muraoka (U.S. Pat. No. 4,987,557 A) in view of Sharma (U.S. Pub. No. 2024/0144936 A1), and further in view of Jelcicova et al. (U.S. Pat. No. 11,330378 B1, hereinafter "Jelcicova").
Regarding Claim 3, Muraoka in view of Sharma teach wherein, to reduce the multiple output channels, the one or more processors are further configured to:
perform a combination operation on channels of the adjusted multi-channel output data to generate the single-stream output data (Sharma, performing 104 speech processing on the plurality of modified signals includes combining 120 the plurality of modified signals, thus defining a combined signal, Para. [0028]; the various modified speech signals received by the channel combining system (e.g., channel combining system 214) are processed at the front end into a single channel representation (e.g., combined signal 216), Fig. 2, Para. [0030]).
Muraoka in view of Sharma fails to explicitly teach perform one or more second operations on at least one of the multiple output channels to generate adjusted multi-channel output data, the one or more second operations corresponding to inverse operations of the one or more first operations.
However, Jelcicova teaches perform one or more second operations on at least one of the multiple output channels to generate adjusted multi-channel output data, the one or more second operations corresponding to inverse operations of the one or more first operations (a synthesis filter bank (FB-S) for converting the frequency sub-band signal OUT(k′,t) to a time domain signal OUT(t), Figs. 4A and 4B, Col. 29, Lns. 23-46; i.e. an inverse of the process of filter bank (FB-A)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the device (as taught by Muraoka in view of Sharma) to include the inverse operations (as taught by Jelcicova). Doing so preserves the natural sound quality of the audio leading to improved signal fidelity and integrity.
Regarding Claim 13, Muraoka in view of Sharma fails to explicitly teach wherein the one or more processors are configured to:
train the recurrent network using multi-stream augmented training data; and use the trained recurrent network to process the multi-stream augmented data during an inference operation.
However, Jelcicova teaches wherein the one or more processors are configured to:
train the recurrent network using multi-stream augmented training data (a neural network comprising the Peak GRU RNN algorithm according to the present disclosure may be trained with a baseline GRU RNN providing optimal weights for a trained network, Fig. 8, Col. 32, Lns. 54-57); and use the trained recurrent network to process the multi-stream augmented data during an inference operation (the Peak GRU constraints may then be applied to the trained network, Col. 32, Lns. 54-57).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the device (as taught by Muraoka in view of Sharma) to include the training the recurrent network used during an inference operation (as taught by Jelcicova). Doing so improves the noise reduction (Jelcicova Col. 29, Lns. 5-13).
Regarding Claim 19, Muraoka in view of Sharma fails to explicitly teach further comprising one or more speakers configured to output audio of the single-stream output data.
However, Jelcicova teaches further comprising one or more speakers configured to output audio of the single-stream output data (loudspeaker SPK, Figs. 4A and 4B, Col. 29, Lns. 23-46).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the device (as taught by Muraoka in view of Sharma) to include the speaker (as taught by Jelcicova). Doing so, cleaner and high fidelity audio can be delivered.
Regarding Claim 21, Muraoka in view of Sharma fails to explicitly teach further comprising a modem configured to receive the single-stream data from a second device via wireless transmission.
However, Jelcicova teaches further comprising a modem configured to receive the single-stream data from a second device via wireless transmission (a wireless receiver for wirelessly receiving a direct electric input signal (e.g. an audio signal) from another device, Col. 9, Lns. 14-26 and Lns. 51-56).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the device (as taught by Muraoka in view of Sharma) to include the modem (as taught by Jelcicova). Doing so, the device can be configured to receive electromagnetic signals using wide range of frequencies (Jelcicova Col. 9, Lns. 14-26).
4. Claim(s) 4 is rejected under 35 U.S.C. 103 as being unpatentable over Muraoka (U.S. Pat. No. 4,987,557 A) in view of Sharma (U.S. Pub. No. 2024/0144936 A1) in view of Jelcicova et al. (U.S. Pat. No. 11,330378 B1, hereinafter "Jelcicova"), and further in view of Nesta et al. (U.S. Pub. No. 2020/0184985 A1, hereinafter "Nesta").
Regarding Claim 4, Muraoka in view of Sharma, and further in view of Jelcicova fail to explicitly teach wherein the combination operation includes averaging values of the channels of the adjusted multi-channel output data.
However, Nesta teaches wherein the combination operation includes averaging values of the channels of the adjusted multi-channel output data (a recurrent neural network can be implemented to predict the speech activity or produce a normalized score related to the average SNR to predict the quality of the signal and thus produce a combined channel, Para. [0034]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the device (as taught by Muraoka in view of Sharma, and further in view of Jelcicova) to include the average values of the channels (as taught by Nesta). Doing so maximizes the SNR for applications (Nesta Para. [0034]).
5. Claim(s) 5 and 6 are rejected under 35 U.S.C. 103 as being unpatentable over Muraoka (U.S. Pat. No. 4,987,557 A) in view of Sharma (U.S. Pub. No. 2024/0144936 A1), and further in view of Yang et al. (U.S. Pat. No. 9,607,626 B1, hereinafter "Yang").
Regarding Claim 5, Muraoka in view of Sharma fails to explicitly teach wherein the one or more first operations include a frequency-domain phase shift.
However, Yang teaches wherein the one or more first operations include a frequency-domain phase shift (reconfigurable filter bank used to separate audio data into multiple frequency bands. Each frequency band of the multiband output may have an identical phase shift such that the frequency bands are in-phase and can be added together, Fig. 1, Col. 1, Ln. 61 thru Col. 2, Ln. 27).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the device (as taught by Muraoka in view of Sharma) to include the frequency-domain phase shift (as taught by Yang). Doing so leads to reduced computational complexity and/or latency (Yang Col. 2, Lns. 11 and 12).
Regarding Claim 6, it is similarly rejected as Claim 5.
6. Claim(s) 7 is rejected under 35 U.S.C. 103 as being unpatentable over Muraoka (U.S. Pat. No. 4,987,557 A) in view of Sharma (U.S. Pub. No. 2024/0144936 A1), and further in view of Suzuki (U.S. Pub. No. 2020/0090029 A1).
Regarding Claim 7, Muraoka in view of Sharma fails to explicitly teach wherein the one or more first operations include a time-domain shift.
However, Suzuki teaches wherein the one or more first operations include a time-domain shift (input section 120 includes a shift register circuit 121 [used to perform a time-domain shift], Fig. 1, Para. [0061] and [0063]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the device (as taught by Muraoka in view of Sharma) to include the time-domain shift (as taught by Suzuki). Doing so, there is zero overhead buffering and low computational overhead.
7. Muraoka (U.S. Pat. No. 4,987,557 A) in view of Sharma (U.S. Pub. No. 2024/0144936 A1) in view of Jelcicova et al. (U.S. Pat. No. 11,330378 B1, hereinafter "Jelcicova"), and further in view of Zhang et al. (U.S. Pub. No. 2019/0341058 A1, hereinafter "Zhang").
Regarding Claim 22, Muraoka in view of Sharma, and further in view of Jelcicova teach wherein the one or more processors are further configured to send the single-stream output data to the second device via the modem (Jelcicova, transmitter Tx is used to transmit wirelessly to second device, Fig. 6, Col. 31, Lns. 55-57).
Muraoka in view of Sharma, and further in view of Jelcicova fail to explicitly teach wherein the single-stream data is received in connection with a federated learning network.
However, Zhang teaches wherein the single-stream data is received in connection with a federated learning network (trained joint neural network 1019, Fig. 14, Para. [0065]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the device (as taught by Muraoka in view of Sharma, and further in view of Jelcicova) to include the federated learning network (as taught by Zhang). Doing so results in a more robust and generalizable predictions.
Conclusion
8. Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHIMEZIE E BEKEE whose telephone number is (571)272-0202. The examiner can normally be reached M-F 7.30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Duc Nguyen can be reached at 571-272-7503. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHIMEZIE EZERIWE BEKEE/Examiner, Art Unit 2691
/DUC NGUYEN/Supervisory Patent Examiner, Art Unit 2691
Prosecution Insights