Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAIL ACTION
Priority
This application claims priority to U.S provisional Patent Application No. 63367350, filed on 6/30/2022 and is hereby incorporated by references.
Information Disclosure Statement
The information disclosure statement (IDS) was submitted on 12/26/2024. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-15, 18-16 and 31 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claimed invention is directed to non-statutory subject matter because the claim(s) as a whole, considering all claim elements both individually and in combination, do not amount to significantly more than an abstract idea. As summarized in the 2019 Revised Patent Subject Matter Eligibility Guidance, examiners must perform a Two-Part Analysis for Judicial Exceptions.
Step 1
In Step 1, it must be determined whether the claimed invention is directed to a process, machine, manufacture or composition of matter. The instant invention encompasses four sets of claims: a method in claims 1-9 (i.e., a process), a method in claims 18-26 (i.e., a process), a device in claim 26 (i.e., a manufacture) and a system in claims 31 (i.e., a manufacture). All claims are directed to one of the four statutory categories and meet the requirements of step 1.
Step 2A
Prong One
The claimed invention is directed to an abstract idea without significant more. The instant invention is broadly directed to “applying a trained neural network for sound separation based on distance estimation”. Claim 1 recites the following (with emphasis added):
Claim 1: A computer-implemented method of applying a trained neural network for sound separation based on distance estimation, comprising:
receiving, by an audio input component of a computing device, an audio mixture from one or more sources;
predicting, by a trained distance estimation neural network and based on the audio mixture, respective distances of the one or more sources from the audio input component;
determining one or more near sounds and one or more far sounds based on the respective distances, wherein the one or more near sounds correspond to sources that are located within a threshold distance of the audio input component, and the one or more far sounds correspond to sources that are not located within the threshold distance of the audio input component; and
providing, by the computing device, the predicted one or more near sounds.
The bold portions of claim 1 encompass the abstract idea, which is also encompassed by the dependent claims 2-15, and substantially also encompassed by claims 18-25, 26 and 31.
Claims 1, 18, 26 and 31 recite the steps to generate sound separation based on distance estimation by a neural network processing. These limitations, when given their broadest reasonable interpretation, are directed to certain performing of organizing human activity and mental processes, which is abstract idea.
Prong Two
This judicial exception is not integrated into a practical application because mere instruction to implement on computers (i.e. storage medium or processors in claim 1 and 26) or a computer model (neural network here in claim 1), or merely using computers as a tool to perform the abstract idea, adding insignificant extra solution activity, and/or generally linking the use of the abstract idea to a technological environment for field of use is not considered integration into a practical application. Claim 1 recites using neural network to predict sound separation based on distance estimation. Using a trained neural network model to process audio input data is a generic feature of neural network process, which does not represent a technological improvement. The using of the computer and neural network process does not add improvement to the functioning of a computer or to any other technology field, which failed to enable the abstract idea to integrate into a practical application. The claims are drafted in a result-oriented fashion, without the requisite specificity needed to provide a nonabstract technological solution. The computing system and neural network process are directed to the components of a system amount to merely field of use type limitations and/or extra solution activity to implement the abstract idea as presented.
Step 2B
Step 2B in the analysis requires us to determine whether the claims do significantly more than simply describe that abstract method. Mayo, 132 S. Ct. at 1297. We must examine the limitations of the claims to determine whether the claims contain an "inventive concept" to "transform" the claimed abstract idea into patent-eligible subject matter. Alice, 134 S. Ct. at 2357 (quoting Mayo, 132 S. Ct. at 1294, 1298). The transformation of an abstract idea into patent-eligible subject matter "requires 'more than simply stat[ing] the [abstract idea] while adding the words 'apply it."' Id. (quoting Mayo, 132 S. Ct. at 1294) (alterations in original). "A claim that recites an abstract idea must include 'additional features' to ensure 'that the [claim] is more than a drafting effort designed to monopolize the [abstract idea].'" Id. (quoting Mayo, 132 S. Ct. at 1297) (alterations in original). Those "additional features" must be more than "well-understood, routine, conventional activity." Mayo, 132 S. Ct. at 1298.
The present claims include the additional elements other than the abstract idea which include a computer device, storage medium, neural network model and client device with user interface (in claim 1 and 26). These additional elements are merely conventional computer and computer model. Any potentially technical aspects of the claims are well-known generic computer components performing conventional functions (e.g., a processor performing a mental process). The present claims have been analyzed both individually and in combination and, the instant claims do not provide any improvement of the functioning of the computer or improvement to computer technology or any other technical field. There do not appear to be any meaningful limitations other than those that are well-understood, routine and conventional in the field. Thus, the present claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. Thus, the claims 1-15 are not patent eligible.
Claims 18-25, 26 and 31 recite similar limitations of claims 1-15, thus are abstract idea and not patent eligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-5, 8, 10,13-15, 18, 20, 24-26 and 31 is/are rejected under 35 U.S.C. 103 as being unpatentable over KOIZUMI et al (US 20210219048 A1) in view of SATO et al (US 20220335965 A1).
Regarding claim 1, KOIZUMI discloses a computer-implemented method of applying a neural network [e.g. FIG. 2; learning device with neural network] for sound separation based on distance estimation [e.g. FIG. 1; [0001 and 0026]; the near/distant sound source separation is estimated with deep learning by using the acoustic feature value], comprising: receiving, by an audio input component of a computing device [e.g. FIG. 1; 12 and 13], an audio mixture from one or more sources [e.g. sound sources collected by the microphone array]; predicting, by a distance estimation neural network [e.g. FIG. 1-2; [0027-0033]; deep learning neural network] based on the audio mixture, respective distances of the one or more sources from the audio input component [e.g. [0059-0060]; estimated value of short-distance acoustic signal or long-distance acoustic signal]; determining one or more near sounds [e.g. short-distance acoustic signal] and one or more far sounds [e.g. long-distance acoustic signal] based on the respective distances, wherein the one or more near sounds correspond to sources that are located within a threshold distance of the audio input component [e.g. [0015 and 0084]; each microphone to each near sound source is not more than 30 cm], and the one or more far sounds correspond to sources that are not located within the threshold distance of the audio input component [e.g. the distance from each microphone to each distant sound source is not less than 1 m]; and providing, by the computing device, the predicted one or more near sounds [e.g. FIG. 3-5; [0061-0062]; estimated value of short distance acoustic signal emitted from the position close to a plurality of microphones].
It is noted that KOIZUMI differs to the present invention in that KOIZUMI fails to explicitly disclose a trained distance estimation neural network.
However, SATO teaches the well-known concept of applying a trained neural network [e.g. FIG. 1; [0035 and 0042]; trained neural network for sound extraction] for sound separation.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the sound separation system disclosed by KOIZUMI to exploit the well-known providing sound separation by using a trained neural network model technique taught by SATO as above, in order to provide improved performance of extracting an audio signal of a target speaker [See SATO; [0117 and 129]].
Regarding claim 2, KOIZUMI and SATO further disclose predicting, by a trained audio separation neural network [e.g. KOIZUMI: FIG. 2; SATO: FIG. 1], one or more sounds corresponding to the one or more sources; and predicting the respective distances for the predicted one or more sounds [e.g. KOIZUMI: FIG. 2; [e.g. [0059-0060]; estimated value of short-distance acoustic signal or long-distance acoustic signal; SATO: FIG. 1].
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the sound separation system disclosed by KOIZUMI to exploit the well-known providing sound separation by using a trained neural network model technique taught by SATO as above, in order to provide improved performance of extracting an audio signal of a target speaker [See SATO; [0117 and 129]].
Regarding claim 3, KOIZUMI and SATO further disclose the distance estimation neural network is a convolutional neural network [e.g. KOIZUMI: FIG. 1-2; SATO: FIG. 1 and 11; [0134]; a convolution neural network (CNN)].
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the sound separation system disclosed by KOIZUMI to exploit the well-known providing sound separation by using a trained neural network model technique taught by SATO as above, in order to provide improved performance of extracting an audio signal of a target speaker [See SATO; [0117 and 129]].
Regarding claim 4, KOIZUMI and SATO further disclose the distance estimation neural network is a recurrent neural network [e.g. KOIZUMI: FIG. 1-2; [0022]; SATO: FIG. 1 and 11; [0134]; a recurrent neural network (RNN)].
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the sound separation system disclosed by KOIZUMI to exploit the well-known providing sound separation by using a trained neural network model technique taught by SATO as above, in order to provide improved performance of extracting an audio signal of a target speaker [See SATO; [0117 and 129]].
Regarding claim 5, KOIZUMI and SATO further disclose the recurrent neural network is a Long Short-Term Memory (LSTM) network [e.g. KOIZUMI: FIG. 1-2; SATO: FIG. 1 and 11; [0134]; a long short-term memory (LSTM)].
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the sound separation system disclosed by KOIZUMI to exploit the well-known providing sound separation by using a trained neural network model technique taught by SATO as above, in order to provide improved performance of extracting an audio signal of a target speaker [See SATO; [0117 and 129]].
Regarding claim 8, KOIZUMI and SATO further disclose separating human speech from ambient noise [e.g. KOIZUMI: FIG. 1-2; [0021 and 0095]; suppress noise signal; SATO: FIG. 1; background noise].
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the sound separation system disclosed by KOIZUMI to exploit the well-known providing sound separation by using a trained neural network model technique taught by SATO as above, in order to provide improved performance of extracting an audio signal of a target speaker [See SATO; [0117 and 129]].
Regarding claim 10, KOIZUMI and SATO further disclose determining a time-frequency mask [e.g. KOIZUMI: Estimation of Time-Frequency Mask]; and applying the time-frequency mask to the audio mixture to predict a signal comprising the human speech [e.g. KOIZUMI: FIG. 1-2 and 5].
Regarding claim 13, KOIZUMI and SATO further disclose suppressing, by the computing device, at least one of the one or more far sounds [e.g. KOIZUMI: FIG. 1-3; SATO: FIG. 1; extracting an audio signal of a speaker of interest from a mixed audio signal e.g. background noise].
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the sound separation system disclosed by KOIZUMI to exploit the well-known providing sound separation by using a trained neural network model technique taught by SATO as above, in order to provide improved performance of extracting an audio signal of a target speaker [See SATO; [0117 and 129]].
Regarding claim 14, KOIZUMI and SATO further disclose enhancing, by the computing device, at least one of the one or more near sounds [e.g. KOIZUMI: FIG. 1-3; SATO: FIG. 1; extracting an audio signal of a speaker of interest from a mixed audio signal e.g. background noise].
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the sound separation system disclosed by KOIZUMI to exploit the well-known providing sound separation by using a trained neural network model technique taught by SATO as above, in order to provide improved performance of extracting an audio signal of a target speaker [See SATO; [0117 and 129]].
Regarding claim 15, KOIZUMI and SATO further disclose providing, by the computing device, the predicted one or more far sounds [e.g. KOIZUMI: FIG. 1-3].
Regarding claim 18 and 24, this is method that includes same limitation as in claim 1 above, the rejection of which are incorporated herein. Furthermore, KOIZUMI and SATO disclose training, based on the training data and for an input audio mixture, a distance estimation neural network [e.g. KOIZUMI: FIG. 1-2 and 4; SATO: FIG. 2].
Regarding claim 20, KOIZUMI and SATO further disclose reproducing, by the acoustic simulator, one or more acoustic properties of component sounds in a given audio mixture of the plurality of audio mixtures [e.g. KOIZUMI: FIG. 1-2 and 4; SATO: FIG.1- 2].
Regarding claim 25, KOIZUMI and SATO further disclose the training of the neural network is performed at the computing device [e.g. KOIZUMI: FIG. 1-2 and 4; SATO: FIG. 2 and 14].
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the sound separation system disclosed by KOIZUMI to exploit the well-known providing sound separation by using a trained neural network model technique taught by SATO as above, in order to provide improved performance of extracting an audio signal of a target speaker [See SATO; [0117 and 129]].
Regarding claim 26, this is a device that includes same limitation as in claim 1 above, the rejection of which are incorporated herein.
Regarding claim 31, this is a device that includes same limitation as in claim 18 above, the rejection of which are incorporated herein.
Claim(s) 6-7, 19 and 21-23 is/are rejected under 35 U.S.C. 103 as being unpatentable over KOIZUMI et al (US 20210219048 A1) in view of SATO et al (US 20220335965 A1) and Robinson et al (US 11112389 B1).
Regarding claim 6, KOIZUMI and SATO further disclose the trained distance estimation neural network having been trained to predict the respective distances [e.g. KOIZUMI: FIG. 2; SATO: FIG. 1], but KOIZUMI and SATO fail to disclose the detail of the estimation.
However, Robinson teaches the well-known concept of determining a plurality of room impulse responses (RIRs) with an image method room simulator [e.g. FIG. 3-4; room impulse response generated by a simulation using a model of a room generated from image data]; and determining, based on the plurality of RIRs, the direct-to-reverberation ratio (DRR) [e.g. acoustic parameters for the room (e.g., room impulse response, reverberation time, direct to reverberant ratio, etc.].
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the sound separation system disclosed by KOIZUMI to exploit the well-known providing sound separation by using a trained neural network model technique taught by SATO and the well-known determining room acoustic parameters using image data technique taught by Robinson as above, in order to provide improved performance of extracting an audio signal of a target speaker [See SATO; [0117 and 129]] and improved model of the room or the simulation of sound propagation [See Robinson; column 1 lines 36-49].
Regarding claim 7, KOIZUMI, SATO and Robinson further disclose the determining of the plurality of RIRs is performed with frequency-dependent wall filters [e.g. Robinson: FIG. 3-6; frequency filtering caused by a reflection off the wall].
Regarding claim 19, this is method that includes same limitation as in claim 7 above, the rejection of which are incorporated herein.
Regarding claim 21, this is method that includes same limitation as in claim 6 above, the rejection of which are incorporated herein.
Regarding claim 22, KOIZUMI, SATO and Robinson further disclose a location of the audio input component in the room is randomized [e.g. KOIZUMI: sound source is randomly selected; FIG. 2 and 4; SATO: FIG. 2; Robinson: FIG. 3-6].
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the sound separation system disclosed by KOIZUMI to exploit the well-known providing sound separation by using a trained neural network model technique taught by SATO and the well-known determining room acoustic parameters using image data technique taught by Robinson as above, in order to provide improved performance of extracting an audio signal of a target speaker [See SATO; [0117 and 129]] and improved model of the room or the simulation of sound propagation [See Robinson; column 1 lines 36-49].
Regarding claim 23, KOIZUMI, SATO and Robinson further disclose a location of at least one sound source in the room is randomized [e.g. KOIZUMI: sound source is randomly selected; FIG. 2 and 4; SATO: FIG. 2; Robinson: FIG. 3-6].
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the sound separation system disclosed by KOIZUMI to exploit the well-known providing sound separation by using a trained neural network model technique taught by SATO and the well-known determining room acoustic parameters using image data technique taught by Robinson as above, in order to provide improved performance of extracting an audio signal of a target speaker [See SATO; [0117 and 129]] and improved model of the room or the simulation of sound propagation [See Robinson; column 1 lines 36-49].
Claim(s) 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over KOIZUMI et al (US 20210219048 A1) in view of SATO et al (US 20220335965 A1) and Chi et al (US 20220238128 A1).
Regarding claim 9, KOIZUMI and SATO further disclose performing the separating of the human speech from the ambient noise by using a neural network [e.g. KOIZUMI: FIG. 2; SATO: FIG. 1], but KOIZUMI and SATO fail to disclose the detail of the performing.
However, Chi teaches the well-known concept of the separating of the human speech from the ambient noise is performed by a TasNet model [e.g. FIG. 1; [0023]].
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the sound separation system disclosed by KOIZUMI to exploit the well-known providing sound separation by using a trained neural network model technique taught by SATO and the well-known determining neural network mode technique taught by Chi as above, in order to provide improved performance of extracting an audio signal of a target speaker [See SATO; [0117 and 129]] and improved performance for a neural network architecture of the audio processing [See Chi; [0004]].
Claim(s) 11-12 is/are rejected under 35 U.S.C. 103 as being unpatentable over KOIZUMI et al (US 20210219048 A1) in view of SATO et al (US 20220335965 A1) and Munoz et al (US 20210006925 A1).
Regarding claim 11, KOIZUMI and SATO further disclose receiving the threshold distance [e.g. KOIZUMI: [0015 and 0084]; each microphone to each near sound source is not more than 30 cm], but KOIZUMI and SATO fail to disclose the detail of receiving the threshold.
However, Munoz teaches the well-known concept of a user-adjustable control to receive the threshold distance [e.g. FIG. 1; user interface; source distance threshold defined or specified by the user].
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the sound separation system disclosed by KOIZUMI to exploit the well-known providing sound separation by using a trained neural network model technique taught by SATO and the well-known defining sound source distance threshold technique taught by Munoz as above, in order to provide improved performance of extracting an audio signal of a target speaker [See SATO; [0117 and 129]] and accurate localization of sound sources [See Munoz; [0030]].
Regarding claim 12, KOIZUMI, SATO and Munoz further disclose receiving, by the user-adjustable control, a particular threshold distance input by a user of the computing device, and wherein the providing of the predicted one or more near sounds and the one or more far sounds is based on the particular threshold distance [e.g. KOIZUMI: [0015 and 0084]; each microphone to each near sound source is not more than 30 cm; Munoz: user interface; source distance threshold defined or specified by the user ].
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the sound separation system disclosed by KOIZUMI to exploit the well-known providing sound separation by using a trained neural network model technique taught by SATO and the well-known defining sound source distance threshold technique taught by Munoz as above, in order to provide improved performance of extracting an audio signal of a target speaker [See SATO; [0117 and 129]] and accurate localization of sound sources [See Munoz; [0030]].
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
HIGUCHI et al (US 20200395037 A1).
Wichern et al (US 20210116894 A1).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ZHUBING REN whose telephone number is (571)272-2788. The examiner can normally be reached Monday-Friday 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Richemond Dorvil can be reached at 571-272-7602. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ZHUBING REN/Primary Examiner, Art Unit 2658
Prosecution Insights