DETAILED ACTION
In the response to this office action, the examiner respectfully requests that support be shown for language added to any original claims on amendment and any new claims. That is, indicate support for newly added claim language by specifically pointing to page(s) and line numbers in the specification and/or drawing figure(s). This will assist the examiner in prosecuting this application.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Drawings
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because they do not include the following reference sign(s) mentioned in the description:
Paragraph [0043] states “an equalizer 115 may adjust or filter the frequencies of the audio signals”, however no item 115 is found in the figures.
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Objections
Claim 1-20 are objected to because of the following informalities:
Claim 1 states “generating localization map data indicating locations of the two or more audio sources relative to microphones providing the binaural audio signals and comprising inputting at least one version of the binaural audio signals into at least one neural network (NN)” which is not proper grammar. This would be better as “generating localization map data indicating locations of the two or more audio sources relative to microphones providing the binaural audio signals, the generating comprising inputting at least one version of the binaural audio signals into at least one neural network (NN)”.
Claim 15 is objected in an analogous manner.
Claims 2-10 and 16-20 are objected as inheriting the problems as above.
Claim 1 states “training a neural network (NN) comprising inputting at least one version of the binaural audio signals into the NN, outputting output localization map data indicating locations of the two or more audio sources relative to microphones providing the binaural audio signals, and” which is not proper grammar. This would be better as “training a neural network (NN) comprising inputting at least one version of the binaural audio signals into the NN, and outputting output localization map data indicating locations of the two or more audio sources relative to microphones providing the binaural audio signals, and”.
Claims 12-14 are objected as inheriting the problems as above.
Claim 20 states “wherein locations of audio sources on the localization map data have an average error of ten degrees” which would make more sense as “wherein locations of audio sources on the localization map data have an average error of less than ten degrees“ since there is no disclosure on how to hit an exact number for the average error.
Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is not directed to patent eligible subject matter. Based upon consideration of all of the relevant factors with respect to the claim as a whole, claim(s) 1-20 is/are determined to be directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more. In this instance the claims are to an abstract idea. The rationale for this determination is explained below:
Regarding method claim 1, the claim is to an abstract idea of a mathematical algorithm. The mathematical algorithm including generating localization map data indicating locations of the two or more audio sources relative to microphones providing binaural audio signals and comprising inputting at least one version of the binaural audio signals into at least one neural network (NN) as explained below. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional steps provided do not add a meaningful limitation to the method, they merely involve a data gathering. Also, stating the method is “computer implemented” only . The recited computer is only mentioned at a high level of generality and a general purpose computer implementing an abstract idea is still not patentable eligible subject matter. Thus, taken alone, the additional elements do not amount to significantly more than the above-identified judicial exception (the abstract idea).
Claim 1 recites, in part, a method of receiving binaural signals and generating a map indicating locations of audio sources. This generating step including inputting the binaural signals into a neural network. In total, this is a formula encoding audio signals into mapped values (locations). These steps describe the concept of a mathematical formula similar to mathematical calculations such as calculating the difference between local and average data values which correspond to concepts identified as abstract ideas by the courts, see In re Abele. All of these concepts relate to practices in which information is manipulated and calculation are made through mathematical correlations. The concept described in claim 1 is not meaningfully different than those mathematical calculations found by the courts to be abstract ideas. As such, the description in claim 1 of performing mathematical calculations based on received audio signals is an abstract idea.
Dependent claims 2-10, when analyzed as a whole are held to be patent ineligible under 35 USC 101 because the additional recited limitations fail to establish that the claims are not directed to an abstract idea. Claims 2-10 only further defined the calculations made.
Regarding system claims 15-20, the claims are rejected in an analogous manner as including the abstract idea of claims 1-10 as above. The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements when considered both individually and as an ordered combination do not amount to significantly more than the abstract idea. The recited devices, are general use components that make up a general purpose computer and a general purpose computer implementing an abstract idea is still not patentable eligible subject matter. The additional limitations are recited at a high level of generality and are recited as performing generic computer functions routinely used in computer applications. Generic computer components recited as performing generic computer functions that are well-understood, routine and conventional activities amount to no more than implementing the abstract idea with a computerized system. The use of generic computer components to mathematically process audio signals does not impose any meaningful limit on the computer implementation of the abstract idea. Thus, taken alone, the additional elements do not amount to significantly more than the above-identified judicial exception (the abstract idea). Looking at the limitations as an ordered combination adds nothing that is not already present when looking at the elements taken individually. There is no indication that the combination of elements improves the functioning of a computer or improves any other technology. Their collective functions merely provide conventional computer implementation.
Similar to claim 1, claim 11 recites, in part, a method of receiving binaural signals and training a neural network to generate a map indicating locations of audio sources. In total, this is a formula encoding audio signals into mapped values (locations) with an additional learning formula applied. The recited computer readable media is only mentioned at a high level of generality and a general purpose computer components implementing an abstract idea is still not patentable eligible subject matter. Thus, taken alone, the additional elements do not amount to significantly more than the above-identified judicial exception (the abstract idea).
Dependent claims 12-14, when analyzed as a whole are held to be patent ineligible under 35 USC 101 because the additional recited limitations fail to establish that the claims are not directed to an abstract idea. Claims 12-14 only further defined the calculations made.
Claims 1-20 are therefore not drawn to eligible subject matter as they are directed to an abstract idea without significantly more.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 3, 6, 8-12, and 14-16 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Francl et al. (“Deep neural network models of sound localization reveal how perception is adapted to real-world environments”, Nature Human Behaviour, volume 6, pp. 111-133, 2022)(see IDS dated 06/27/23)
Regarding claim 1, Francl discloses a computer-implemented (see at least mention of supercomputer and Gigabyte limit on page 126 and other hints throughout) method of audio processing, comprising:
receiving, by processor circuitry, binaural audio signals at least overlapping at a same time and of a same two or more audio sources (see figure 1 and description on page 112, specifically figure 1C, “Natural Sound” are the sound sources, see section “Natural sound sources” page 123, figure 1C bottom left shows simultaneous left and right signals); and
generating localization map data indicating locations of the two or more audio sources relative to microphones (see figures 1E and 1F for real-world test) providing the binaural audio signals (Output of figure 1C, azimuth and elevation) and comprising inputting at least one version of the binaural audio signals into at least one neural network (NN) (bottom right figure 1C and figure 1D).
Regarding claim 3, Francl discloses wherein the at least one version of the binaural audio signals are the only audio signals input to the NN (figure 1C or figure 1D, no mention of any other audio signals being used).
Regarding claim 6, Francl discloses wherein the NN comprises a time domain encoder comprising a sequence of convolutional encoder blocks (see “Neural Network Models” section starting page 124).
Regarding claim 8, Francl discloses wherein the NN comprises a frequency domain encoder (may be considered encoding inputs) comprising a series of fully connected layers (see “Neural Network Models” section starting page 124).
Regarding claim 9, Francl discloses wherein the NN comprises a decoder (may be considered decoding outputs) comprising a series of fully connected layers (see “Neural Network Models” section starting page 124).
Regarding claim 10, Francl discloses wherein the NN is trained by using at least two overlapping audio sources (figure 1C bottom left shows simultaneous left and right signals).
Regarding claim 11, Francl discloses at least one non-transitory computer readable medium comprising a plurality of instructions that in response to being executed on a computing device (see at least mention of supercomputer and Gigabyte limit on page 126 and other hints throughout), causes the computing device to operate by:
receiving, by processor circuitry, binaural audio signals at least overlapping at a same time and of a same two or more audio sources (see figure 1 and description on page 112, specifically figure 1C, “Natural Sound” are the sound sources, see section “Natural sound sources” page 123, figure 1C bottom left shows simultaneous left and right signals); and
training a neural network (NN) comprising inputting at least one version of the binaural audio signals into the NN (bottom of figure 1C), outputting output localization map data indicating locations of the two or more audio sources relative to microphones (see figures 1E and 1F for real-world test) providing the binaural audio signals (Output of figure 1C, azimuth and elevation), and
comparing a version of the output localization map data to a version of ground truth localization map data (see figure 1 description, see figures 2A and 2B for real-world test, see “Model construction” section for training explanation, see section “Model Evaluation in Real-World Conditions” page 114, “performance on the real-world test set demonstrates that training a neural network in a virtual world produces a model that can accurately localize sounds in realistic conditions” of page 114, among others).
Regarding claim 12, Francl discloses wherein the training comprises generating binaural audio signals with audio of simultaneous audio sources each at different randomly selected angles relative to a location of the microphones (“rendered them at a large set of randomly selected locations”, page 119).
Regarding claim 14, Francl discloses wherein the training comprises minimizing a loss comprising using binary mask map data as ground truth localization map data (see figure 2, evaluated versus real world data, see figures 1G and 1H for instance, see section “Real-World Evaluation” starting page 126).
Regarding claim 15, Francl discloses computer-implemented system (see at least mention of supercomputer and Gigabyte limit on page 126 and other hints throughout), comprising:
memory to hold binaural audio signals (required to store signals of bottom left of figure 1C), wherein the binaural audio signals at least overlap in time and are associated with a same at least two audio sources (see figure 1 and description on page 112, specifically figure 1C, “Natural Sound” are the sound sources, see section “Natural sound sources” page 123, figure 1C bottom left shows simultaneous left and right signals); and
processor circuitry communicatively connected to the memory (see at least mention of supercomputer and Gigabyte limit on page 126 and other hints throughout), the processor circuitry being arranged to operate by:
generating localization map data indicating locations of the at least two audio sources relative to microphones (see figures 1E and 1F for real-world test) providing the binaural audio signals (Output of figure 1C, azimuth and elevation) and comprising inputting at least one version of the binaural audio signals into at least one neural network (NN)(bottom right figure 1C and figure 1D).
Regarding claim 16, Francl discloses wherein the localization map data provides data for a location of the at least two audio sources being in any direction relative to a location of the microphones (Output of figure 1C, azimuth and elevation).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 18 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Francl et al. (“Deep neural network models of sound localization reveal how perception is adapted to real-world environments”, Nature Human Behaviour, volume 6, pp. 111-133, 2022)(see IDS dated 06/27/23).
Regarding claim 18, although Francl does not expressly disclose earbuds or glasses, the examiner takes official notice that both earbuds and glasses with microphones near the ear canal (noise cancelling microphones for example) were well known in the art. Therefore, it would have been obvious to one of ordinary skill in the art to further comprise wherein the microphones are on earbuds or glasses comprising microphones arranged to be held within at most 3 inches from an opening of an ear canal in the system of Francl for the benefit of reusing existing microphones already arranged close to human ear drums.
Regarding claim 20, although Francl does not expressly disclose the average error rate, it would have been obvious to the designer that a more highly trained NN would have a lower error rate, at the cost of more training, and that the designer may set that threshold at any level, at their preference. Therefore, it would have been obvious to one of ordinary skill in the art to further comprise wherein locations of audio sources on the localization map data have an average error of ten degrees in the system of Francl for the benefit of having a fairly close location estimate, while not requiring the highest amount of training.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DOUGLAS JOHN SUTHERS whose telephone number is (571)272-0563. The examiner can normally be reached M-F, 8 am -5 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vivian Chin can be reached at 571-272-7848. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DOUGLAS J SUTHERS/ Examiner, Art Unit 2695
/VIVIAN C CHIN/ Supervisory Patent Examiner, Art Unit 2695
Prosecution Insights