DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statements (IDSs) submitted on 06/12/2024 and 08/06/2024 were filed in compliance with the provisions of 37 CFR 1.97and 1.98. Accordingly, the information disclosure statements are being considered by the examiner.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-10 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Bisani et al. (US 2015/0095026).
Regarding Claim 1, Bisani et al discloses a method of operating an audio device system comprising the steps of: a) providing a plurality of sound source signals each from a sound source of a present sound environment (The ASR device 100 may receive audio, including speech from the user and/or other audio from the dishwasher 122, stereo output 124, etc. from multiple directions, as shown in block 102) (page 1, paragraph [0018]); b) selecting a first sound signal comprising speech (When steerable, the microphone array may allow for electronic focusing, where data processing techniques may be employed to focus the microphone array on audio coming from a particular direction, known as a beam. Such steering may be used to isolate the user's speech by focusing the ASR processing on audio received from the direction of the user) (pages 6 and 7, paragraph [0060]); c) comparing the speech content of said first sound signal with the speech content of the provided plurality of sound source signals (When receiving audio for speech recognition processing, an ASR device may receive both desired audio, such as speech from a user, as well as undesired audio, such as audio from other people talking to each other, noise from household appliances, etc.) (pages 6 and 7, paragraph [0060]); d) selecting, based on said comparison, as output signal the sound source signal that most likely is part of a conversation that the audio device system user is paying attention to (A device may listen to audio from different directions, and process the audio using speech detection and/or speech recognition techniques. The device may then decide which direction includes audio that most closely corresponds to the desired utterance from the user, and focus on the speech recognition results/audio from that direction for further processing) (page 7, paragraph [0063]); and e) providing an audio output based on said output signal (Following ASR processing, the ASR results may be sent by the ASR module 314 to another component of the ASR device 302, such as the controller/processor 308 for further processing (such as execution of a command included in the interpreted text) or to the output device 307 for sending to an external device) (page 6, paragraph [0053]), wherein the contribution to the audio output from the remaining sound source signals is suppressed compared to the contribution from the output signal (If the direction of audio from the user is known, then the microphone array may be steered in the known direction. The audio from the desired location may then be processed to improve the quality of the desired audio. For example, a device may increase signal-to-noise ratio for an audio signal from the direction of the speaker. Other techniques such as reverberation, echo cancellation, etc., may also be used to improve the reception of the desired audio) (page 7, paragraph [0061]).
Regarding Claim 2, Bisani et al discloses the method, wherein the step of providing a plurality of sound source signals each from a sound source of a present sound environment comprises the further steps of: using an encoder-decoder (Fig. 3, encoder/decoder 322) neural network (Fig. 9, neural network) that has been obtained by feeding a mixed audio signal comprising a plurality of speech signals and a plurality of noise signals to the neural network and subsequently train the neural network to provide only said plurality of speech signals (Processed feature vectors may be output from the ASR module 314 and sent to the output device 407 for transmission to another device for further processing. The feature vectors may be encoded and/or compressed by the encoder/decoder 322 prior to transmission. The encoder/decoder 322 may be customized for encoding and decoding ASR data, such as digitized audio data, feature vectors, etc.) (page 3, paragraph [0032]), (Neural networks may also be used to perform ASR processing including acoustic model processing and language model processing) (page 6, paragraph [0054]), (The training corpus may be used to train the speech recognition models, including the acoustic models and language models, in advance. The models may then be used during ASR processing) (page 3, paragraph [0033]) ; or using a plurality of beam formers each adapted to point in a desired direction different from the other beam formers (In one aspect of the disclosure, an ASR device may be equipped with a microphone array and a beamformer, and the beamformer may output multiple channels of audio such that each channel isolates audio in a particular direction. The ASR device receives the multi-channel audio from the beamformer) (page 1, paragraph [0017]).
Regarding Claim 3, Bisani et al discloses the method, wherein the step of using a plurality of beam formers each adapted to point in a desired direction different from the other beam formers an ASR device may be equipped with a microphone array and a beamformer, and the beamformer may output multiple channels of audio such that each channel isolates audio in a particular direction. The ASR device receives the multi-channel audio from the beamformer) (page 1, paragraph [0017])comprise the further step of: determining that a beam former is pointing in a desired direction if speech is detected in the beam former output signal (To perform beamforming, a device may include components for direction-based processing, such as a microphone array and related components) (page 6 ns 7, paragraph [0060]).
Regarding Claim 4, Bisani et al discloses the method, wherein the step of selecting a first sound signal is carried out by at least one of the steps of: detecting a sound from a source that is positioned directly in front of the user (If the direction of audio from the user is known, then the microphone array may be steered in the known direction) (page 7, paragraph [0061]) or detecting a sound from a source that the user is looking at or detecting the audio device system users own voice or by identifying the sound source signal that exhibits the highest similarity with an EEG signal of the user (If the direction of the audio from the user is unknown, however, a number of techniques and considerations may be used to determine the direction of the desired speech. For example, audio from the desired user may have a higher intensity at a particular location of the ASR device)n (page 7, paragraph [0061]); and by subsequently carrying out at least one of the steps of: selecting the first sound signal from at least one of said detected sounds in response to a predetermined interaction between the user and the audio device system wherein said predetermined interaction comprises at least one of the interactions: making a specific head movement, tapping an audio device of the audio device system, operating an audio device control means, speaking a control word (The confidence score may be based on a number of factors including, for example, the similarity of the sound/audio from the different directions to models for language sounds (e.g., an acoustic model), the likelihood that a particular word that matches the audio would be included in the sentence at the specific location (e.g., using a language or grammar model)) (page 7, paragraph [0065]) and operating a graphical user interface of the audio device system
Regarding Claim 5, Bisani et al discloses the method, wherein said step of comparing the speech content of said first sound signal with the speech content of the provided plurality of sound source signals comprises at least one of: assigning a numerical representation to at least some of the words comprised in the first sound signal and in said plurality of provided sound source signals and providing a word embedding similarity measure in order to estimate the similarity between the first sound signal and said plurality of provided sound source signals; and determining the timing of speech ending for the first sound signal and determining timing of speech onset for said plurality of provided sound source signals and subsequently identifying at least one sound source signals with speech onset within a predetermined duration after speech ending for the first sound signal; assigning a numerical representation to at least one of syntactic and semantic information comprised in the first sound signal and in said plurality of provided sound source signals; and providing at least one of a syntactic similarity and a semantic similarity between the first sound signal and said plurality of provided sound source signals (The confidence score may be based on a number of factors including, for example, the similarity of the sound/audio from the different directions to models for language sounds (e.g., an acoustic model), the likelihood that a particular word that matches the audio would be included in the sentence at the specific location (e.g., using a language or grammar model)) (page 7, paragraph [0065]).
Regarding Claim 6, Bisani et al discloses the method, wherein the step of selecting, based on said comparison, as output signal the sound source signal that the audio device system user is most likely paying attention to comprises at least one of the steps of: selecting as output signal the sound source signal having the word embedding similarity measure that is most similar with the word embedding similarity measure of the first sound source signal; selecting as output signal the sound source signal having a speech onset within a predetermined duration after speech ending of the first sound signal; selecting as output signal the sound source signal having the highest score of at least one of the semantic similarity measure and the syntactic similarity measure (The confidence score may be based on a number of factors including, for example, the similarity of the sound/audio from the different directions to models for language sounds (e.g., an acoustic model), the likelihood that a particular word that matches the audio would be included in the sentence at the specific location (e.g., using a language or grammar model)) (page 7, paragraph [0065]); and selecting as output signal the sound source signal having a highest combined score, wherein the combined score is obtained by combining at least some of: the word embedding similarity measure score, the semantic similarity measure, the syntactic similarity measure, a sound pressure level score reflecting the strength of the signal, a previous participant score reflecting whether the speaker representing the sound source signal has previously participated in the conversation and having a speech onset within said predetermined duration after speech ending of the first sound signal.
Regarding Claim 7, Bisani et al discloses the method, wherein the step of providing an audio output based on said output signal, wherein the contribution to the audio output from the remaining sound source signals is suppressed compared to the contribution from the output signal, comprises at least one of the steps of: suppressing the contribution to the audio output from the remaining sound source signals such that the combined level of the remaining sound source signals is in the range between 3 and 24 dB or between 6 and 18 dB below the output signal level; enabling the user to control the ratio between the output signal level and the combined level of the remaining sound source signals (For example, a device may increase signal-to-noise ratio for an audio signal from the direction of the speaker. Other techniques such as reverberation, echo cancellation, etc., may also be used to improve the reception of the desired audio) (page 7, paragraph [0061]).
Regarding Claim 8, Bisani et al discloses the method, comprising the further steps of:- replacing the current first sound signal with the current output signal; and - triggering that steps c), d) and e) are carried out in response to a detection of said output signal reaching a speech ending (Each phoneme may be represented by multiple potential states corresponding to different known pronunciations of the phonemes and their parts (such as the beginning, middle, and end of a spoken language sound)) (page 4, paragraph [0038]).
Regarding Claim 9, Bisani et al discloses the method, comprising the further step of:- triggering that steps c), d) and e) are carried out in response to a detection of said output signal reaching a speech ending (Each phoneme may be represented by multiple potential states corresponding to different known pronunciations of the phonemes and their parts (such as the beginning, middle, and end of a spoken language sound)) (page 4, paragraph [0038]).
Regarding Claim 9, Bisani et al discloses an audio device system comprising at least one audio device, wherein said at least one audio device comprises an acoustical-electrical input transducer (the speech recognition engine 318 may use a finite state transducer (FST ) (page 5, paragraph [0050]) and an electrical-acoustical output transducer (each speech recognition result simultaneously obtained for each of the different directions may be implemented in a finite state transducer (FST)) (pages 7 and 8, paragraph [0068]), and wherein said audio device system further comprises a sound source signal separator adapted to receive an input signal from said acoustical electrical input transducer and to provide a plurality of sound source signals each representing a sound source of a present sound environment (The ASR device 100 may receive audio, including speech from the user and/or other audio from the dishwasher 122, stereo output 124, etc. from multiple directions, as shown in block 102) (page 1, paragraph [0018]); a first sound signal selector adapted to enable a user to select a first sound signal comprising speech, wherein said first sound signal is based on said audio input signal (When steerable, the microphone array may allow for electronic focusing, where data processing techniques may be employed to focus the microphone array on audio coming from a particular direction, known as a beam. Such steering may be used to isolate the user's speech by focusing the ASR processing on audio received from the direction of the user) (pages 6 and 7, paragraph [0060]); a speech content comparator adapted to compare the speech content of said first sound signal with the speech content of the provided plurality of sound source signals (When receiving audio for speech recognition processing, an ASR device may receive both desired audio, such as speech from a user, as well as undesired audio, such as audio from other people talking to each other, noise from household appliances, etc.) (pages 6 and 7, paragraph [0060]), and adapted to select, based on said comparison, as output signal the sound source signal that most likely is part of a conversation that the audio device system user is paying attention to (A device may listen to audio from different directions, and process the audio using speech detection and/or speech recognition techniques. The device may then decide which direction includes audio that most closely corresponds to the desired utterance from the user, and focus on the speech recognition results/audio from that direction for further processing) (page 7, paragraph [0063]); and a digital signal processor adapted to process the output signal, such that the contribution to the audio output from the remaining sound source signals is suppressed compared to the contribution from the output signal (If the direction of audio from the user is known, then the microphone array may be steered in the known direction. The audio from the desired location may then be processed to improve the quality of the desired audio. For example, a device may increase signal-to-noise ratio for an audio signal from the direction of the speaker. Other techniques such as reverberation, echo cancellation, etc., may also be used to improve the reception of the desired audio) (page 7, paragraph [0061]); wherein said processed output signal is provided to the electrical-acoustical output transducer in order to provide the audio output, and wherein the acoustical-electrical input transducer provides an input signal representing the present sound environment and provides the input signal to the sound source signal separator and to the first sound signal selector (each speech recognition result simultaneously obtained for each of the different directions may be implemented in a finite state transducer (FST). In this aspect, a decoding graph represents the results of the simultaneous performance of speech recognition in the different directions) (pages 7 and 8, paragraph [0068]).
Cited Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Zhang et al. (US 8,612,217) discloses noise reduction techniques.
Nakadai et al. (US 2009/0030552) discloses robotics visual and auditory system.
Bryan et al. (US 2019/0066710) discloses transparent near-end user control over far-end speech enhancement processing.
Bryan et al. (US 2019/0156847) discloses transparent near-end user control over far-end speech enhancement processing.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SATWANT K SINGH whose telephone number is (571)272-7468. The examiner can normally be reached Monday thru Friday 9:00 AM to 6:00 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Paras D Shah can be reached at (571}270-1650. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SATWANT K SINGH/Primary Examiner, Art Unit 2653