Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1,2,4,9-14,17-20 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Huang et al (WO2022231977) .
Regarding claim 1, Huang et al (WO2022231977) discloses a method of discerning and reproducing speech, comprising:
separately capturing sound conducted through air in an environment in which the user is situated, and through bone of a user while the user is speaking to produce, as a preliminary audio output, channels of streams of audio signals, wherein the preliminary audio output has a low signal-to-noise ratio (SNR) (para [0072]-[0075] "one or more external microphones may be beneficial in capturing a user's full- band speech; however, noisy environments significantly hinder use of such microphones. Accordingly, to provide optimal audio quality while maintaining intelligibility in noisy environments, speech captured by both in-ear microphone(s) and external microphone(s) may be processed to produce an output signal with a high SNR and increased intelligibility. Further, an external microphone of the wearable audio device may better inform predictions of high-frequency band information by the speech enhancement deep learning model 412 to enhance audio quality of voice an in-ear microphone 218 may be configured to capture audio signals, e.g., bone and tissue conducted speech, in an ear canal of a user while one or more external microphones 122 may be configured to capture at least one external signal, e.g., air conducted speech Adaptive canceller 508 may perform similar functions as adaptive canceller 408 in FIG. 4; however, one or more domain converted external signals from one or more external microphones 222 may be used in calculating filter coefficients used to eliminate echo, reverberation, and/or noise. Accordingly, the noise reduced signal may have an increased SNR");
“transforming the streams of audio signals constituting the preliminary audio output and extracting features from a resulting transform of the audio signals” (para [0072]-[0080] "an external processing system 516 to process one or more external signals captured by one or more external microphones 222. External processing system 516 may include a null beamformer, such as a delay and subtract (D&S) beamformer. The D&S beamformer may time align and equalize the two external microphone to mouth direction signals and subtract to provide a noise correlated reference signal. D&S beamformer may be used to null out speech captured by one or more external microphones 222 and isolate only noise signal. Isolated noise signal may be fed to STSA speech enhancement system 510 which uses the noise correlated signal as a reference in performing spectral subtraction to remove noise from a mixed signal... a noise correlated reference signal may be an input to STSA speech enhancement system 510 in cases where ANR is triggered to remove superfluous noise captured as a result of the wearable audio output device being located in a noisy environment. STSA speech enhancement system 510 may produce an output signal (e.g., noise reduced audio signal) with an improved SNR the noise reduced audio signal may be further processed in accordance with a speech enhancement deep learning model 412 to restore high frequencies lost when using an in-ear microphone to capture speech. Further, the output signal from speech enhancement deep learning model 512 may be passed through inverse domain converter 520 to convert the signal from the frequency domain to the time domain such that the audio may be output for communication. The recovered audio output, after processing through implementation 500, may have an improved SNR and a dynamic range greater than a dynamic range of the speech captured by in-ear microphone 218. Accordingly, a wearable audio output device comprising both an external microphone and an in-ear microphone may be an ideal implementation to overcome the shortcomings of each microphone being used in isolation to capture speech");
“denoising the preliminary audio output to produce a processed signal having a higher SNR than that of the preliminary audio output” see (para [0060] "Adaptive canceller 408 may apply these coefficients to internal audio signal(s) to eliminate echo, reverberation, and/or noise. The noise reduced internal signal produced by adaptive canceller 408 may have a high SNR due to an occlusion boost of the voice signal in the ear canal and the cancellation of noise using calculated and/or preloaded coefficients"; para [0072]-[0080]),
“the denoising comprising inputting the extracted features to a statistical model or a neural network (para [0079]); and generating humanly perceptible output, which expresses the speech of the user, from the processed signal” see (para [0067] "in addition to restoring high frequencies of the audio signal captured by in-ear microphone speech enhancement deep learning model 412 may also be used to improve audio quality of low frequencies of the audio signal. For example, unnaturalness in lower frequencies of bone and tissue conducted speech captured by in-ear microphone 118 may occur where phonemes become exaggerated. Speech enhancement deep learning model 412 may be used to correct such distortions in these lower frequencies. Weights in the network (e.g., operating in time-domain or frequency-domain) may learn to modulate both high frequency and low frequency signals to match signals of a reference microphone. For example, the network may receive a muffled sound (e.g., "shhhh") and attempt to translate/encode the sound to an intermediate representation (e.g., <sh>); The network may then decode the intermediate representation into a more natural sounding audio signal (e.g., more natural sounding "shhhh") to be used for output. In the process of decoding the intermediate representation back to an audio signal, the network may predict how the intermediate representation may sound in both the high frequency domain and the low frequency domain.").
As per claim 2, Huang et al (WO2022231977) teaches the method as claimed in claim 1, wherein the respective channels are time synchronized (para [0065] "a network operating in time-domain may receive a window of audio stream (or multiple windows of audio streams where multiple microphone inputs are being used, such as when both in-ear and external microphones are used) and use this audio stream to learn an ideal mapping between low-frequency bands and high-frequency bands in audio signals. In such a case, domain converter 404 may be configured to translate the time-domain audio signals into frequency domain audio signals, and vice versa.").
As per claim 4, Huang et al (WO2022231977) teaches the method as claimed in claim 1, “wherein the transforming comprises transforming the audio signals constituting the preliminary audio output to a frequency domain representation of the preliminary audio output” -- see (para [0065] "a network operating in time-domain may receive a window of audio stream (or multiple windows of audio streams where multiple microphone inputs are being used, such as when both in-ear and external microphones are used) and use this audio stream to learn an ideal mapping between low-frequency bands and high-frequency bands in audio signals. In such a case, domain converter 404 may be configured to translate the time-domain audio signals into frequency domain audio signals, and vice versa"),
and the features extracted are frequency components of the channels (para [0065]).
As per claim 9, Huang et al (WO2022231977) teaches a method of discerning and reproducing speech, comprising: capturing sounds on multiple sensors of a device worn by a user while the user is speaking (para [0008] "the wearable audio output device may include an in-ear microphone acoustically coupled to an environment inside an ear canal of a user, and in some cases, additionally, an external microphone acoustically coupled to an environment outside the ear canal of the user."; para [0044] "Sound waves generated by a user's vocal chords and modulated by the user's vocal tract may be received by in-ear microphone 118 through the ear canal of the user"), the sounds including sound conducted through air in an environment in which the user is situated and speech of the user conducted through bone of the user to one of the sensors, wherein the sensors produce, as a preliminary audio output, channels of streams of audio signals, and the preliminary audio output has a low signal-to-noise ratio (SNR) (para [0072]-[0075]); transform-ing the streams of audio signals constituting the preliminary audio output and extracting features from a resulting transform of the audio signals (para [0072]-[0080]); denoising the preliminary audio output to produce a processed signal having a higher SNR than that of the preliminary audio output (para [0060]; para [0072]-[0080]), the denoising comprising inputting the extracted features to a statistical model or a neural network (para [0079]); and generating humanly perceptible output, which expresses the speech of the user, from the processed signal (para [0067]).
As per claim 10, Huang et al (WO2022231977) teaches the speech recognition method as claimed in claim 9, wherein the capturing of sounds comprises capturing voiced speech of the user conducted through the air with a first microphone of the device which is oriented towards the user's mouth (para [0075] ''External processing system 516 may include ....(D&S) beamformer. The D&S beamformer may time align and equalize the two external microphone to mouth direction signals and subtract to provide a noise correlated reference signal...the D&S beamformer may be used to null out speech captured by one or more external microphones 222 and isolate only noise signal."), capturing background noise in the environment with a second microphone of the device (para [0072] ''to provide optimal audio quality while maintaining intelligibility in noisy environments, speech captured by both in-ear microphone(s) and external microphone(s) may be processed to produce an output signal with a high SNR and increased intelligibility''),
“and capturing the speech of the user conducted through bone of the user with a third microphone, of the device which is acoustically isolated from the environment while pressed up against skin of the user” – see (para [0043] ''Each earbud 100 may further include at least one in-ear microphone 118 disposed within cavity 112. In implementations where wearable audio output device 10 is ear-mountable, an ear coupling 114 (e.g., an ear tip or ear cushion) may be attached to the casing 110 and surround an opening to the cavity 112. A passage 116 may be formed through the ear coupling 114 and communicate with the opening to the cavity 112. Accordingly, the in-ear microphone 118 may be acoustically coupled to an environment inside an ear canal of a user of the wearable audio output device 10."; para [0073] ''an in-ear microphone 218 may be configured to capture audio signals, e.g., bone and tissue conducted speech, in an ear canal of a user while one or more external microphones 122 may be configured to capture at least one external signal, e.g., air conducted speech''; fig. 2),
“and wherein the respective audio channels produced include audio signals representing background noise and voiced speech, respectively” -- see (para [0058] ''internal processing system 406 includes an adaptive canceller 408.....adaptive canceller 408 may clean and filter domain converted sidetone reference 402 and domain converted audio signal from in-ear microphone 118 to produce a single output for short-time spectral amplitude (STSA) speech enhancement system 410. The output may be a noise reduced internal signal."; para [0075]-[0077] ''an external processing system 516 to process one or more external signals captured by one or more external microphones 222....0utput from external processing system 516 and output from internal processing system 506 may be combined at intelligent mixer 518 to produce a mixed signal.").
As per claim 11, Huang et al (WO2022231977) discloses the method as claimed in claim 9, wherein the respective channels are time synchronized (para [0065]).
As per claim 12, Huang et al (WO2022231977) teaches the method as claimed in claim 9, wherein the multichannel audio signal additionally comprises a reference channel which is a drive signal being fed to an audio output device (para [0067]).
As per claim 13, Huang et al (WO2022231977) teaches the method as claimed in claim 9, Huang et al (WO2022231977) further discloses wherein the transforming comprises transforming the audio signals constituting the preliminary audio output to a frequency domain representation of the preliminary audio output(para [0065]), and the features that are extracted are frequency components of the channels (para [0065]).
As per claim 14, Huang et al (WO2022231977) teaches the method as claimed in claim 9, “wherein the denoising comprises producing the processed signal directly in response to the features being input to a trained statistical model” see (para [0012] ''the method further comprises processing the audio signal using active noise reduction (ANR) to produce a noise reduced signal, wherein the noise reduced signal is generated in response to the external signal and has a third frequency band; predicting high-frequency band information for the noise reduced signal using the trained model; and wherein the output signal is based, at least in part, on the third frequency band of the noise reduced signal and the predicted high-frequency band information for the noise reduced signal. In certain aspects, processing the audio signal using ANR to produce a noise reduced signal comprises calculating a set of noise cancellation parameters in response to the external signal and utilizing the set of noise cancellation parameters to process the audio signal."; para [0017] ''in order to predict high-frequency band information for the audio signal using the model trained using training data of known high-frequency bands associated with low-frequency bands, the memory further includes instructions executable by the at least one processor to cause the wearable audio output device to: extract low-frequency band information of the first frequency band; and select the high- frequency band information based at least in part on a mapping between the low- frequency band information and the high-frequency band information in the trained model.").
Regarding claim 17, Huang et al (WO2022231977) teaches a system for use in discerning and reproducing speech, comprising:
“multiple sensors constituting a wearable device and operative to capture sounds including sound conducted through air in an environment in which a user wearing the device is situated and speech of the user conducted through bone of the user to one of the sensors to produce, as a preliminary audio output, channels of streams of audio signals, and wherein the preliminary audio output has a low signal-to- noise ratio (SNR)” see- (para [0008]; para [0044];para [0072]-[0075]); and
“a computer system configured to receive the channels of audio signals from the wearable device and comprising a processing unit, and non-transitory computer-readable media (CRM) storing operating instructions (in Huang et al (WO2022231977) see claim 19 ''The computer-readable medium of claim 17, wherein predicting high-frequency band information for the audio signal using the model trained using training data of known high-frequency bands associated with low-frequency bands comprises: extracting low-frequency band information of the first frequency band; and selecting the high-frequency band information based at least in part on a mapping between the low-frequency band information and the high-frequency band information in the trained model."), the processing unit having a denoising module comprising a statistical model or a neural network and configured to execute the operating instructions to: transform the streams of audio signals constituting the preliminary audio output and extract features from a resulting transform of the audio signals, and denoise the preliminary audio output to produce a processed signal having a higher SNR than that of the preliminary audio output, the denoising comprising inputting the extracted features to the statistical model or neural network (para [0060]; para [0067]; para [0072]-[0080]).
As per claim 18, Huang et al (WO2022231977) teaches the system as claimed in claim 17, “comprising an earbud as the wearable device, and wherein the multiple sensors include an in-ear microphone occluding an ear canal of the user when the earbud is worn by the user so as to capture speech of the user conducted through bone of the ear of the user, and a second micro-phone oriented to capture external sound conducted through the air towards the earbud” see (para [0044] ''Sound waves generated by a user's vocal chords and modulated by the user's vocal tract may be received by in-ear microphone 118 through the ear canal of the user. Because each earbud 100 fills, or otherwise blocks, the outer portion of the user's ear canal, bone-conducted sound vibrations of a person's own voice in the space between a tip of the ear mold and the user's eardrum may cause voice captured by microphone 118 to be muffled. This phenomenon is known as the occlusion effect''; fig. 2).
As per claim 19, Huang et al (WO2022231977) teaches the system as claimed in claim 17, “wherein the processing unit is configured to execute the operating instructions to transform the audio signals constituting the preliminary audio output to a frequency domain representation of the preliminary audio output, and extract frequency components of the channels as said features” (para [0065]).
As per claim 20, Huang et al (WO2022231977) teaches the system as claimed in claim 17, “wherein the processing unit is configured to execute the operating instructions to further generate humanly perceptible output, which expresses the bone-conducted speech, from the processed signal” -- see (para [0067]).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 3,15 are rejected under 35 U.S.C. 103 as being unpatentable over Huang et al (WO2022231977) in view of Robben et al (20230326474).
As per claim 3, Huang et al (WO2022231977) teaches the method as claimed in claim 1, Huang et al (WO2022231977) further discloses wherein the capturing of sound conducted through the air comprises separately capturing voiced speech of the user conducted through the air and background noise in the environment (para [0073] ''external microphones 122 may be configured to capture at least one external signal, e.g., air conducted speech."). Huang et al (WO2022231977) does not explicitly teach: “whereby preliminary audio output is constituted by at least three channels of streams of audio signals”. However, Robben et al (20230326474) teaches, “whereby preliminary audio output is constituted by at least three channels of streams of audio signals” -- see (para [0102] ''the audio signals produced by these external air conduction sensors 13 may be combined beforehand (e.g. beamforming) to produce the third audio signal to be mixed with the audio signals produced by the bone conduction sensor(s) 11 and by the internal air conduction sensor(s) 12. Accordingly, in the present disclosure, the third audio signal may be produced by one or more external air conduction sensors 13. Similarly, the first audio signal may be produced by one or more bone conduction sensors 11 and the second audio signal may be produced by one or more internal air conduction sensors 12."). Therefore, it would have been obvious to one of ordinary skill in the art to modify the audio processing found in Huang et al (WO2022231977), to include the three streams of audio signals, as taught by Robben et al (20230326474), because it allows third audio signal to be mixed with the audio signals produced by the bone conduction sensor(s) and by the internal air conduction sensor(s) (Robben et al (20230326474), para 0102).
As per claim 15, Huang et al (WO2022231977) teaches the method as claimed in claim 9, Huang et al (WO2022231977) further discloses wherein the denoising comprises in response to the extracted features being input to a trained statistical model (para [0058]-[0059] ''adaptive canceller 408 may clean and filter domain converted sidetone reference 402 and domain converted audio signal from in-ear microphone 118 to produce a single output for short-time spectral amplitude (STSA) speech enhancement system 410. The output may be a noise reduced internal signal.....adaptive canceller 408 may be preloaded with noise reduction parameters (e.g., predetermined filter coefficients) to be applied to internal audio signal(s) to eliminate echo, reverberation, and/or noise....adaptive canceller 408 calculates noise reduction parameters (e.g., filter coefficients) based on external signal(s) and applies the parameters to internal audio signals (e.g., audio signals captured by
in-ear microphone 118). Adaptive canceller 408 may adaptively determine filter coefficients, during periods where no voice signal is detected (e.g., via a voice activity detector (VAD)), using any well-known adaptive algorithms, such as the normalized leas means square (NLMS) algorithm''; para [0062]-[0064] ''the noise reduced audio signal may be further processed in accordance with a speech enhancement deep learning model 412. The speech enhancement deep learning model...the speech enhancement deep learning model 412 may be used to predict high frequency band information for the noise reduced signal. High frequencies lost in audio signals captured by in-ear microphones may be restored using the predictive high frequency band information ....The trained model may provide mapping between low-frequency bands and high-frequency bands in audio signals.").
Huang et al (WO2022231977) does not explicitly teach producing filter parameters to the applied to the preliminary audio output, and filtering the preliminary audio output to produce the processed signal based on at least the filter parameters. However, Robben et al (20230326474) in the related art of audio signal processing, discloses producing filter parameters to the applied to the preliminary audio output, and filtering the preliminary audio output to produce the processed signal based on at least the filter parameters (para [0119] ''For instance, the audio system 10 may comprise a first filter bank and a second filter bank. The first filter bank is configured to filter and to add together two input audio signals based on a first cutoff frequency fCO1 and the second filter bank is configured to filter and to add together two input audio signals based on a second cutoff frequency fC02."; para [0127]-[0128] ''the audio system 10 comprises a first filter bank 151 and a second filter bank 152, which are applied successively and are implemented by the processing circuit 15. In this example, the first filter bank 151 processes the first audio signal and the second audio signal based on a first cutoff frequency fCO1, to produce an intermediate audio signal. The second filter bank 152 processes the intermediate signal and the third audio signal based on a second cutoff frequency fC02....Each filter bank filters and adds together its input audio signals based on its cutoff frequency. The filtering may be performed in time or frequency domain and the addition of the filtered audio signals may be performed in time domain or in frequency domain."). It would have been obvious to one of ordinary skill in the art to modify the audio signal filtering process, as taught by Robben et al (20230326474), to include audio system, as disclosed by Robben et al (20230326474), because it would advantageously allow for processing the third audio signal, which is improves upon the noise processing at higher frequencies with the use of the external air conduction sensor (Robben et al (20230326474) para 0124).
Claims 5-6, 16 are rejected under 35 U.S.C. 103 as being unpatentable over Huang et al (WO2022231977) in view of Wagner et al (20230209283).
As per claim 5, Huang et al (WO2022231977) teaches the method as claimed in claim 1 (see mapping above); however, Huang et al (WO2022231977) does not explicitly teach “further comprising generating a predetermined audio waveform as part of the preliminary audio output”.
However, Wagner et al (20230209283) in the related art of neural networks for audio signal processing, discloses further comprising generating a predetermined audio waveform as part of the preliminary audio output (para [0016] ''an audio signal may comprise unprocessed or raw audio data, for example raw audio recordings or raw audio wave forms ....extracted audio features, compressed audio data, a spectrum, in particular a frequency spectrum, a cepstrum and/or cepstral coefficients...the input audio signal received by the audio input unit may be an unprocessed recording of ambient sound. The audio signal to be processed by the neural network, e.g. an audio signal input which is provided to the neural network's input, may be processed audio data, in particular may be in the form of the processed input audio signal. For example, the audio signal to be processed by the neural network may be based on a spectrum, in particular a frequency spectrum, of the input audio signal. For example, the input audio signal may be transformed by a Fast Fourier Transformation (FFT). The audio signal inputted to the neural network may comprise a cepstrum. For example, the audio signal inputted to the neural network may comprise Mel-Frequency Cepstral Coefficients (MFCC) and/or other cepstral coefficients."; para [0022] ''a neural network adapted for regression-based noise cancellation might directly output denoised audio signals. Additionally or alternatively, the network adapted for regression-based noise cancellation may output a filter mask with which the input audio signals may be filtered to remove noise."). Therefore, it would have been obvious to one of ordinary skill in the art to modify the method, as disclosed by Huang et al (WO2022231977), to include neural network, as disclosed by Wagner et al (20230209283), because it allows audio signal to be processed by the neural network (See Wagner et al (20230209283)-para [0016]).
As per claim 6, Huang et al (WO2022231977) in view of Wagner et al (20230209283) teaches the method as claimed in claim 5, Wagner et al (20230209283) further discloses wherein the denoising comprises inputting the predetermined audio waveform to the statistical model or neural network (para [0016]; para [0022]).
As per claim 16, Huang et al (WO2022231977) discloses the method as claimed in claim 9, Huang et al (WO2022231977) further teaches “to the extracted features in response to the extracted features being input to a trained statistical model, to the extracted features” -- see (para [0072]-[0080]). Huang et al (WO2022231977) does not disclose wherein the denoising comprises producing magnitude weights and phase shifts to be applied, a reweighting step of applying the magnitude weights and phase shifts to produce reweighted features, and a combining step of combining the reweighted features into a single channel representation to produce the processed signal. However, Wagner et al (20230209283) in the related art of neural networks for audio signal processing, discloses wherein the denoising comprises producing magnitude weights and phase shifts to be applied, a reweighting step of applying the magnitude weights and phase shifts to produce reweighted features, and a combining step of combining the reweighted features into a single channel representation to produce the processed signal (para [0008] ''audio signal processing by the neural network...determining a confidence parameter resembling the reliability of the audio signal processing by the neural network...Determining the confidence parameter, the method allows to monitor the reliability of the audio signal processing by the neural network."; para [0040] ''based on the confidence parameter...Suitable network tuning parameters may comprise feature vectors, network weights...in a low confidence situation, the network weights of the network can be updated with network weights which have been obtained by training for less aggressive, but more stable audio signal processing."; para [0080] ''the regression-based audio processing by the neural network for each audio sample multiple times, wherein in each run, the audio sample is combined with a random perturbation. For example, one or more of the following perturbations may be added to the audio samples: noise... reverberation and/or non-perceivable phase-shifts ....After several runs for each audio sample with different perturbations, the stability of the network processing may be estimated by assessing the variation of the network output audio data. For example, assessment of the variations may comprise evaluating suitable loss-functions, short-time objective intelligibility (STOI), perceptual evaluation of speech quality (PESQ) and/or signal-to-noise-ratios (SNR). The confidence parameter may then for example be related to a standard deviation of the network output audio data..the standard deviation may be mapped towards the confidence parameter. For example, if the standard deviation exceeds a certain threshold, the confidence label can be set accordingly. Some perturbations, in particular time shifts, may also have to be imposed on the labels if calculations need an alignment between in- and output ....The calculation results of perturbed audio sample inputs may also be presented to listeners for manual evaluation. The listeners can then evaluate each perturbed audio sample individually and a confidence parameter can be calculated based on the individual ratings for each audio sample. The neural network may be further trained based on the second training data set containing the determined confidence parameter labels."). It would have been obvious to one of ordinary skill in the art to modify the method, as disclosed by Huang et al (WO2022231977), to include confidence parameter, as disclosed by Wagner et al (20230209283), because it allows network tuning parameters comprising network weights, and to monitor the reliability of the audio signal processing by the neural network, and to include perturbations, as disclosed by Wagner et al (20230209283), because it allows the stability of the network processing to be estimated by assessing the variation of the network output audio data (See Wagner et al (20230209283)- para [0040]; para [0080]).
Claim(s) 7-8 are rejected under 35 U.S.C. 103 as being unpatentable over Huang et al (WO2022231977) in view of Sprague et al (20150230033).
As per claim 7, Huang et al (WO2022231977) teaches the method as claimed in claim 1 (see mapping above); however, Huang et al (WO2022231977) does not disclose wherein the generating of humanly perceptible output comprises generating text which expresses the user's speech. However, Sprague et al (20150230033) in the related art of a hearing assistance system, discloses wherein the generating of humanly perceptible output comprises generating text which expresses the user's speech (para [0032] ''Then a controller may be configured to perform speech recognition of the first digital representation of the audio signal, in which the first digital representation is translated to text and all noise not recognized as speech removed during the translation....the controller may compare the text to a lookup table in memory and generate corresponding new text in a different language....when the audio is converted to textual representation, the text may also be visually displayed to the user on other devices communicated with the hearing assistance device, such as a mobile phone or laptop, or on the lens of the glasses."). It would have been obvious to one of ordinary skill in the art to modify the method, as disclosed by Huang et al (WO2022231977), to include textual representation, as disclosed by Sprague et al (20150230033), because it allows text to be visually displayed to the user on other devices communicated with the hearing assistance device, such as a mobile phone or laptop, or on the lens of the glasses. (See Sprague et al (20150230033)- para [0032])
As per claim 8, Huang et al (WO2022231977) teaches the method as claimed in claim 1 (see mapping above); however, Huang et al (WO2022231977) does not explicitly teach “wherein the generating of humanly perceptible output comprises generating a voiced version of the user's speech.” However, Sprague et al (20150230033) in the related art of a hearing assistance system, discloses wherein the generating of humanly perceptible output comprises generating a voiced version of the user's speech -- see (para [0174] ''The controller converts the second digital or textual representation of the audio signal (or may first convert the textual representation to the digital representation) to a voice modulated audio signal of the second language. The controller controls as speaker (an ear bud in some examples) which outputs or emits the voice modulated audio signal of the second language to the wearer so the wearer can understand the speech of the first language and hear the translation in a voice modulated manner ....the translation may now not only provide the translation for the user, but the translation is presented to the user as new generated speech (using a different human voice or modulated voice...For example, German is spoken by an individual in proximity to the hearing assistance device and is the audio signal of speech of a first language..the user wearing the hearing assistance device hears the emitted audio signal in English, the second language, and as new speech more audible than the original spoken words...two or more users, conversing in two or more different languages, may each hear the speech from the other users in that respective user's own native or chosen language, and may communicate back to the other users in that respective user's own native or chosen language."). Therefore, it would have been obvious to one of ordinary skill in the art to modify the method, as disclosed by Huang et al (WO2022231977), to include translations, as disclosed by Sprague et al (20150230033), because it allows to output or emit the voice modulated audio signal of the second language to the wearer so the wearer can understand the speech of the first language and hear the translation in a voice modulated manner (See Sprague et al (20150230033)- para [0174]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
DeHaan (20210092531) teaches audio filtering of multiple data streams (para 0026); the multiple data streams due to the third-bone conductor signal source – para 0024.
Kwasiborski et al (20200294508) teaches machine learned models processing multiple audio streams operating on voice features (para 0039, 0040); the multiple streams due to the additional audio signals.
Norris et al (20180138565) teaches a third audio source (through bone conduction) in processing audio signals via headphones and speakers (para 0075) and calculating ITD effects (para 0063-0067).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Michael Opsasnick, telephone number (571)272-7623, who is available Monday-Friday, 9am-5pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Mr. Richemond Dorvil, can be reached at (571)272-7602. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
/Michael N Opsasnick/Primary Examiner, Art Unit 2658 6/12/2026
Prosecution Insights