Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This office action is in response to Applicant’s submission filed on 3/15/2025 (with Apparent priority date of 5/6/19). Claims 1-21 are pending of which claims 1, 16 and 21 are independent. As such, claims 1-21 have been examined.
This Application was published as US20250210031.
This Application is a continuation of application 17509892 issued as U.S. 12340789. A Terminal Disclaimer over the term of the parent is required as provided below.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-21 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-22 of U.S. Patent No. 12340789 (hereinafter as the ‘789 patent). Although the claims at issue are not identical, they are not patentably distinct from each other because the claims of the issued patent are narrower in scope than that of the instant application.
Patent (‘789 Patent)
1. A hearing apparatus comprising: a bone conduction sensor configured to provide a bone conduction signal indicative of bone-conducted vibration conducted by a bone of a wearer of the hearing apparatus;
and a signal processing unit comprising a speech model, the speech model comprising a neural network model configured to obtain a representation of the bone conduction signal as a control input, wherein the signal processing unit is configured to provide a synthetic speech signal;
wherein the signal processing unit is configured to predict a current sample of a time series from one or more previous samples of the time series, the time series representing a speech waveform, wherein the signal processing unit is configured to predict the current sample of the time series based on the representation of the bone conduction signal;
wherein the neural network model comprises a layer implemented as a part of the neural network model of the signal processing unit that provides the synthetic speech signal;
wherein the neural network model is trained based on a plurality of training speech samples;
and wherein at least one of the training speech samples comprises a training bone conduction data representing a speech and a corresponding training microphone data representing airborne sound of the speech,
the training microphone data and the training bone conduction data corresponding with each other temporally.
2. The hearing apparatus according to claim 1, wherein the speech model defines an internal state that evolves over time.
3. The hearing apparatus according to claim 1, wherein the neural network comprises a recurrent neural network.
4. The hearing apparatus according to claim 3, wherein the recurrent neural network has a density estimation mode during operation.
5. The hearing apparatus according to claim 1, wherein the neural network comprises a layered neural network comprising two or more layers, at least one of the two or more layers being a softmax layer.
6. The hearing apparatus according to claim 1, wherein the speech model comprises an autoregressive speech model.
7. The hearing apparatus according to claim 1, wherein the speech model is configured to compute a probability distribution over a plurality of output classes, at least one of the output classes representing a sample value of a sample of a sampled audio waveform.
8. The hearing apparatus according to claim 1, further comprising a head-worn hearing device, the head-worn hearing device comprising the bone conduction sensor and a first communication interface.
9. The hearing apparatus according to claim 8, wherein the head-worn hearing device further comprises the signal processing unit, and wherein the head-worn hearing device is configured to communicate the synthetic speech signal via the first communication interface to a handheld communication device.
10. The hearing apparatus according to claim 8, further comprising a signal processing device, the signal processing device comprising the signal processing unit and a second communication interface; wherein the first communication interface of the head-worn hearing device is configured to communicate the bone conduction signal to the second communication interface of the signal processing device.
11. The hearing apparatus according to claim 1, further comprising an ambient microphone configured to detect air-borne speech spoken by the wearer of the hearing apparatus, and to provide an ambient microphone signal indicative of the detected air-borne speech.
12. The hearing apparatus according to claim 1, further comprising a memory configured to store training data, the training data comprising one or more signal pairs, at least one of the signal pairs comprising the training bone conduction data and the training microphone data.
15. The hearing apparatus according to claim 1, wherein the hearing apparatus is a hearing aid.
Claim 1 – wherein the signal processing unit is configured to predict a current sample of a time series from one or more previous samples of the time series, the time series representing a speech waveform,
configured to obtain a representation of the bone conduction signal as a control input,
17. A hearing apparatus comprising: a bone conduction sensor configured to provide a bone conduction signal indicative of bone-conducted vibration conducted by a bone of a wearer of the hearing apparatus;
and a signal processing unit comprising a speech model, the speech model comprising a neural network model configured to obtain a representation of the bone conduction signal as a control input, wherein the signal processing unit is configured to provide a synthetic speech signal;
wherein the signal processing unit is configured to generate a synthetic filtered signal corresponding to a speech signal filtered by a first filter;
and wherein the signal processing unit is configured to receive an ambient microphone signal associated with an ambient microphone, the ambient microphone signal and the bone conduction signal corresponding with each other temporally,
and wherein the signal processing unit is configured to create a filtered version of the received ambient microphone signal using a second filter, and to combine the generated synthetic filtered signal with the created filtered version of the received ambient microphone signal to create the synthetic speech signal.
20. The hearing apparatus according to claim 17, wherein the signal processing unit, when in a training mode, is configured to adapt one or more model parameters of the speech model.
21. The hearing apparatus according to claim 20, wherein the adapted one or more model parameters are configured to allow the speech model to provide an improved match between a model output representing the synthetic speech and a corresponding training ambient microphone signal.
18. The hearing apparatus according to claim 17, wherein the speech model is a machine learning model, and wherein the machine learning model is trained based on a plurality of training speech samples.
19. The hearing apparatus according to claim 18, wherein at least one of the training speech samples comprises a training bone conduction data and a corresponding training microphone data, the training microphone data and the training bone conduction data corresponding with each other temporally.
22. A processor-implemented method of obtaining a synthetic speech signal, comprising: receiving, by a processing unit of an apparatus, a bone conduction signal from a bone conduction sensor, the bone conduction sensor configured to detect a bone-conducted vibration conducted by a bone of a person;
and using the signal processing unit to predict a current sample of a time series from one or more previous samples of the time series, the time series representing a speech waveform, wherein the signal processing unit is configured to predict the current sample of the time series based on the bone conduction signal,
wherein the signal processing unit comprises a neural network model configured to receive the bone conduction signal as a control input;
wherein the neural network model comprises a layer implemented as a part of the neural network model of the signal processing unit;
wherein the neural network model is trained based on a plurality of training speech samples;
and wherein at least one of the training speech samples comprises a training bone conduction data representing a speech and a corresponding training microphone data representing airborne sound of the speech,
the training microphone data and the training bone conduction data corresponding with each other temporally.
Current App. (17509892)1. A hearing apparatus comprising: a bone conduction sensor configured to provide a bone conduction signal indicative of bone-conducted vibration conducted by a bone of a wearer of the hearing apparatus;
and a signal processing unit comprising a speech model, the speech model comprising a neural network model configured to obtain a representation of the bone conduction signal, wherein the signal processing unit is configured to provide samples of a synthetic speech waveform;
wherein the samples of the synthetic speech waveform comprise a current sample and a previous sample, and wherein the current sample is based on the previous sample and also based on the bone conduction signal;
wherein the neural network model is trained based on a plurality of training speech samples;
and wherein at least one of the training speech samples comprises a training bone conduction data and a training microphone data,
the training microphone data and the training bone conduction data corresponding with each other temporally.
2. The hearing apparatus according to claim 1, wherein the speech model defines an internal state that evolves over time.
3. The hearing apparatus according to claim 1, wherein the neural network model comprises a recurrent neural network model.
4. The hearing apparatus according to claim 3, wherein the recurrent neural network model has a density estimation mode.
5. The hearing apparatus according to claim 1, wherein the neural network model comprises two or more layers, at least one of the two or more layers being a softmax layer.
6. The hearing apparatus according to claim 1, wherein the speech model comprises an autoregressive speech model.
7. The hearing apparatus according to claim 1, wherein the speech model is configured to compute a probability distribution over a plurality of output classes, at least one of the output classes associated with a sample value of one of the samples.
8. The hearing apparatus according to claim 1, further comprising a head-worn hearing device, the head-worn hearing device comprising the bone conduction sensor and a first communication interface.
9. The hearing apparatus according to claim 8, wherein the head-worn hearing device further comprises the signal processing unit, and wherein the head-worn hearing device is configured to communicate the samples of the synthetic speech waveform via the first communication interface to a handheld communication device.
10. The hearing apparatus according to claim 8, further comprising a signal processing device, the signal processing device comprising the signal processing unit and a second communication interface; wherein the first communication interface of the head-worn hearing device is configured to communicate the bone conduction signal to the second communication interface of the signal processing device.
11. The hearing apparatus according to claim 1, further comprising an ambient microphone configured to detect air-borne speech spoken by the wearer of the hearing apparatus, and to provide an ambient microphone signal indicative of the detected air- borne speech.
12. The hearing apparatus according to claim 1, further comprising a memory configured to store the training bone conduction data and the training microphone data.
13. The hearing apparatus according to claim 1, wherein the hearing apparatus is a hearing aid.
14. The hearing apparatus according to claim 1, wherein the current sample of the synthetic speech waveform is a predicted sample.
15. The hearing apparatus according to claim 1, wherein the representation of the bone conduction signal comprises bone conduction data.
16. A hearing apparatus comprising: a bone conduction sensor configured to provide a bone conduction signal indicative of bone-conducted vibration conducted by a bone of a wearer of the hearing apparatus;
and a signal processing unit comprising a speech model, the speech model comprising a neural network model configured to obtain a representation of the bone conduction signal, wherein the signal processing unit is configured to provide samples of a synthetic speech waveform;
wherein the signal processing unit is configured to obtain a synthetic filtered signal corresponding to a speech signal filtered by a first filter;
and wherein the signal processing unit is configured to receive an ambient microphone signal associated with an ambient microphone, the ambient microphone signal and the bone conduction signal corresponding with each other temporally,
and wherein the signal processing unit is configured to create a filtered version of the received ambient microphone signal using a second filter, and to combine the synthetic filtered signal with the created filtered version of the received ambient microphone signal.
17. The hearing apparatus according to claim 16, wherein the signal processing unit, when in a training mode, is configured to adapt one or more model parameters of the speech model.
18. The hearing apparatus according to claim 17, wherein the adapted one or more model parameters are configured to allow the speech model to provide an improved match between a model output and a corresponding training ambient microphone signal.
19. The hearing apparatus according to claim 16, wherein the speech model is a machine learning model, and wherein the machine learning model is trained based on a plurality of training speech samples.
20. The hearing apparatus according to claim 19, wherein at least one of the training speech samples comprises a training bone conduction data and a training microphone data, the training microphone data and the training bone conduction data corresponding with each other temporally.
21. A processor-implemented method, comprising: receiving, by a processing unit of an apparatus, a bone conduction signal from a bone conduction sensor, the bone conduction sensor configured to detect a bone- conducted vibration conducted by a bone of a person;
and using the signal processing unit to determine samples of a synthetic speech waveform, wherein the samples of the synthetic speech waveform comprise a current sample and a previous sample, and wherein the current sample is based on the previous sample and also based on the bone conduction signal;
wherein the signal processing unit comprises a neural network model configured to receive a representation of the bone conduction signal;
wherein the neural network model is trained based on a plurality of training speech samples;
and wherein at least one of the training speech samples comprises a training bone conduction data and a training microphone data,
the training microphone data and the training bone conduction data corresponding with each other temporally.
Potentially Allowable Subject Matter
Claims 1-21 would be potentially allowable if amended to overcome the pertinent rejections under section of the double patenting rejections.
The following is a statement of reasons for the indication of potentially allowable subject matter:
With respect to Claim 1, The closest prior arts found during the search are as follows: Oba (already of reference) US 7676372, which discloses: A hearing apparatus comprising: a bone conduction sensor configured to provide a bone conduction signal indicative of bone-conducted vibration conducted by a bone of a wearer of the hearing apparatus; and a signal processing unit comprising a speech model, the speech model comprising a neural network model configured to obtain a representation of the bone conduction signal, wherein the signal processing unit is configured to provide samples of a synthetic speech waveform; However, Oba does not disclose or suggest “wherein the samples of the synthetic speech waveform comprise a current sample and a previous sample, and wherein the current sample is based on the previous sample and also based on the bone conduction signal; wherein the neural network model is trained based on a plurality of training speech samples; and wherein at least one of the training speech samples comprises a training bone conduction data and a training microphone data, the training microphone data and the training bone conduction data corresponding with each other temporally.” See col.1 lines 9-39, col. 2, lines, 30-33, lines 51-54, col. 3, lines 12-22, col. 6, lines 1-4, col. 13, lines 25- col. 14, lines 3, col. 20, lines 36-48, col. 21, lines 23-34, and figs. 1-2 and associated description for additional details.
Watts (already of reference) US 20180367882 – discloses processing of bone conduction sensor signal, linear prediction cepstral coefficients, autoregressive coefficients, and line spectral frequencies to model the vocal tract which reads on “wherein the samples of the synthetic speech waveform comprise a current sample and a previous sample, and wherein the current sample is based on the previous sample and also based on the bone conduction signal;” See para 0025-0033 and figs. 3-9. However, it silent on use of neural network and/or temporal alignment of training microphone and bone conduction data.
Maruri (already of record), H. A. C., Lopez-Meyer, P., Huang, J., Beltman, W. M., Nachman, L., & Lu, H. (2018). V-speech: Noise-robust speech capturing glasses using vibration sensors. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2(4), 1-23.-teaches bone conduction sensor in a wearable hearing apparatus to detect bone vibration, a signal processing unit including a trained speech/neural network model, it also implies the mapping of vibration signal to clean microphone signal but does not explicitly teach time-aligned training sets or temporal association as the claim requires, it is also silent on synthetic speech waveform comprising autoregressive generation (current sample based on previous sample) aspect of the claim which is required. See sections 1-3 and figs 7-8 and 12-13.
Srinvasan (already of record) Srinivasan, S., & Kechichian, P. (2012, September). Robustness analysis of speech enhancement using a bone conduction microphone-preliminary results. In IWAENC 2012; International Workshop on Acoustic Signal Enhancement (pp. 1-4). VDE. – discloses an ambient microphone signal associated with the ambient microphone, the ambient microphone signal and the bone conduction signal corresponding with each other temporally. See sect 3 and fig. 1 for details. However, the reference is silent on using a neural network for the training.
Accordingly, the prior art of record fails to explicitly teach or fairly suggest the invention set forth in claim 1. Other independent claim 21, although different in statutory category, but contains similar limitations as claim 1, therefore also contains potentially allowable subject matter. Further dependent claims 2-15 inherit the potentially allowable subject matter from claim 1, and thus, also contain potentially allowable subject matter by virtue of their dependency.
Regarding Claim 16, the claim scope has some variation, which includes the following element, “wherein the signal processing unit is configured to obtain a synthetic filtered signal corresponding to a speech signal filtered by a first filter; and wherein the signal processing unit is configured to create a filtered version of the received ambient microphone signal using a second filter, and to combine the synthetic filtered signal with the created filtered version of the received ambient microphone signal.” Although Herve (already of reference) US 20120278070 discloses filters microphone and sensor signals in para 0049 and fig. 2, however it is silent on filtering to create a synthetic filter signal. Herve in para 0053 does appear to disclose combining or summing two filtered signals. Accordingly, the prior art of record fails to explicitly teach or fairly suggest the invention set forth in claim 16. Further dependent claims 17-20 inherit the potentially allowable subject matter from claim 16, and thus, also contain potentially allowable subject matter by virtue of their dependency.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Philip H Lam whose telephone number is (571)272-1721. The examiner can normally be reached 9 AM-3 PM Pacific time.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Bhavesh Mehta can be reached on 571-272-7453. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PHILIP H LAM/ Examiner, Art Unit 2656
Prosecution Insights