DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections - 37 CFR 1.75(a)
1. The following is a quotation of 37 CFR 1.75(a):
The specification must conclude with a claim particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention or discovery.
2. Claim 13 is objected to under 37 CFR 1.75(a), as failing to conform to particularly point out and distinctly claim the subject matter which application regards as his invention or discovery.
Claim 13 states “The brain-computer interface of claim 1, further comprising providing multiple bins to the RNN at once.”
Here, claim 13 should be dependent on claim 11 and read as follows… “The method of speech decoding using a brain-computer interface of claim 11, further comprising providing multiple bins to the RNN at once.”
This appears to be a typographical error. For continued examination purposes and in the best interests of compact prosecution, Examiner assumes that Claim 13 is dependent from claim 11 and reads as stated above.
The Examiner has tried to interpret the claims, as best the Examiner can ascertain, to develop an appropriate prior art rejection in the interests of compact prosecution. If any interpretation of the Examiner's is considered incorrect or off-base, the Examiner invites the Applicant to show the portions of the Applicant's specification which give a more proper interpretation of the claimed subject matter.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
1. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
2. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
3. Claims 1, 2, 4, 8, 11, 12, 14 & 18 are rejected under 35 U.S.C. 103 as being unpatentable over Stavisky et al. (US 20190333505 A1 hereinafter, Stavisky ‘505) in view of Lee et al. (US 20210183392 A1 hereinafter, Lee ‘392).
Regarding claim 1; Stavisky ‘505 discloses a brain-computer interface (Fig. 1, System 100) for decoding intended speech (i.e. One embodiment includes a neuronal speech system for decoding intended speech from neuronal signals includes a neuronal signal recorder implanted into a user's brain, including a multielectrode array, controller circuitry, and a communication circuitry capable of transmitting data to a neuronal signal decoder, the neuronal signal decoder located externally from the user's body, including a processor, an input/output interface. Paragraph 0004)
comprising: a microelectrode array (Not Shown in Drawings i.e. One embodiment includes a neuronal speech system for decoding intended speech from neuronal signals including a neuronal signal recorder implanted into a human user's brain, including a multielectrode array. Paragraph 0004);
a processor (Fig. 2, Processor 210) communicatively coupled to the microelectrode array (i.e. One embodiment includes a neuronal speech system for decoding intended speech from neuronal signals including a neuronal signal recorder implanted into a human user's brain, including a multielectrode array, controller circuitry, and a communication circuitry capable of transmitting data to a neuronal signal decoder, the neuronal signal decoder located externally from the user's body, including a processor. Paragraph 0004);
and a memory (Fig. 2, Memory 230),
the memory containing a speech decoding application (Fig. 2, Speech Decoding Application 232 i.e. Fig. 2 shows wherein Speech Decoding Application 232 is located in Memory 230)
that configures the processor to: receive neural signals from a user's brain recorded by a microelectrode array (i.e. One embodiment includes a neuronal speech system for decoding intended speech from neuronal signals includes a neuronal signal recorder implanted into a user's brain, including a multielectrode array, controller circuitry, and a communication circuitry capable of transmitting data to a neuronal signal decoder, the neuronal signal decoder located externally from the user's body, including a processor, an input/output interface, and a memory, where the memory contains a neuronal speech application that directs the processor to obtain neuronal signal data from the neuronal signal recorder. See Abstract and Paragraph 0041);
where the neural signals comprise action potential spikes (i.e. Neuronal signal recorders record electrical activity (e.g. action potentials, also known as “spikes”, and/or local field potentials) in the brain as neuronal signal data. Paragraph 0035);
bin the received action potential spikes by time (i.e. Action potential features and local field potentials can be represented as a time-series of values. In many embodiments, the decoder bins these input features by time. Paragraph 0053);
provide the bins to a recurrent neural network (RNN) to receive a likely phoneme at the time of each provided bin (i.e. The decoder bins these input features by time, and the values in the bins can be concatenated as a single input vector (e.g. as a single data structure containing both the action potential features and the local field potential features of a particular time window. Paragraph 0053)
generate an estimated intended speech using a phoneme decoder provided with the likely phonemes (i.e. Neuronal speech processes convert neuronal signals into the intended speech of the user from which the neuronal signals were obtained. The morpho-phonological encoding stage breaks down the words into individual syllables. The phonetic encoding stage involves selection of the appropriate phonemes. Finally, the articulation step involves the execution of the appropriate physical movements required to enact the production of the sound wave in accordance with the order of the phonemes. Paragraph 0045)
and vocalize the estimated intended speech using a loudspeaker (i.e. Fig. 1 shows wherein a speaker is attached to the Vocalizer 130. This output can be display of the speech string text and/or the vocalization of the speech string via a speaker. Paragraph 0054) communicatively coupled to the brain-computer interface (i.e. Fig. 1 shows wherein a speaker is attached to the Vocalizer 130 and the Brain Interface 110 via the Neural Signal Decoder 120.)
Stavisky ‘505 does not expressly disclose the limitation as expressed below.
Lee ‘392 discloses where the phoneme decoder comprises a language model formatted as a weighted finite-state transducer (i.e. Examples of the speech recognition engines include dynamic time distortion-based engines and weighted finite state transducer (WFST)-based engines. The one or more speech recognition models and the one or more speech recognition engines may be used to process the extracted representative features of the front-end speech pre-processor to generate intermediate recognition results (e.g. phonemes, phoneme strings, and sub-words), and ultimately text recognition results (e.g. words, word strings, or a sequence of tokens). Paragraph 0187)
Stavisky ‘505 and Lee ‘392 are combinable because they are from same field of endeavor of speech systems (Lee ‘392 at “Field of the Invention”).
Before the effective filing date, it would have been obvious to a person of ordinary skill in the art to modify the speech system as taught by Stavisky ‘505 by adding the limitation as taught by Lee ‘392. The motivation for doing so would have been advantageous to enhance the speech recognition system because there is a problem in that the performance of speech recognition is deteriorated due to various named entities input in different languages and/or accents, and it is necessary to process efficiently the named entities that vary due to diversity of languages and/or accents. Therefore, it would have been obvious to combine Stavisky ‘505 with Lee ‘392 to obtain the invention as specified.
Regarding claim 2; Stavisky ‘505 discloses wherein the RNN is trained to output an interword demarcator between phonemes that begin and end new words (i.e. Speech features are constructed (370) into a “speech string” by concatenating sequential output speech features. In numerous embodiments, a secondary predictive model can be used to convert phonemes and/or sound waves in the speech string into specific words. This speech string is then output (380) via a display device. Paragraph 0054).
Regarding claim 4; Stavisky ‘505 discloses wherein multiple bins are provided to the RNN at once (i.e. The decoder bins these input features by time, and the values in the bins can be concatenated as a single input vector (e.g. as a single data structure containing both the action potential features and the local field potential features of a particular time window. In numerous embodiments, non-overlapping 100 ms bins are utilized. However, the size of the bins can vary depending upon the training protocol utilized for the neuronal decoding model. Paragraph 0053).
Regarding claim 8; Stavisky ‘505 discloses wherein the microelectrode array is positioned to record neural activity at a ventral premotor cortex of the user's brain (i.e. The neuronal signal recorder is implanted at Broca's area, Wernicke's area, face areas of (ventral) motor cortex, or elsewhere in the brain. Paragraph 0035).
Regarding claim 11; Claim 11 contains substantially the same subject matter as claim 1. Therefore claim 11 is rejected on the same grounds as claim 1.
Regarding claim 12; Claim 12 contains substantially the same subject matter as claim 2. Therefore claim 12 is rejected on the same grounds as claim 2.
Regarding claim 14; Claim 14 contains substantially the same subject matter as claim 4. Therefore claim 14 is rejected on the same grounds as claim 4.
Regarding claim 18; Claim 18 contains substantially the same subject matter as claim 8. Therefore claim 18 is rejected on the same grounds as claim 8.
4. Claims 3 & 13 are rejected under 35 U.S.C. 103 as being unpatentable over Stavisky ‘505 with Lee ‘392 and further in view of Mosayyebpour et al. (US 20200125951 A1 hereinafter, Mosayyebpour ‘951).
Regarding claim 3; Stavisky ‘505 as modified does not expressly disclose the limitation as expressed below.
Mosayyebpour ‘951 discloses wherein the RNN is trained using connectionist temporal classification (i.e. In various embodiments, a novel One Spike Connectionist Temporal Classification (OSCTC) algorithm is proposed to train a recurrent neural network, such as an LSTM network, for multi-class classification, such as phoneme recognition. Paragraph 0065)
Stavisky ‘505 and Mosayyebpour ‘951 are combinable because they are from same field of endeavor of speech systems (Mosayyebpour ‘951 at “Technical Field”).
Before the effective filing date, it would have been obvious to a person of ordinary skill in the art to modify the speech system as taught by Stavisky ‘505 by adding the limitation as taught by Mosayyebpour ‘951. The motivation for doing so would have been advantageous because there is a need in the art for solutions to optimize information classification systems for training neural networks that are both fast and resource efficient. Therefore, it would have been obvious to combine Stavisky ‘505 with Mosayyebpour ‘951 to obtain the invention as specified.
Regarding claim 13; Claim 13 contains substantially the same subject matter as claim 3. Therefore claim 13 is rejected on the same grounds as claim 3.
5. Claims 5 & 15 are rejected under 35 U.S.C. 103 as being unpatentable over Stavisky ‘505 with Lee ‘392 and further in view of Mehr et al. (US 20210191363 A1 hereinafter, Mehr ‘363).
Regarding claim 5; Stavisky ‘505 as modified does not expressly disclose the limitation as expressed below.
Mehr ‘363 discloses wherein the RNN comprises a unique input layer trained for each day of training data using a softsign activation function (i.e. Artificial neural networks generally comprise an interconnected group of nodes organized into multiple layers of nodes (see FIG. 10). For example, the ANN architecture may comprise at least an input layer, one or more hidden layers, and an output layer. In some cases, the node may sum up the products of all pairs of inputs, and their associated weights (FIG. 11). The activation function may be, for example, a rectified linear unit (ReLU) activation function, softsign or any combination thereof. Paragraph 0158).
Stavisky ‘505 and Mehr ‘363 are combinable because they are from same field of endeavor of speech systems (Mehr ‘363 at “Background of the Invention”).
Before the effective filing date, it would have been obvious to a person of ordinary skill in the art to modify the speech system as taught by Stavisky ‘505 by adding the limitation as taught by Mehr ‘363. The motivation for doing so would have been advantageous because there is a need better perform automated classification of object defects using machine learning algorithms. Therefore, it would have been obvious to combine Stavisky ‘505 with Mehr ‘363 to obtain the invention as specified.
Regarding claim 15; Claim 15 contains substantially the same subject matter as claim 5. Therefore claim 15 is rejected on the same grounds as claim 5.
6. Claims 6 & 16 are rejected under 35 U.S.C. 103 as being unpatentable over Stavisky ‘505 with Lee ‘392 and further in view of Lee et al. (US 20180190268 A1 hereinafter, Lee ‘268).
Regarding claim 6; Stavisky ‘505 as modified does not expressly disclose the limitation as expressed below.
Lee ‘268 discloses wherein each bin further comprises high- frequency spectral power features (i.e. The feature values of the signals of the different frequency components may have amplitudes represented by sizes of respective bins for the different frequency components, and the respective applying of the determined attention weight to the feature values may include selectively adjusting the sizes of the respective bins for the different frequency components based on the applied determined attention weight. Paragraph 0015)
Stavisky ‘505 and Lee ‘268 are combinable because they are from same field of endeavor of speech systems (Lee ‘268 at “Field”).
Before the effective filing date, it would have been obvious to a person of ordinary skill in the art to modify the speech system as taught by Stavisky ‘505 by adding the limitation as taught by Lee ‘268. The motivation for doing so would have been advantageous because a speech recognizing technology analyzes a speech language of a human for communication and converts the speech language into character or text data. As a result, it would enhance speech recognizing technology to develop further in response to a desire for convenience. Therefore, it would have been obvious to combine Stavisky ‘505 with Lee ‘268 to obtain the invention as specified.
Regarding claim 16; Claim 16 contains substantially the same subject matter as claim 6. Therefore claim 16 is rejected on the same grounds as claim 6.
7. Claims 7 & 17 are rejected under 35 U.S.C. 103 as being unpatentable over Stavisky ‘505 with Lee ‘392 and further in view of Francis et al. (US 20190025917 A1 herainafter, Francis ‘917).
Regarding claim 7; Stavisky ‘505 as modified does not expressly disclose the limitation as expressed below.
Francis ‘917 discloses wherein rolling z-scoring is applied to the bins (i.e. Principal Component (PC) scores were calculated on the standard scores (z-scores) of the neural data binned at 100 ms from the pruned trials (controlled for time to reward and maximum speed). A two-sample Kolmogorov-Smirnov test was performed at every bin (neural data was binned in 100 ms bins) to test the significance of the differentiability between the rewarding and non-rewarding PC score distributions at the corresponding time points. Paragraph 0250).
Stavisky ‘505 and Francis ‘917 are combinable because they are from same field of endeavor of speech systems (Francis ‘917 at “Background”).
Before the effective filing date, it would have been obvious to a person of ordinary skill in the art to modify the speech system as taught by Stavisky ‘505 by adding the limitation as taught by Francis ‘917. The motivation for doing so would have been advantageous for Brain-machine interfaces (BMIs) to better utilize mathematical algorithms to translate/decode users' intentions via their neural activity. Therefore, it would have been obvious to combine Stavisky ‘505 with Francis ‘917 to obtain the invention as specified.
Regarding claim 17; Claim 17 contains substantially the same subject matter as claim 7. Therefore claim 17 is rejected on the same grounds as claim 7.
8. Claims 9 & 19 are rejected under 35 U.S.C. 103 as being unpatentable over Stavisky ‘505 with Lee ‘392 and further in view of Shenoy et al. (US 20210064135 A1 hereinafter, Shenoy ‘135).
Regarding claim 9; Stavisky ‘505 as modified does not expressly disclose the limitation as expressed below.
Shenoy ‘135 discloses wherein the phoneme decoder traverses the language model using a Viterbi search (i.e. The Viterbi algorithm is used to find the most probable start time for each character given the neural activity. Paragraph 0061)
Stavisky ‘505 and Shenoy ‘135 are combinable because they are from same field of endeavor of speech systems (Shenoy ‘135 at “Field of Invention”).
Before the effective filing date, it would have been obvious to a person of ordinary skill in the art to modify the speech system as taught by Stavisky ‘505 by adding the limitation as taught by Shenoy ‘135. The motivation for doing so would have been advantageous to better record the activity of a specific or small group of specific neurons of the brain. Therefore, it would have been obvious to combine Stavisky ‘505 with Shenoy ‘135 to obtain the invention as specified.
Regarding claim 19; Claim 19 contains substantially the same subject matter as claim 9. Therefore claim 19 is rejected on the same grounds as claim 9.
9.. Claims 10 & 20 are rejected under 35 U.S.C. 103 as being unpatentable over Stavisky ‘505 with Lee ‘392 and further in view of Aleksic et al. (US 20170186432 A1 hereinafter, Aleksic ‘432).
Regarding claim 10; Stavisky ‘505 as modified does not expressly disclose the limitation as expressed below.
Aleksic ‘432 discloses wherein the phoneme decoder produces a word lattice using the language model (i.e. As shown in FIG. 1, the decoder 120 may include three decoding stages: a first pass decoding 122, a lattice expansion pass 124, and a second pass decoding 126. Paragraph 0041)
and wherein the phoneme decoder rescores the word lattice using an n-gram language model such that the best path through the rescored word lattice represents the estimated intended speech (i.e. Once the expanded list 206 of class-based terms is determined, the speech recognizer (e.g., decoder 120) may re-score the lattice 200b to determine new probabilities for all or some of the edges 204 in the lattice 200b. The new set of probabilities may be determined in part based on how closely the corresponding term at each edge of the lattice 200b phonetically matches the audio for a corresponding portion of the utterance (e.g., acoustic re-scoring). In some implementations, each of the class-based terms in the expanded list(s) in the lattice 200b can be evaluated against the audio signal to determine how closely each term matches what was spoken in the utterance. Paragraph 0053).
Stavisky ‘505 and Aleksic ‘432 are combinable because they are from same field of endeavor of speech systems (Aleksic ‘432 at “Technical Field”).
Before the effective filing date, it would have been obvious to a person of ordinary skill in the art to modify the speech system as taught by Stavisky ‘505 by adding the limitation as taught by Aleksic ‘432. The motivation for doing so would have been advantageous to better predict the likelihood that one or more sequences of terms occur in the speech. Therefore, it would have been obvious to combine Stavisky ‘505 with Aleksic ‘432 to obtain the invention as specified.
Regarding claim 20; Claim 20 contains substantially the same subject matter as claim 10. Therefore claim 20 is rejected on the same grounds as claim 10.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MARCUS T. RILEY, ESQ. whose telephone number is (571)270-1581. The examiner can normally be reached 9-5 M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hai Phan can be reached at 571-272-6338. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
MARCUS T. RILEY, ESQ.
Primary Examiner
Art Unit 2654
/MARCUS T RILEY/Primary Examiner, Art Unit 2654
Prosecution Insights