DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
In response to the Advisory Action from 5/28/2026, Applicant has filed a Request for Continued Examination (RCE) on 6/11/2026. In this response, the Amendments after Final that were not entered per the Advisory are resubmitted (See RCE transmittal form from 6/11/2026) and have now been entered. These amendments add limitations regarding "presenting on a graphical user interface (GUI), a prompt comprising a set of specified words or phrases, wherein an individual word or phrase of the set of specified words or phrases is visually highlighted in the prompt to trigger a user to produce inner speech corresponding to the individual word or phrase," the modification of the "collecting" step/function being "in response to the visually highlighting of the individual word or phrase," the predicted speech output narrowed to be "corresponding to the individual word or phrase", and an additional step of "receiving a user input based on accuracy of the predicted speech, the user input being a first user input when the predicted speech is correct and a second user input when the predicted speech is incorrect" that is then associated with the ground-truth information.
Applicant traverses the prior art rejection of the grounds that the prior art fails to teach the limitations to affirmative verification input and the dual-path verification scheme to derive the ground truth information (Remarks from 5/11/2026, Page 9). These arguments have been fully considered, however, are moot with respect to the new grounds of rejection further in view of Jimenez, et al. (U.S. PG Publication: 2025/0118289 A1).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-9, 11, and 13-21 are rejected under 35 U.S.C. 103 as being unpatentable over Kothari, et al (U.S. PG Publication: 2024/0221741 A1) in view of Jimenez, et al. (U.S. PG Publication: 2025/0118289 A1).
Discussion of Jimenez, et al: Before proceeding to the rejection under 35 U.S.C. 103, a discussion of Jimenez as prior art qualifying under 35 U.S.C. 102(a)(2) is merited. While Jiminez includes inventors in common, at least another inventor is named as required by 35 U.S.C. 102(a)(2). Moreover, while both Jiminez and the instant application are commonly owned as of the composition of this Office action, the common assignment occurred on 7/16/2025, after the effective filing date of the instant application (see MPEP 717.02(I)). Thus, Jimenez, which antedates the instant application effective filing date and names another inventor qualifies as prior art and would not apply to a prior art exception under 35 U.S.C. 102(b)(2)(C).
With respect to Claim 1, Kothari discloses:
A method comprising:
accessing a machine learning (ML) model that has been trained based on a collection of training data to detect presence of speech (Paragraph 0008- "analyze the signal using a trained machine learning model to determine whether the user is speaking;" Paragraph 0058- accessing stored machine learning models "process the signals to determine if the user is speaking silently or voiced and may determine one or more words or phrases from the signals;" See also Paragraphs 0064, 0080 (discussing various types of collected training data), and 0084);
collecting, by a speech signal detection device, a combination of signals comprising electromyograph (EMG) data signals and one or more non-EMG data signals (sensors that capture and measure signals associated with speech including an EMG sensor and non-EMG sensors such as a microphone/IMU, Paragraph 0066; Fig. 3, Elements 311-313);
processing the combination of signals by the ML model to predict presence of speech (signals are passed to the ML model to predict whether a using is speaking in a silent or voiced manner where the signals are from a combination of sensors (EMG and non-EMG), Paragraphs 0008, 0056, 0058, 0066, and 0084);
updating the collection of training data based on the combination of signals and prediction made by the ML model ("generating training data" that is collected during spontaneous speech in continual training, Paragraph 0143; see also iterative training using the multi-modal gathered data discussed at Paragraphs 0158-0160); and
retraining the ML model in an online learning approach using the updated collection of training data (continual updating/tuning of the ML model while the user is operating using the device "in their usual environment" (i.e., the approach is online), Paragraphs 0143 and 0158-0160).
Although Kothari does describe continual updating/tuning of the ML model while the device is in use in the user environment, Kothari does not teach the interactive online approach set forth in amended claim 1. Jimenez, however, discloses:
presenting, on a graphical user interface (GUI), a prompt comprising a set of specified words or phrases, wherein an individual word or phrase of the set of specified words or phrases is visually highlighted in the prompt to trigger a user to produce inner speech corresponding to the individual word or phrase (interactive training that includes a graphical user interface that prompts the user with a specified set of "words or phrases" wherein "user interface 700 can highlight an individual word or phrase 714 to trigger or instruct the user to begin producing inner speech for that individual word or phrase 714," Paragraphs 0116 and 0124; see also the GUI depicted in Fig. 7);
the collection of input signals in response to the visually highlighting of the individual word or phrase (in “response to highlighting the individual word or phrase…. Instructs” acquisition of training speech data, Paragraph 0116; note that in the case of Kothari, the training data is multi-model in the form of EMG and non-EMG);
processing training signals by the ML model to predict presence of speech and generate a predicted speech output comprising one or more words, phrases, sentences, or phonemes corresponding to the individual word or phrase (predictions are made using the training speech to generate a decoded output corresponding to the words or phrases that were prompted to the user via highlighting, Paragraphs 0116 and 0119; See also Fig. 7, Element 720; note that in the case of Kothari, the training data is multi-modal in the form of EMG and non-EMG);
presenting, on the graphical user interface (GUI), the predicted speech output (the predictions "can be presented...in a portion...of the user interface," Paragraph 0119 and Fig. 7, Element 720);
receiving a user input based on accuracy of the predicted speech, the user input being a first user input when the predicted speech is correct and a second user input when the predicted speech is incorrect ("[i]nput can be received from the user indicating whether or not to terminate the training iterations" wherein such an indication by the user after presenting "visually distinguished" information that the words or phrases were inaccurately decoded as opposed to correct or unhighlighted terms, Paragraphs 0125 and 0127; Fig. 7; in this case, the "input received from the user" indicating to terminate training is an indication that the predicted speech is correct whereas the request for another iteration corresponds to an indication of an incorrect decoding);
associating the first user input or the second user input with the combination of signals as ground-truth information (based upon the user feedback/input, the current ML model associating a ground truth between a particular input speech data and a decoding/transcription of words or phrases is output (user termination of interactive training procedure) or a set of "ground truth information" is formed for another iteration comprising the set of words of phrase (user continues procedure), Paragraphs 0125 and 0127);
Note also that based upon the user input in paragraphs 0125 and 0127, Jiminez also discloses the updating of training information and retraining (e.g., via a second or further iteration of the interactive procedure).
Kothari and Jimenez are analogous art because they are from a similar field of endeavor in silent speech detection. Thus, it would have been obvious to one of ordinary skill in the art, before the effective filing date to utilize the particular iterative ML training approach taught by Jiminez for the updating/tuning taught by Kothari to provide a predictable result of interactive training that more accurately decodes inner/silent speech (Jimenez, Paragraphs 0019 and 0120).
With respect to Claim 2, Kothari further discloses:
The method of claim 1, wherein the ML model is implemented by an individual device external to the speech signal detection device (various configuration of system components including an "external device" that implements a trained ML model outside of a speech detection device having sensors, Paragraphs 0054-0055; Fig. 2, Elements 210, 211, 220 and 221), the speech comprising inner speech, silent speech, or any other form of speech (silent or voiced speech, Paragraph 0008, 0058, and 0086), while Jimenez further discloses:
the predicted speech being presented in a region of the GUI separate from the prompt and together with the prompt, the predicted speech output being visually distinguished based on whether the predicted speech output matches the individual word or phrase (see Fig. 7 showing two separate GUI portions- one featuring the prompt together with another featuring the predicted speech decoding; see also Paragraph 0119 discussing the GUI having "visually distinguished" portions to show incorrect or inaccurate decoding results).
With respect to Claim 3, Kothari further discloses:
The method of claim 2, further comprising: converting the combination of signals into a digital signature (additional processing where analog signals from the sensors are converted into a digital signature suing an analog-to-digital converter (ADC), Paragraphs 0076 and 0083); and wirelessly transmitting the digital signature from the speech signal detection device to the individual device (after digital processing, signals from the sensor device are sent to the external device via a communication module that include wireless modalities (Bluetooth, wi-fi, etc.), Paragraphs 0058-0059; Fig. 2, Elements 216 and 220).
With respect to Claim 4, Kothari further discloses:
The method of claim 1, wherein the non-EMG data signals represent movement of certain muscles in a face and neck region, physical movements associated with inner speech, and muscle twitches (IMU sensor that "measures facial movements" including "muscle" strain and frequencies (i.e., a measurement of twitches) for "sub-vocalized" or “silent speech”, Paragraphs 0031-0032 and 0038; see also "neck muscle movement" at Paragraphs 0047 and 0050-0051).
With respect to Claim 5, Kothari further discloses:
The method of claim 1, wherein the non-EMG data signals comprise at least one of inertial measurement unit (IMU) movement or audio data (the system of Kothari features both types of non-EMG signal sensor data- IMU movement or audio data from a microphone, Paragraph 0038).
With respect to Claim 6, Kothari further discloses:
The method of claim 1, wherein the non-EMG data signals are received from at least one of an array of biopotential sensors, motion sensors, sound sensors, or photonic sensors that are independent of the EMG data signals (generation of separate, non-EMG data signals for silent speech detection including an array of biopotential sensors (e.g., EEG sensors), motion sensors (e.g., IMUs filtered at different frequencies), and sound sensors (e.g., ultrasound, multiple individual microphones), Paragraphs 0038, 0056, and 0073).
With respect to Claim 7, Kothari further discloses:
the EMG communication device being positioned adjacent to and underneath a neck region (EMG communication device portion positions adjacent to and underneath a neck region, Paragraph 0050 and see Fig. 1A, Element 121), and the EMG communication device comprising a plurality of electrodes configured to collect the combination of signals (electrodes "contact" this zone, Paragraphs 0050 and 0146), while Jimenez further discloses that an EMG silent speech communication device may be connected to an AR headset (Paragraph 0023).
With respect to Claim 8, Jimenez further discloses:
The method of claim 1, wherein the ML model is trained in real time (real-time interactive training, Paragraph 0021).
With respect to Claim 9, Kothari further discloses:
The method of claim 1, wherein the combination of signals is collected during a first portion of a recording session in which input comprising inner speech for a word or phrase is received, further comprising: collecting, by the speech signal detection device, during a second portion of the recording session, an additional combination of signals comprising EMG data signals and one or more non-EMG data signals associated with inner speech for the word or phrase; and processing the additional combination of signals by the ML model to predict additional presence of inner speech (embodiment where there is a first portion of a recording session for input indicative of silent/subvocal/inner speech using fewer signals and second portion of a recording session where a combination of signals from EMG sensors along with additional sensors (e.g., microphone, IMU, etc.) are relied upon and processed by a machine learning model to additionally detect words and phrases in the silent speech, Paragraphs 0031, 0077-0078, 0080 (describing a neural network trained to process EMG, IMUs and microphones), 0081-0082, and 0087).
With respect to Claim 11, Jimenez further discloses:
the ML model comprises a convolutional neural network (CNN) comprising two convolutional two-dimensional (2D) layers with max pooling followed by two fully-connected layers (CNN comprising two convolutional 2D layers with a followed by max pooling followed by two fully connected layers, Paragraph 0123, to provide a predictable result of enabling data processing (e.g., pattern detection, dimensionality reduction, making an overall prediction, etc.) that enables a CNN to make a prediction of silent speech detection).
With respect to Claim 13, Kothari further discloses:
The method of claim 1, further comprising: prompting a user to produce a signal for a set of inner speech words or phrases (training subject/user is presented with a prompt comprising phrases, Paragraphs 0139 and 0142); and triggering collection of the combination of signals representing the set of inner speech words or phrases in response to receiving input for initiating a recording session based on prompting of the user (responsive to the prompt training data is collected for the silent speech words and/or phrases using a combination of sensors (EMG and others), Paragraphs 0031, 0139, and 0145-0146).
With respect to Claim 14, Kothari discloses:
The method of claim 13, further comprising: forming a set of initial trials based on collecting multiple combinations of signals associated with production of inner speech for the set of inner speech words or phrases, wherein the collection of training data comprises the set of initial trials (initial training using various combinations of signals from different sensors (see "subset of modalities") for words or phrases is a silent speech domain, Paragraphs 0139 and 0159-0160).
With respect to Claim 15, Kothari further discloses:
The method of claim 14, further comprising: after training the ML model using the collection of training data, presenting results comprising prediction of the presence of inner speech for an additional set of trials associated with a portion of the combination of signals (feedback that is presented to a training subject related to the prediction of silent speech that is used in additional training iterations "during training data collection," Paragraph 0141 and 0159).
With respect to Claim 16, Jimenez further discloses:
The method of claim 13, wherein: the set of inner speech words or phrases comprises a plurality of identical words or phrases presented in a first sequence during a first training iteration; and the method further comprises prompting the user to produce signals for the plurality of identical words or phrases presented in a second sequence different from the first sequence during a second training iteration ("identical" words or phrases can be presented to the user in a sequence that are different from the identical words or phrases associated with the first training iteration, Paragraphs 0116 and 0126; Fig. 7).
With respect to Claim 17, Kothari further discloses:
The method of claim 1, further comprising: applying binary labels to portions of the combination of signals, wherein each binary label comprises either a positive label representing inner speech or a negative label representing signals other than inner speech; under-sampling portions of the combination of signals associated with negative labels; and over-sampling portions of the combination of signals associated with positive labels to balance the collection of training data (positive label that serves to "confirm the silent speech from the user" to move to an active state wherein a negative/non-silent speech does not move to a second state, Paragraphs 0092-0094; in the positive/confirmed state the EMG portions of the combination of signals are sampled with a "high frequency" or oversampled, Paragraph 0095 wherein in the negative state those samples are undersampled at a "lower sampling rate" and do not proceed to the active state, Paragraph 0095-0096; note that the “to balance” is an intended result of a step positively recited wherein the claimed step that is actually recited is addressed by the teachings of Kothari).
With respect to Claim 18, Kothari further discloses:
The method of claim 1, wherein processing the combination of signals comprises: generating a plurality of buffers, each buffer having a specified length and containing a subset of the combination of signals (one or more components such as memory buffers to temporarily store signals recorded by the wearable device, Paragraph 0120; see also that buffer length is discussed "memory buffers may store the last 5 seconds of recorded signals, the last 10 seconds of recorded signals, the last 20 seconds of recorded signals, the last 30 seconds of recorded, or the last minute of recorded signals" and note that these signals include the combination of signals, Paragraph 0066; Fig. 3, Elements 311-313):extracting features from each buffer using at least one of spectral features, temporal features, or muscle activation sequences over a specified time period; and generating a plurality of feature vectors based on the extracted features (recorded buffer signals are passed on/accessed for "processing and analysis" that includes feature extraction over the recorded periods with respect to at least muscle activation patterns, Paragraphs 0058, 0079, and 0132; plurality of features that together comprise a vector are obtained via feature extraction, Paragraph 0079).
Claim 19 involves an embodiment of the method of claim 1 practiced using a system comprising at least one processor and at least one memory storing processor-executable instructions, and thus, is rejected under similar rationale. Furthermore, Kothari teaches system implementation of the method of claim 1 comprising at least one processor and at least one memory storing processor-executable instructions (Paragraphs 0162-0164).
Claim 20 involves an embodiment of the method of claim 1 practiced using processor-executable instructions stored on a non-transitory computer-readable storage medium, and thus, is rejected under similar rationale. Furthermore, Kothari teaches system implementation of the method of claim 1 as a program stored on a non-transitory computer-readable storage medium (Paragraphs 0162-0164).
With respect to Claim 21, Jimenez further discloses:
The method of claim 1, wherein updating the collection of training data comprises: assigning temporal weights to portions of the combination of signals based on temporal order in which corresponding speech was produced by a user, wherein more recent portions receive higher weights than less recent portions; and wherein retraining the ML model comprises updating parameters of the ML model based on the temporal weights (time-based weighting applied to training signals including EMG data wherein a more recent word or phrase in the sequence is associated with a higher weight than a less recent word or phrase in the sequence and wherein the weights are a part of updated ML training data set, Paragraphs 0125 and 0127; note that in the case of Kothari, the training data is multi-modal in the form of EMG and non-EMG).
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Kothari, et al. in view of Jimenez, et al. and further in view of Wang, et al. ("Silent Speech Decoding Using Spectrogram Features Based on Neuromuscular Activities," 2020).
With respect to Claim 12, Kothari in view of Jimenez discloses teaches the silent speech detection method using machine learning model processing of EMG signals as applied to Claim 1. Kothari also discloses an LLM model as a type of transformer (Paragraph 0065). Kothari in view of Jimenez do not teach the image-based processing of spectrograms by the machine learning model according to the approach set forth in claim 12. Wang, however, discloses:
converting the EMG data signals into one or more time-frequency spectrograms, wherein each frequency band in the spectrograms represents a single feature (Abstract- “transforming the sEMG data into spectrograms that contain abundant information in time and frequency domains and are regarded as channel-interactive; note that the absence of a spectrogram feature at a particular band (measured in Hz) or its presence over time relates to a spectrogram feature, Fig. 5; Sections 3.1-3.2, Page 5; Fig. 6b features of six-channel sEMG);
processing the one or more spectrograms by the ML model as image data (by relying upon spectrogram images "the silent speech decoding becomes a video classification, explored by deep learning methods;" see ML decoding shown in Fig. 6);
concatenating spectrograms from different EMG channels or different electrodes to form a composite image (use of multiple sEMG (e.g., six channel), Section 2.1, Pages 2-3 and Fig. 1 showing electrode sites; wherein the output channels are concatenated, Section 3.2, Page 5); and
processing the composite image by the ML model (the silent speech decoding becomes a video classification, explored by deep learning methods;" see ML decoding shown in Fig. 6 with neural network processing leading to a prediction output).
Kothari, Jimenez, and Wang are analogous art because they are from a similar field of endeavor in speech detection. Thus, it would have been obvious to one of ordinary skill in the art, before the effective filing date to utilize the spectrogram-based detection taught by Wang in the silent speech detection taught by Kothari in view of Jimenez to provide a predictable result of enabling multi-channel silent speech detection that takes spatial correlation into account (Wang, Section 1, Page 2).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Laufer, et al. (U.S. PG Publication: 2025/0118290 A1)- teaches the highlight of a target word for collecting inner speech in a training data generation process (Paragraphs 0120-0121).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JAMES S WOZNIAK whose telephone number is (571)272-7632. The examiner can normally be reached 7-3, off alternate Fridays.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant may use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Flanders can be reached at (571)272-7516. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
JAMES S. WOZNIAK
Primary Examiner
Art Unit 2655
/JAMES S WOZNIAK/Primary Examiner, Art Unit 2655
Prosecution Insights