DETAILED ACTION
This action is written in response to the remarks and amendments dated 12/15/25. This action is made final. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
The Applicants argue that the previous art of record does not anticipate or render obvious the claims as currently amended. The Examiner provides updated prior art rejections below necessitated by the current amendments.
Subject Matter Eligibility
In determining whether the claims are subject matter eligible, the examiner has considered and applied the 2019 USPTO Patent Eligibility Guidelines, as well as guidance in the MPEP chapter 2106. The examiner finds that each independent claim is directed to the practical application of classifying electromyography (EMG) signal data, ie detecting and identifying physical movement in a subject.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
The following are the references relied upon in the rejections below:
Vajaklija (Vujaklija, Ivan, Vahid Shalchyan, Ernest N. Kamavuako, Ning Jiang, Hamid R. Marateb, and Dario Farina. "Online mapping of EMG signals into kinematics by autoencoding." Journal of neuroengineering and rehabilitation 15 (2018): 1-9. Cited by Applicant in IDS dated 7/1/22.)
Hu (Hu (Hu B, Simon AM, Hargrove L. Deep generative models with data augmentation to learn robust representations of movement intention for powered leg prostheses. IEEE Transactions on Medical Robotics and Bionics. 2019 Nov 7;1(4):267-78.)
Im (Im, Daniel, Sungjin Ahn, Roland Memisevic, and Yoshua Bengio. "Denoising criterion for variational auto-encoding framework." In Proceedings of the AAAI conference on artificial intelligence, vol. 31, no. 1. 2017.)
Friesen (Friesen GM, Jannett TC, Jadallah MA, Yates SL, Quint SR, Nagle HT. A comparison of the noise sensitivity of nine QRS detection algorithms. IEEE Transactions on biomedical engineering. 1990 Jan;37(1):85-98.)
Negi (Negi S, Kumar Y, Mishra VM. Feature extraction and classification for EMG signals using linear discriminant analysis. In2016 2nd international conference on advances in computing, communication, & automation (ICACCA)(Fall) 2016 Sep 30 (pp. 1-6). IEEE.)
Claims 1-5, 8-11, 13-18 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Vajaklija, Hu and Im.
Regarding claim 1, Vajaklija discloses a system that leverages data augmentation and deep learning to improve noise tolerance of myoelectric pattern recognition, comprising:
an electromyography (EMG) array operable for measuring EMG data; and
P. 3, first col., “For each DoF, the EMG feature vector was applied to the input and the output of an AEN with the structure shown in Fig. 1.”
(same page) “In the current application, two AEN networks were used to extract the activation signals corresponding to two DoFs (i.e., wrist flexion/extension and radial/ulnar deviation)”.
a processor in operative communication with the EMG array, the processor executing a controller defining:
A processor is inherent throughout the Vajaklija disclosure.
Hu also discloses this limitation at p. 273, first col. “Models were implemented in Keras using the Tensorflow backend on a laptop computer (Intel Core i7-7700, 2.80 GHz, 16 GB RAM) with GPU (4GB Nvidia GeForce GTX 1050) running Windows.” (Emphasis added.)
a feature extraction module configured to extract a plurality of features from an original set and a corrupted set of EMG data and produce a plurality of feature windows; and
P. 2, first and second col., “In this study, we used the root mean square (RMS) values of the R-channel surface EMG as P(t). The RMS values were obtained by non-overlapping 100 ms processing windows”.P. 3, first col, “Once the parameters were selected, the control signals … for the i-th DoF could be extracted from each new feature vector by projection on the weight matrix Iwi”.
a neural network configured to reconstruct and classify each feature window of the plurality of feature windows produced by the feature extraction module, the neural network comprising:
an encoder module configured as a supervised ... autoencoder to project an input feature window of the plurality of feature windows to a latent distribution, wherein a latent variable is sampled for the input feature window from the latent distribution;
Abstract: “In this paper, we propose a nonlinear minimally supervised method based on autoencoding (AEN) of EMG for myocontrol. The proposed method was tested against the state-of-the-art (SOA) control scheme using a Fitts’ law approach.” (Emphasis added.)
P. 2, first col., “In this study, we propose a different neural network approach for these estimates, based on AEN.” [Autoencoder neural network]. The examiner notes that autoencoders inherently comprise encoding and decoding layers, as illustrated in fig. 1 (reproduced below).
PNG
media_image1.png
328
750
media_image1.png
Greyscale
a decoder module configured to generate a reconstructed set of EMG data associated with the input feature window of the plurality of feature windows from the associated latent variable,
Id. (decoder portion is on the right side of the autoencoder).
See abstract and passim discussion use of EMG data for myocontrol.
P. 2, first and second col.: “The RMS values were obtained by non-overlapping 100 ms processing windows [23], which resulted in an output rate of the control of 10 Hz.” (Emphasis added.)
a classifier module configured to predict movement classes associated with the input feature window of the plurality of feature windows from the latent variable…
P. 1, first col., “Major advances in myocontrol have been made with pattern recognition approaches. These methods are based on the assumption that sufficiently distinguishable patterns can be observed in the EMG recordings during different motions. Each signal can be represented using a certain set of features which can be used as input to a classifier. The trained classifier is then capable of discriminating the intended motions. With state of the art pattern recognition methods, the classification accuracy exceeds > 95% when discriminating > 10 classes [3].”
Hu discloses the following further limitation which Vajaklija does not disclose:
a classifier module configured to predict movement classes associated with the input feature window of the plurality of feature windows from the latent variable, the classifier module trained using training data augmentation,
P. 268, first col. “Therefore, we developed and validated techniques for training deep generative models of multivariate sensor data using only tens or hundreds of experimentally-collected training examples by first applying data augmentation techniques including shifting, scaling, and additive random noise.”
wherein the neural network is trained to minimize one or more losses including at least a reconstruction loss between original and reconstructed EMG data, a divergence loss to regularize the latent distribution, and a classification loss between ground truth labels and predicted class labels.
P. 272, second col., “Reconstruction: Enc, EncZ, EncY and Dec were trained to minimize the reconstruction loss”. (Emphasis added.)
At the time of filing, it would have been obvious to a person of ordinary skill to apply the data augmentation technique disclosed by Hu to the Vajaklija system because it would likely improve classification performance in noisy environments. Both disclosures pertain to electromyography (EMG).
Im discloses the following further limitation which neither Vajaklija/Hu discloses:
an encoder module configured as a supervised denoising variational autoencoder
Abstract and passim: “Experimentally, we find that the proposed denoising variational autoencoder (DVAE) yields better average log-likelihood than the VAE and the importance weighted autoencoder on the MNIST and Frey Face datasets”. The Examiner notes that autoencoders are always supervised (ie this feature is inherent) because the input is always known.
At the time of filing, it would have been obvious to a person of ordinary skill to combine the DVAE technique disclosed by In with the Vajaklija/Hu system because, as noted by Im in the passage cited above, DVAEs provide superior classification results compared to vanilla DAE or VAE approaches.
Regarding independent claim 18, Vajaklija discloses its further limitation comprising a device … for control of a prosthesis. See p. 1, background (“Myoelectric signals (EMG) have been used to drive prosthetic devices for more than half a century.”) and passim.
Regarding claim 2, Hu discloses the further limitation comprising:
a signal corruptor module configured to corrupt EMG data measured by the EMG array, wherein each channel of the EMG data is corrupted to produce the original set of EMG data and the corrupted set of EMG data.
P. 268, first col. “Therefore, we developed and validated techniques for training deep generative models of multivariate sensor data using only tens or hundreds of experimentally-collected training examples by first applying data augmentation techniques including shifting, scaling, and additive random noise.”
P. 271, first col. “Similarly, we applied global (i.e., across all channels for all time steps) transformations consisting of 2 additional shifted copies (±10 ms relative to the original window), 8 additional scaled copies (the original window was multiplied by a scaling factor sampled from a uniform distribution between 0.95 and 1.05 on a per channel basis), and 10 shifted-scaled copies (5 scaled copies for each of the shifted copies) (Figure 2) for combined 20-fold data augmentation.” (Emphasis added.)
Regarding claim 3, Hu discloses the further limitation comprising:
a decoder module configured to generate a reconstructed set of EMG data associated with the input feature window of the plurality of feature windows from the associated latent variable.
P. 270, first col. “Guided by our working assumption, we build off previous work describing autoencoders [19] to learn a concise representation of our sensor data by jointly learning an encoder and decoder. The encoder maps data samples to a lower-dimensional latent space whereas the decoder maps the encoding from the latent space back to the original data space.” (Emphasis added.)
Regarding claim 4, Hu discloses the further limitation comprising:
a data separation module operable for separating the EMG data into a training subset and a testing subset.
P. 273, second col., “We performed analyses for both individual (i.e., the training data came from the same user for the testing data) and pooled (i.e., the training data did not include any data from the user for the testing data) user configurations.”
Regarding claim 5, Hu discloses the further limitation wherein the original set includes a plurality of EMG channels and wherein the corrupted set includes a plurality of corrupted copies of the original set, wherein each copy of the plurality of corrupted copies includes at least one corrupted segment.
P. 271, first col. “Similarly, we applied global (i.e., across all channels for all time steps) transformations consisting of 2 additional shifted copies (±10 ms relative to the original window), 8 additional scaled copies (the original window was multiplied by a scaling factor sampled from a uniform distribution between 0.95 and 1.05 on a per channel basis), and 10 shifted-scaled copies (5 scaled copies for each of the shifted copies) (Figure 2) for combined 20-fold data augmentation.” (Emphasis added.)P. 270, second col. “Data from the 17 prosthesis sensor channels were recorded at 1 kHz and segmented into 300 ms windows beginning 210 ms before the gait event and ending 90 ms after the gait event.”
Regarding claim 8, Hu discloses the further limitation wherein the neural network is a supervised denoising variational autoencoder, and the neural network minimizes a mean squared error between an original set of EMG data and a reconstructed set of EMG data.
P. 271, second col. “variational autoencoder”.
P. 272, second col. “To provide smooth, non-saturating gradients for the discriminators, we used the least-squares generative adversarial network (LSGAN) loss function [33] formulation”
P. 277, “We found that using the least-squares loss provided non-vanishing gradients”
[Vajaklija also discloses the use of mean squared error at p. 2: “In this study, we used the root mean square (RMS) values of the R-channel surface EMG as P(t).”]
Regarding claim 9, Hu discloses the further limitation wherein the neural network minimizes a Kullback-Leibler divergence between the latent distribution and a standard normal distribution.
P. 270 Variational approaches minimize the Kullback-Leibler divergence between the latent space and prior distributions”.
Regarding claim 10, Hu discloses the further limitation wherein the neural network minimizes a cross-entropy loss between a set of ground truth class labels and a set of predicted movement class labels associated with the movement classes predicted by the classifier module.
P. 273, “Semi-Supervised Classification: Enc and EncY were trained to predict the class by passing in the multivariate time series input (x) and its corresponding ground truth one-hot encodings (y) to minimize categorical cross entropy loss [equation (5)].” (Emphasis added.)
Regarding claim 11, Vajaklija discloses a method for improved interface noise tolerance with pattern recognition controllers, comprising:
accessing, by a processor, a plurality of input signals from an input device associated with a limb, the plurality of signals representing an intended control of a prosthesis; and
P. 3, first col., “For each DoF, the EMG feature vector was applied to the input and the output of an AEN with the structure shown in Fig. 1.”
(same page) “In the current application, two AEN networks were used to extract the activation signals corresponding to two DoFs (i.e., wrist flexion/extension and radial/ulnar deviation)”.
executing, by the processor, a predetermined pattern recognition controller by applying the plurality of input signals to at least one ML model, the at least one ML model … the at least one ML model configured for:
P. 2, first and second col., “In this study, we used the root mean square (RMS) values of the R-channel surface EMG as P(t). The RMS values were obtained by non-overlapping 100 ms processing windows”.
P. 3, first col, “Once the parameters were selected, the control signals … for the i-th DoF could be extracted from each new feature vector by projection on the weight matrix Iwi”.
P. 2, first col., “In this study, we propose a different neural network approach for these estimates, based on AEN.” [Autoencoder neural network]. The examiner notes that autoencoders inherently comprise encoding and decoding layers, as illustrated in fig. 1 (reproduced supra).
applying the plurality of input signals to a latent encoder defined by the at least one ML model to align the plurality of input signals to a low-dimensional manifold optimized to preserve salient features for movement intention recognition, and
P. 2, first and second col., “In this study, we used the root mean square (RMS) values of the R-channel surface EMG as P(t). The RMS values were obtained by non-overlapping 100 ms processing windows”.P. 3, first col, “Once the parameters were selected, the control signals … for the i-th DoF could be extracted from each new feature vector by projection on the weight matrix Iwi”.
See also fig. 1 (reproduced supra).
identifying a command for moving the prosthesis from the salient features.
P. 1, first col., “Major advances in myocontrol have been made with pattern recognition approaches. These methods are based on the assumption that sufficiently distinguishable patterns can be observed in the EMG recordings during different motions. Each signal can be represented using a certain set of features which can be used as input to a classifier. The trained classifier is then capable of discriminating the intended motions. With state of the art pattern recognition methods, the classification accuracy exceeds > 95% when discriminating > 10 classes [3].”
Hu discloses the following further limitation which Vajaklija does not disclose:
at least one ML model trained using data augmentation that artificially introduces training data variability,
P. 268, first col. “Therefore, we developed and validated techniques for training deep generative models of multivariate sensor data using only tens or hundreds of experimentally-collected training examples by first applying data augmentation techniques including shifting, scaling, and additive random noise.”
The obviousness analysis of claim 1 applies equally here.
Regarding claim 13, Hu discloses the further limitation further comprising training the at least one ML model using a training data set that is augmented with synthetic noise.
P. 268, first col. “Therefore, we developed and validated techniques for training deep generative models of multivariate sensor data using only tens or hundreds of experimentally-collected training examples by first applying data augmentation techniques including shifting, scaling, and additive random noise.” (Emphasis added.)
See also p. 271, second col. “zero-centered Gaussian noise”.
Regarding claim 14, Hu discloses the further limitation wherein the at least one ML model includes an LDA classifier that is trained with latent features of the training data set, the training data set being augmented with the synthetic noise.
P. 268, first col. “Therefore, we developed and validated techniques for training deep generative models of multivariate sensor data using only tens or hundreds of experimentally-collected training examples by first applying data augmentation techniques including shifting, scaling, and additive random noise.” (Emphasis added.)
See also p. 271, second col. “zero-centered Gaussian noise”.
Regarding claim 15, Hu discloses the further limitation further comprising constructing the training data set by systematically corrupting a predetermined number of a plurality of channels of raw training signals from the input device.
P. 268, first col. “Therefore, we developed and validated techniques for training deep generative models of multivariate sensor data using only tens or hundreds of experimentally-collected training examples by first applying data augmentation techniques including shifting, scaling, and additive random noise.” (Emphasis added.)
Regarding claim 16, Hu discloses the further limitation wherein systematically corrupting the predetermined number of the plurality of channels of the raw training signals includes flatlining, applying Gaussian noise, or a randomized mixture thereof.
P. 268, first col. “Therefore, we developed and validated techniques for training deep generative models of multivariate sensor data using only tens or hundreds of experimentally-collected training examples by first applying data augmentation techniques including shifting, scaling, and additive random noise.” (Emphasis added.)
See also p. 271, second col. “zero-centered Gaussian noise”.
Regarding claim 17, Hu discloses the further limitation wherein the plurality of signals includes electromyographic (EMG) signals and the input device includes an array of EMG electrodes that measure muscle activity indicative of the intended control of the prosthesis.
P. 1, “Myoelectric signals (EMG) have been used to drive prosthetic devices for more than half a century. However, the commercially available products still mainly rely on a simple direct and sequential control. This control strategy offers robust and reliable handling of the prosthetic in daily life, but it allows limited recovery of functionality and requires high cognitive load by the user [1, 2]. Therefore, several attempts have been made for establishing a more intuitive interface for active prosthesis control.”
P. 2, “Given the R-dimensional features of the surface EMG, denoted as P(t) = [p1(t), p2(t),⋯, pR(t)], the goal is to estimate the activation intentions, or motor control signals, for each DoF.”
Regarding claim 18, Vajaklija discloses a device for implementing pattern recognition using data augmentation for improved noise tolerance, comprising:
an input device that generates a plurality of signals, the plurality of signals representing an intended control of a prosthesis; and
P. 3, first col., “For each DoF, the EMG feature vector was applied to the input and the output of an AEN with the structure shown in Fig. 1.”
(same page) “In the current application, two AEN networks were used to extract the activation signals corresponding to two DoFs (i.e., wrist flexion/extension and radial/ulnar deviation)”.
a processor in operable communication with the input device, the processor executing a predetermined pattern recognition controller that applies the plurality of input signals to at least one ML model … the at least one ML model configured to:
P. 2, first col., “In this study, we propose a different neural network approach for these estimates, based on AEN.” [Autoencoder neural network]. The examiner notes that autoencoders inherently comprise encoding and decoding layers, as illustrated in fig. 1 (reproduced supra).
align the plurality of input signals to a low-dimensional manifold defining features, and
P. 2, second col., “The standard MLP structure with one hidden layer containing two neurons was used for each DoF for mapping the association of the EMG feature vector P(t) to itself, while capturing the low-dimensional controls in the hidden layer, with reduced number of neurons.” (Emphasis added.) See fig. 1 (reproduced supra).
identify from the features a command for moving the prosthesis.
P. 268, first col. “Therefore, we developed and validated techniques for training deep generative models of multivariate sensor data using only tens or hundreds of experimentally-collected training examples by first applying data augmentation techniques including shifting, scaling, and additive random noise.”
Hu discloses the following further limitation which Vajaklija does not disclose wherein the at least one ML model trained using a training data set augmented with synthetic noise,
P. 268, first col. “Therefore, we developed and validated techniques for training deep generative models of multivariate sensor data using only tens or hundreds of experimentally-collected training examples by first applying data augmentation techniques including shifting, scaling, and additive random noise.”
The obviousness analysis of claim 1 applies equally here.
Regarding claim 20, Vajaklija discloses the further limitation wherein the plurality of signals includes electromyographic (EMG) signals and the input device includes an array of EMG electrodes that measure muscle activity indicative of the intended control of the prosthesis.
P. 1, “Myoelectric signals (EMG) have been used to drive prosthetic devices for more than half a century. However, the commercially available products still mainly rely on a simple direct and sequential control. This control strategy offers robust and reliable handling of the prosthetic in daily life, but it allows limited recovery of functionality and requires high cognitive load by the user [1, 2]. Therefore, several attempts have been made for establishing a more intuitive interface for active prosthesis control.”
P. 3, first col., “For each DoF, the EMG feature vector was applied to the input and the output of an AEN with the structure shown in Fig. 1.”
(same page) “In the current application, two AEN networks were used to extract the activation signals corresponding to two DoFs (i.e., wrist flexion/extension and radial/ulnar deviation)”.
Claims 6-7 are rejected under 35 U.S.C. 103 as being unpatentable over Vajaklija, Hu, Im and Friesen.
Regarding claim 6, Friesen discloses the following further limitation which neither Vajaklija/Hu/Im discloses wherein at least one corrupted segment defines powerline interference superimposed onto the original signal on one or more channels.
P. 87, second col. “Since the purpose of this study was to evaluate the noise rejection properties of nine QRS detection algorithms, we selected four different representative noise sources for simulation: 1) electromyographic interference because of its random properties and high frequency content, 2) powerline interference because it is ubiquitous,”
At the time of filing, it would have been obvious to a person of ordinary skill to combine the noise simulation technique of Friesen with the combined system of Vajaklija/Hu/Im because it can improve classifier performance in the presences of powerline interference, which (as noted by Friesen) is ubiquitous.
Regarding claim 7, Friesen discloses the further limitation wherein the powerline interference superimposed is 50 hz to 60 hz.
Power line interference consists of 60 Hz pickup (in the U.S.) and harmonics which can be modeled as sinusoids and combination of sinusoids [2]. See Fig. l(a). Characteristics which might need to be varied in a model of power line noise include the amplitude and frequency content of the signal. These characteristics are generally consistent for a given measurement situation and, once set, will not change during a detector evaluation.”
Claims 12 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Vajaklija, Hu, Im and Negi.
Regarding claim 12, Negi discloses the further limitation which Vajaklija/Hu/Im do not disclose wherein the at least one ML model includes a convolutional neural network that computes latent features and a linear discriminant analysis (LDA) classifier that classifies the latent features as one or more gestures.
P. 2, second col., “In present study the LDA classifier is used. The advantage of LDA classifier is that iterative training is not required and it avoids the under- or over-training [8]. Finally the commands are generated based on the decisions in the pattern recognition block which are used for myoelectric control of various applications.”
At the time of filing, it would have been obvious to a person of ordinary skill to apply LDA to EMG data for classification tasks (as in Vajaklija/Hu/Im) because—as noted by Negi—iterative training is not required.
Regarding claim 19, Hu discloses the further limitation wherein … the training data set [is] constructed by systemic corruption of a predetermined number of a plurality of channels of raw training signals from the input device.
P. 268, first col. “Therefore, we developed and validated techniques for training deep generative models of multivariate sensor data using only tens or hundreds of experimentally-collected training examples by first applying data augmentation techniques including shifting, scaling, and additive random noise.”
Negi discloses the following further limitation which Vajaklija/Hu/Im do not disclose:
wherein the at least one ML model includes an LDA classifier that is trained with latent features of the training data set…
P. 2, second col., “In present study the LDA classifier is used. The advantage of LDA classifier is that iterative training is not required and it avoids the under- or over-training [8]. Finally the commands are generated based on the decisions in the pattern recognition block which are used for myoelectric control of various applications.”
The obviousness analysis of claim 12 applies equally here.
Additional Relevant Prior Art
The following references were identified by the Examiner as being relevant to the disclosed invention, but are not relied upon in any particular prior art rejection:
Ishizaka discloses a machine learning system for classification tasks featuring data augmentation and Kullback-Leibler divergence. See [0043]-[0047]. (US 20230267713 A1)
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Vincent Gonzales whose telephone number is (571) 270-3837. The examiner can normally be reached on Monday-Friday 7 a.m. to 4 p.m. MT. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Miranda Huang, can be reached at (571) 270-7092.
Information regarding the status of an application may be obtained from the USPTO Patent Center.
/Vincent Gonzales/Primary Examiner, Art Unit 2124