DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Notice of Pre-AIA or AIA Status
Claims 1-9 are pending for examination.
Claims 1 are independent Claims.
Claims 1-9 are rejected under 35 U.S.C. §103.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-4 and 8-9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Peralta et al. (“SepaConvNet for Localizing the Subthalamic Nucleus Using One Second Micro-electrode Recordings” hereinafter Peralta) in view of Davalos et al. (U.S. 2005/0130230 hereinafter Davalos) in further view of Jia et al. (“Attention-based Graph ResNet for Motor Intent Detection from Raw EEG signals” hereinafter Jia).
As Claim 1, Peralta teaches a method of creating a computer implemented classification tool for classification of microelectrode recordings taken during a deep brain stimulation using deep residual neural network with attention comprising steps of:
collecting a data set of recordings taken during a deep brain stimulation (Peralta (III.Material and Methods, A. Data aquisistion), “The MER signals in this study were recorded from 57 PD patients having undergone DBS (Deep Brain Simulation) surgery at London Health Sciences Centre at Western University hospital. The Bens gun was used to advance 3-5 microelectrodes as a unit. Signals were recorded from 10.0 mm above the preoperatively determined target zero point to 4.0/5.0 mm below the zero-point, the end-point being detectable activity from the Substantia Nigra”, system collect data using a deep brain simulation procedure);
splitting recordings into overlapping time chunks (Peralta (III.Material and Methods, C. Singal preprocessing), “The STFT was performed with a Hann window of 512 samples width and a hop length of 10 samples, giving us input spectograms with 257 frequency bands and 2400 time points.” Hop length is 10 samples which provides an overlap of 512 samples), and converting time chunks into spectrograms (Peralta (III.Material and Methods, C. Singal preprocessing), “the temporal input signal is converted into a spectrogram representation. To compute the spectrogram, a Short Term Fourier Transform (STFT) was used, which applies a sliding window to the signal and computes the Fourier transformation for each window”, system collect data using a deep brain simulation procedure);
the network is trained to return zero for time chunks taken from recordings made outside
of the STN region of a brain and to return one for time chunks taken form recordings made
within the STN region of a brain (Peralta (I. Introduction, Constributions (last paragraph)), “propose a deep learning framework named SepaConvNet (Separable Convolution
based neural Network) for the detection of MER signals arising from the STN.” Peralta (III. Material and methods, B. Database construction, last paragraph), “We obtained a database with 9055 one second signals labelled as ‘0’ (outside the STN) and 8255 labelled as ‘1’ (inside the STN), each signal having a length of 24000 samples”),
Peralta may not explicitly disclose:
dividing data set into a training set, a validation set, and a test set
fine tuning the network with the validation set, cross checking the network with the test set
Davalos teaches:
dividing data set into a training set, a validation set, and a test set (Davalos (¶0245 line 8-10), “The dataset that the classifier is trained upon is broken up generally into three different sets: Training, Testing, and Evaluation.”)
fine tuning the network with the validation set, cross checking the network with the test set (Davalos (¶0245 line 11-16), “the use of distinct subsets of the available data for training and testing is required to ensure generalizability. The parameters of the classifier are set with respect to the training data set, and judged versus competitors on the testing data set, and validated on the evaluation data set”).
Peralta discloses a system/method to collect deep brain simulation signal and train machine learning system to recognize STN and non-STN region. Davalos disclose a system/method to train machine learning system using training set, validation set and test set. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify machine training system of Peralta instead be a machine learning system taught by Davalos, with a reasonable expectation of success. The motivation would be to allow “the use of distinct subsets of the available data for training and testing is required to ensure generalizability” (Davalos (¶0245 line 10-13)).
Peralta in view of Davalos may not explicitly disclose:
putting each spectrogram into a deep neural network of ResNet architecture augmented with a self-attention layer added after each of ResNet layers, with a head layer comprising a single 2D convolutional layer followed by batch normalization and ReLU activation function wherein
Jia teaches:
putting each spectrogram into a deep neural network of ResNet architecture augmented with a self-attention layer added after each of ResNet layers (Jia (2. Methods, 2.2 Residual learning and attention mechanism last 3 lines), “Each layer of the graph ResNet is followed and connected with an attention layer. And all the attention layers are jointly trained with the network.”), with a head layer comprising a single 2D convolutional layer (Jia (2. Methods, 2.3 Attention-based gragh ResNet), “Thus, the dimension of the input, i.e., graph Laplacian, was 64x64,”) followed by batch normalization and ReLU activation function (Jia (2. Methods, 2.3 Attentiona-based gragh ResNet), “12 layers of graph convolutions, and each two of them connected with a graph max-pooling layer, which reduced the dimension by two. The order of Chebyshev polynomial approximation for graph convolutional filters we used in our experiment was three. The activation function of graph con-volution was Leaky ReLu, which was used after the batch normalization.”) wherein
Peralta in view of Davalos discloses a Resnet architecture for machine learning. Jia discloses a system/method to incorporate attention layer with Resnet architecture. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Resnet architecture of Peralta in view of Davalos instead be a Resnet with attention layer taught by Jia, with a reasonable expectation of success. The motivation would be so that “full-attention architecture was presented to preserve the details from raw EEG signals, and select the most crucial info” (Jia (2. Methods, 2.2 Residual learning and Attention Mechanism, last 4 lines).
As Claim 2, besides Claim 1, Peralta in view of Davalos in further view of Jia teaches
wherein collecting data set includes signal recordings of 10 seconds (Peralta (I. Introduction, paragraph before the last paragraph), “the MER procedure is time consuming as many depths along the electrode trajectory are to be tested (each depth is an average of 10 seconds in duration”).
As Claim 3, besides Claim 1, Peralta in view of Davalos in further view of Jia teaches wherein the recordings are split into overlapping 500 ms long time chunks (Peralta (III.Material and Methods, C. Singal preprocessing), “The STFT was performed with a Hann window of 512 samples width and a hop length of 10 samples, giving us input spectograms with 257 frequency bands and 2400 time points.” 512 samples provides a time chunk that is close to 500 ms).
As Claim 4, besides Claim 1, Peralta in view of Davalos in further view of Jia teaches wherein each spectrogram has dimensions 129 (frequency) by 53 (time) (Peralta (III.Material and Methods, C. Singal preprocessing), “The STFT was performed with a Hann window of 512 samples width and a hop length of 10 samples, giving us input spectograms with 257 frequency bands and 2400 time points.” Spectragrams with 257 frequency band and 2400 time points are inputted. The specific numbers are construed as intended uses).
As Claim 8, besides Claim 1, Peralta in view of Davalos in further view of Jia teaches presenting the result of classification done by the deep residual neural network with attention in an output block ((Peralta (I. Introduction, Constributions (last paragraph)), “propose a deep learning framework named SepaConvNet (Separable Convolution based neural Network) for the detection of MER signals arising from the STN.”) (The rest of the limitations are rejected for the same reasons as Claim 1).
As Claim 9, the Claim is rejected for the same reasons as Claim 8.
Claim(s) 5 and 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Peralta and Davalos in view of Jia in further view of Zhao et al. (U.S. 2020/0211569 hereinafter Zhao).
As Claim 5, besides Claim 1, Peralta in view of Davalos in further view of Jia may not explicitly disclose:
wherein the 2D convolutional layer has a kernel size 7x7, padding 3 and 64 filters.
Jia teaches:
wherein the 2D convolutional layer has a kernel size 7x7, padding 3 and 64 filters ((Zhao (¶0037), “convolution layer (Conv1), the kernel size may be 3x3, the stride may be (1,1), the padding may be 1, the number of filters may be 256.” Specific numbers are construed as intended uses).
Peralta in view of Davalos in further view of Jia discloses a 2D convolution layer. Zhao disclose a convolution layer with kernal size, stide, padding and filters. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify convolution layer of Peralta in view of Davalos in further view of Jia instead be a convolution arrangement taught by Zhao, with a reasonable expectation of success. The motivation would be to provide the adaptability so that “the configuration of the CNN at least partly depends on the dimension of the matrix input to the CNN.” (Zhao (¶0035 line 3-4)).
As Claim 7, besides Claim 1, Peralta in view of Davalos in further view of Jia may not explicitly disclose:
wherein the convolutional layers in ResNet blocks has kernel size 3x3 and padding 1.
Jia teaches:
wherein the convolutional layers in ResNet blocks has kernel size 3x3 and padding 1 (Zhao (¶0037), “convolution layer (Convl), the kernel size may be 3x3, the stride may be (1,1), the padding may be 1, the number of filters may be 256, and the data size may be 128x64x10.” Specific numbers are construed as intended uses).
Peralta in view of Davalos in further view of Jia discloses a 2D convolution layer. Zhao disclose a convolution layer with kernal size, stide, padding and filters. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify convolution layer of Peralta in view of Davalos in further view of Jia instead be a convolution arrangement taught by Zhao, with a reasonable expectation of success. The motivation would be to provide the adaptability so that “the configuration of the CNN at least partly depends on the dimension of the matrix input to the CNN.” (Zhao (¶0035 line 3-4)).
Claim(s) 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Peralta and Davalos in view of Jia in further view of Wang et al. (U.S. 10,552,968 hereinafter Wang).
As Claim 6, besides Claim 1, Peralta in view of Davalos in further view of Jia may not explicitly disclose:
wherein each ResNet layer consists of six blocks, one lead-in block and five consecutive proper blocks.
Wang teaches:
wherein each ResNet layer consists of six blocks, one lead-in block and five consecutive proper blocks (Wang (col. 13 line 39-42, figs. 10-11), “the shared resnet block layer 1010 shares the same parameter data as the five resnet blocks 910-930 of the feature net 900 (e.g., the attention net 1000 and the feature net 900 use the same five resnet blocks).”, resnet block includes 5 consecutive resnet blocks and lead-in resnet block 910-930.)
Peralta in view of Davalos in further view of Jia discloses a Resnet architecture for machine learning system. Zhao disclose a Resnet structure with multiple Resnet blocks. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Resnet layer of Peralta in view of Davalos in further view of Jia instead be a Resnet arrangement taught by Wang, with a reasonable expectation of success. The motivation would be to “improves image correlation because the size of details of different areas of an image change as the depicted object's scale changes (e.g., the image grows larger).” (Wang (col. 12 line 52-54)).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Moinuddin (U.S. 11,368,349) disclose 2D convolution layer with different configurations of filter, kernel, stride, activation and padding.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NHAT HUY T NGUYEN whose telephone number is (571)270-7333. The examiner can normally be reached M-F: 12:00-8:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Viker Lamardo can be reached at 571-270-5871. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NHAT HUY T NGUYEN/ Primary Examiner, Art Unit 2147
Prosecution Insights