DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
35 USC § 101
Positive Statement: Claims 15-20 state “One or more tangible processor-readable media…”; wherein most of the time this would have been rejected under 35 USC 101, since “signals per se” could be read into the claim. However, Applicant specifically states (special definition) that “Tangible processor-readable storage media excludes intangible, transitory communications signals (such as signals per se)…”. Thus, claims 15-20 are not rejected under 35 USC 101.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-3, 5-10, 12-17, 19, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Huang et al., “Spectral-Spatial Mamba for Hyperspectral Image Classification” (Huang), and further in view of Gu et al., “Mamba: Linear-Time Sequence Modeling with Selective State Spaces” (Gu).
Regarding claim 1, Huang teaches a method for classifying (a method for spectral-spatial Mamba (SS-Mamba) for hyperspectral image (HSI) classification) (p. 1; Abstract) an input dataset (HSI input samples) (p. 3; Figure 1 and Section II., 1st paragraph), the input dataset being divisible into subsets of the dataset (the HSI input being divisible into spatial and spectral patches) (p. 4; Figure 2 and Section B.), the method comprising:
generating embedded subsets by projecting each subset of the subsets into a vector space to generate a corresponding embedded subset (using patch embeddings for each patch by reshaping the data into a tensor; wherein the tensor for spectral uses a 1D positional embedding) (p. 4; Figure 2 and Section B.);
encoding the embedded subsets into an encoded image using a dataset encoder including a gated spectral state space model (using a state space model on spectral tokens and including gated multilayer perception (MLP) on the patched/tokenized data) (pages 3-4; Figure 1 and Section A.), the gated spectral state space model that includes a spectral state space model (using a state space model on spectral tokens and including gated multilayer perception (MLP) on the patched/tokenized data) (pages 3-4; Figure 1 and Section A.), the spectral state space model being a state space model (state space model (SSM) for use with spectral tokens) (pages 3-4; Figure 1 and Section A.) that represents features of the input dataset (based on features of the input HSI) (p. 4; Figure 2 and Section B.) of each embedded subset of the embedded subsets (wherein the spectral information is transformed for the embedded patches; such as enhancement) (p. 3; Figure 1, Section II., 1st paragraph and pages 4-6; Figure 3 and Section C.); and
predicting a classification for the input dataset using the encoded image (predicting classification for the input HSI samples using the linear classifier) (p. 4; 1st paragraph, Figure 1 and pages 4-6; end of Section C.).
Huang teaches a gated multiplayer perception (MLP) (pages 3-4; Section A.) and wherein enhancement is accomplished (pages 4-6; Section C.). However, Huang does not explicitly state a “gated neural network” or a “spectral transformation”.
Gu teaches a new class of selective state space models (p. 2; 1st paragraph); wherein using a gated MLP which is part of modern neural networks (p. 8; Figure 3 and Section 3.5.1), such as the recurrent dynamics of an RNN or the convolution kernel of a CNN (p. 5; Section 3.2, 1st paragraph), for RNN gating mechanisms (p. 5; Section 3.2., 3rd paragraph); and wherein a spectral transformation is used (performing a fast Fourier transform (FFT) on the inputs) (p. 36; 1st paragraph).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Huang to include a gated neural network (gated RNN) within a selective state space models since it improves on prior work on several axes including the ability to efficiently select data and to filter out irrelevant information and remember relevant information indefinitely (Gu; p. 2, 1st and 2nd paragraphs); and to include an FFT since it can make the SSM scan fast and memory-efficient (Gu; p. 28, Section D, 1st paragraph)
Regarding claim 2, Huang teaches wherein a state transition matrix (A) (the state matrix A) (pages 3-4; Section A.), an input matrix (B) (input matrix B) (pages 3-4; Section A.), and an output matrix (C) (output matrix C) (pages 3-4; Section A.) of the spectral state space model (of the state space model) (pages 3-4; Section A.) are trained to generate the spectral state space model (for fast and efficient parallel training for the SSM for use in spectral-spatial model) (pages 3-4; Figure 1 and Section A.).
However, Huang does not explicitly state the that state transition matrix “is a diagonal matrix”.
Gu teaches wherein the state transition matrix A requires imposing structure on the matrix such as a diagonal (p. 4; 2nd paragraph).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Huang to include a diagonal structure for the A matrix since it allows for the SSM to be computed efficiently (Gu; p. 4, 2nd paragraph).
Regarding claim 3, Huang teaches wherein a kernel parameter (structured convolutional kernels) (pages 3-4; Section A.) of the spectral state space model (of the SSM) (pages 3-4; Section A.) is trained to generate the spectral state space model (for fast and efficient parallel training for the SSM for use in spectral-spatial model) (pages 3-4; Figure 1 and Section A.).
Regarding claim 5, Huang teaches the spectral state space model (the SSM for use in spectral-spatial model) (pages 3-4; Figure 1 and Section A.) further representing the features of the input dataset by multiplying the spectral of each embedded subset by a spectral of a kernel parameter to determine a respective product (multiplying the embedded subset x by the structured convolution kernels to determine a respective product y) (pages 3-4; Section A.).
Huang teaches wherein enhancement is accomplished (pages 4-6; Section C.). However, Huang does not explicitly state a “spectral transformation”.
Gu teaches a new class of selective state space models (p. 2; 1st paragraph); wherein a spectral transformation is used (performing a fast Fourier transform on the inputs) (p. 36; 1st paragraph); and wherein representing the features of the input dataset (inputs) (p. 36; 1st paragraph) by multiplying (multiply them in the frequency domain) (p. 36; 1st paragraph) the spectral transformation of each embedded subset (the fast Fourier transform (FFT) in the inputs) (p. 36; 1st paragraph) by a spectral transformation of a kernel parameter (and an FFT on the filter) (p. 36; 1st paragraph) to determine a respective product (to determine a product) (p. 36; 1st paragraph).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Huang to include a FFT since it can make the SSM scan fast and memory-efficient (Gu; p. 28, Section D, 1st paragraph).
Regarding claim 6, Huang teaches the spectral state space model (the SSM for use in spectral-spatial model) (pages 3-4; Figure 1 and Section A.) further representing the features of the input dataset using a spectral state space model feature including a set of subset features (based on the features of the input HSI) (pages 4-6; Sections B. and C.).
However, Huang does not explicitly teach “wherein determining the spectral state space model feature includes performing an inverse spectral transformation of the product to determine a respective subset feature of the set of subset features, the inverse spectral transformation being an inverse of a type of the spectral transformation
Gu teaches wherein determining the spectral state space model (selective SSM) (p. 35; Section E.5, 1st paragraph) feature includes performing an inverse spectral transformation of the product (performing an inverse FFT) (p. 36; 1st paragraph) to determine a respective subset feature of the set of subset features (to obtain the result) (p. 36; 1st paragraph), the inverse spectral transformation being an inverse of a type of the spectral transformation (the inverse FFT being an inverse of the FFT) (p. 36; 1st paragraph).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Huang to include a FFT since it can make the SSM scan fast and memory-efficient (Gu; p. 28, Section D, 1st paragraph).
Regarding claim 7, Huang teaches wherein the input dataset includes an image and the subsets include patches of the image (wherein the inputs are hyperspectral images and the subsets are patches of the image) (p. 4; Figure 2 and Section B.).
Regarding claim 8, see the rejection made to claim 1, as well as prior art Gu for a computing system (wherein the SSM is on modern hardware such as a GPU) (p. 28; Section D, 1st and 2nd paragraphs), the computing system (wherein the SSM is on modern hardware such as a GPU) (p. 28; Section D, 1st and 2nd paragraphs) comprising: one or more hardware processors (wherein the SSM is on modern hardware such as a GPU) (p. 28; Section D, 1st and 2nd paragraphs); an image embedder processor executable by the one or more hardware processors (wherein the SSM is on modern hardware such as a GPU) (p. 28; Section D, 1st and 2nd paragraphs); an image encoder processor executable by the one or more hardware processors (wherein the SSM is on modern hardware such as a GPU) (p. 28; Section D, 1st and 2nd paragraphs); and an image classifier processor executable by the one or more hardware processors (wherein the SSM is on modern hardware such as a GPU) (p. 28; Section D, 1st and 2nd paragraphs), for they teach all the limitations within this claim.
Regarding claim 9, see the rejection made to claim 2, as well as prior art Gu for a computing system (wherein the SSM is on modern hardware such as a GPU) (p. 28; Section D, 1st and 2nd paragraphs), the computing system (wherein the SSM is on modern hardware such as a GPU) (p. 28; Section D, 1st and 2nd paragraphs) comprising: one or more hardware processors (wherein the SSM is on modern hardware such as a GPU) (p. 28; Section D, 1st and 2nd paragraphs); an image embedder processor executable by the one or more hardware processors (wherein the SSM is on modern hardware such as a GPU) (p. 28; Section D, 1st and 2nd paragraphs); an image encoder processor executable by the one or more hardware processors (wherein the SSM is on modern hardware such as a GPU) (p. 28; Section D, 1st and 2nd paragraphs); and an image classifier processor executable by the one or more hardware processors (wherein the SSM is on modern hardware such as a GPU) (p. 28; Section D, 1st and 2nd paragraphs), for they teach all the limitations within this claim.
Regarding claim 10, see the rejection made to claim 3, as well as prior art Gu for a computing system (wherein the SSM is on modern hardware such as a GPU) (p. 28; Section D, 1st and 2nd paragraphs), the computing system (wherein the SSM is on modern hardware such as a GPU) (p. 28; Section D, 1st and 2nd paragraphs) comprising: one or more hardware processors (wherein the SSM is on modern hardware such as a GPU) (p. 28; Section D, 1st and 2nd paragraphs); an image embedder processor executable by the one or more hardware processors (wherein the SSM is on modern hardware such as a GPU) (p. 28; Section D, 1st and 2nd paragraphs); an image encoder processor executable by the one or more hardware processors (wherein the SSM is on modern hardware such as a GPU) (p. 28; Section D, 1st and 2nd paragraphs); and an image classifier processor executable by the one or more hardware processors (wherein the SSM is on modern hardware such as a GPU) (p. 28; Section D, 1st and 2nd paragraphs), for they teach all the limitations within this claim.
Regarding claim 12, Huang teaches the spectral state space model (the SSM for use in spectral-spatial model) (pages 3-4; Figure 1 and Section A.) further representing the features of the input dataset by multiplying the spectral of each embedded subset by a spectral of a kernel parameter to determine a respective product (multiplying the embedded subset x by the structured convolution kernels to determine a respective product y) (pages 3-4; Section A.).
Huang teaches wherein enhancement is accomplished (pages 4-6; Section C.). However, Huang does not explicitly state a “spectral transformation”.
Gu teaches a new class of selective state space models (p. 2; 1st paragraph); the image encoder processor (wherein the SSM is on modern hardware such as a GPU) (p. 28; Section D, 1st and 2nd paragraphs) wherein a spectral transformation is used (performing a fast Fourier transform on the inputs) (p. 36; 1st paragraph); and wherein representing the features of the input dataset (inputs) (p. 36; 1st paragraph) by multiplying (multiply them in the frequency domain) (p. 36; 1st paragraph) the spectral transformation of each embedded subset (the fast Fourier transform (FFT) in the inputs) (p. 36; 1st paragraph) by a spectral transformation of a kernel parameter (and an FFT on the filter) (p. 36; 1st paragraph) to determine a respective product (to determine a product) (p. 36; 1st paragraph).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Huang to include a FFT since it can make the SSM scan fast and memory-efficient (Gu; p. 28, Section D, 1st paragraph).
Regarding claim 13, Huang teaches the spectral state space model (the SSM for use in spectral-spatial model) (pages 3-4; Figure 1 and Section A.) further representing the features of the input dataset using a spectral state space model feature including a set of subset features (based on the features of the input HSI) (pages 4-6; Sections B. and C.).
However, Huang does not explicitly teach “wherein determining the spectral state space model feature includes performing an inverse spectral transformation of the product to determine a respective subset feature of the set of subset features, the inverse spectral transformation being an inverse of a type of the spectral transformation
Gu teaches wherein the image encoder processor (wherein the SSM is on modern hardware such as a GPU) (p. 28; Section D, 1st and 2nd paragraphs) further configured to represent using the spectral state space model (selective SSM) (p. 35; Section E.5, 1st paragraph) feature includes performing an inverse spectral transformation of the product (performing an inverse FFT) (p. 36; 1st paragraph) to determine a respective subset feature of the set of subset features (to obtain the result) (p. 36; 1st paragraph), the inverse spectral transformation being an inverse of a type of the spectral transformation (the inverse FFT being an inverse of the FFT) (p. 36; 1st paragraph).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Huang to include a FFT since it can make the SSM scan fast and memory-efficient (Gu; p. 28, Section D, 1st paragraph).
Regarding claim 14, see the rejection made to claim 7, as well as prior art Gu for a computing system (wherein the SSM is on modern hardware such as a GPU) (p. 28; Section D, 1st and 2nd paragraphs), the computing system (wherein the SSM is on modern hardware such as a GPU) (p. 28; Section D, 1st and 2nd paragraphs) comprising: one or more hardware processors (wherein the SSM is on modern hardware such as a GPU) (p. 28; Section D, 1st and 2nd paragraphs); an image embedder processor executable by the one or more hardware processors (wherein the SSM is on modern hardware such as a GPU) (p. 28; Section D, 1st and 2nd paragraphs); an image encoder processor executable by the one or more hardware processors (wherein the SSM is on modern hardware such as a GPU) (p. 28; Section D, 1st and 2nd paragraphs); and an image classifier processor executable by the one or more hardware processors (wherein the SSM is on modern hardware such as a GPU) (p. 28; Section D, 1st and 2nd paragraphs), for they teach all the limitations within this claim.
Regarding claim 15, see the rejection made to claim 1, as well as prior art Gu for one or more tangible processor-readable storage media embodied with instructions (in GPU HBM; high-bandwidth memory, commonly referred to as GPU memory) (p. 28; Section D, 3rd paragraph) for executing on one or more processors and circuits of a computing device (wherein the SSM is on modern hardware such as a GPU) (p. 28; Section D, 1st and 2nd paragraphs), for they teach all the limitations within this claim.
Regarding claim 16, see the rejection made to claim 2, as well as prior art Gu for one or more tangible processor-readable storage media embodied with instructions (in GPU HBM; high-bandwidth memory, commonly referred to as GPU memory) (p. 28; Section D, 3rd paragraph) for executing on one or more processors and circuits of a computing device (wherein the SSM is on modern hardware such as a GPU) (p. 28; Section D, 1st and 2nd paragraphs), for they teach all the limitations within this claim.
Regarding claim 17, see the rejection made to claim 3, as well as prior art Gu for one or more tangible processor-readable storage media embodied with instructions (in GPU HBM; high-bandwidth memory, commonly referred to as GPU memory) (p. 28; Section D, 3rd paragraph) for executing on one or more processors and circuits of a computing device (wherein the SSM is on modern hardware such as a GPU) (p. 28; Section D, 1st and 2nd paragraphs), for they teach all the limitations within this claim.
Regarding claim 19, see the rejection made to claim 5, as well as prior art Gu for one or more tangible processor-readable storage media embodied with instructions (in GPU HBM; high-bandwidth memory, commonly referred to as GPU memory) (p. 28; Section D, 3rd paragraph) for executing on one or more processors and circuits of a computing device (wherein the SSM is on modern hardware such as a GPU) (p. 28; Section D, 1st and 2nd paragraphs), for they teach all the limitations within this claim.
Regarding claim 20, see the rejection made to claim 6, as well as prior art Gu for one or more tangible processor-readable storage media embodied with instructions (in GPU HBM; high-bandwidth memory, commonly referred to as GPU memory) (p. 28; Section D, 3rd paragraph) for executing on one or more processors and circuits of a computing device (wherein the SSM is on modern hardware such as a GPU) (p. 28; Section D, 1st and 2nd paragraphs), for they teach all the limitations within this claim.
Claim(s) 4, 11, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Huang et al., “Spectral-Spatial Mamba for Hyperspectral Image Classification” (Huang), Gu et al., “Mamba: Linear-Time Sequence Modeling with Selective State Spaces” (Gu) and further in view of Fielden et al., US 10,722,137 B2 (Fielden).
Regarding claim 4, Huang teaches structured convolutional kernels (pages 3-4; Section A.). Gu teaches the convolution kernel of a CNN (p. 5; Section 3.2, 1st paragraph), and kernel fusion (p. 7; Section 3.3.2, 1st paragraph).
However, neither of them explicitly teach “wherein the kernel parameter is determined using a first initial parameter selected from a first Gaussian distribution and a second initial parameter selected from a second Gaussian distribution”.
Fielden teaches an invention relating to magnetic resonance thermometry (Abstract); wherein using a dynamic state-space model (col. 8; lines 10-11); and wherein the kernel parameter (Kalman filter) (col. 8, lines 9-10) is determined using a first initial parameter (first parameter w, representing system noise) (col. 8, lines 20-21) selected from a first Gaussian distribution (wherein w has a white Gaussian distribution) (col. 8, lines 21-23) and a second initial parameter (second parameter v, representing the measurement noise) (col. 8, lines 20-21) selected from a second Gaussian distribution (selected from a second white Gaussian distribution) (col. 8, lines 21-23).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination of prior arts to include Gaussian distribution which can create the Kalman filter which can be a recursive and efficient method to estimate the state of a process described by a dynamic state-space model (Fielden; col. 8, lines 9-11).
Regarding claim 11, see the rejection made to claim 4, as well as prior art Gu for a computing system (wherein the SSM is on modern hardware such as a GPU) (p. 28; Section D, 1st and 2nd paragraphs), the computing system (wherein the SSM is on modern hardware such as a GPU) (p. 28; Section D, 1st and 2nd paragraphs) comprising: one or more hardware processors (wherein the SSM is on modern hardware such as a GPU) (p. 28; Section D, 1st and 2nd paragraphs); an image embedder processor executable by the one or more hardware processors (wherein the SSM is on modern hardware such as a GPU) (p. 28; Section D, 1st and 2nd paragraphs); an image encoder processor executable by the one or more hardware processors (wherein the SSM is on modern hardware such as a GPU) (p. 28; Section D, 1st and 2nd paragraphs); and an image classifier processor executable by the one or more hardware processors (wherein the SSM is on modern hardware such as a GPU) (p. 28; Section D, 1st and 2nd paragraphs), for they teach all the limitations within this claim.
Regarding claim 18, see the rejection made to claim 4, as well as prior art Gu for one or more tangible processor-readable storage media embodied with instructions (in GPU HBM; high-bandwidth memory, commonly referred to as GPU memory) (p. 28; Section D, 3rd paragraph) for executing on one or more processors and circuits of a computing device (wherein the SSM is on modern hardware such as a GPU) (p. 28; Section D, 1st and 2nd paragraphs), for they teach all the limitations within this claim.
Contact
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL J VANCHY JR whose telephone number is (571)270-1193. The examiner can normally be reached Monday - Friday 9am - 5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emily Terrell can be reached at (571) 270-3717. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL J VANCHY JR/Primary Examiner, Art Unit 2666 Michael.Vanchy@uspto.gov
Prosecution Insights