DETAILED ACTION
This action is in response to the claims filed 03/20/2024 for Application number 18/693,730. Claims 1-11 are currently pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 03/20/2024 and 03/31/2026 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-11 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 1 and 11 recites the limitation "the recurrent linear transform". There is insufficient antecedent basis for this limitation in the claim.
Claims 1 and 11 recites the limitation "the resulting artificial neural network in claim 1 and the neural network in claim 11". There is insufficient antecedent basis for this limitation in the claim.
Claims 1 and 11 recites the limitation "the group". There is insufficient antecedent basis for this limitation in the claim.
Claims 2-10 are rejected as being dependent on a rejected base claim without curing any of the deficiencies.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-11 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claim 1,
Step 1 Analysis: Claim 1 is directed to a process, which falls within one of the four statutory categories.
Step 2A Prong 1 Analysis: Claim 1 recites, in part, The limitations of:
reshaping each output vector into an output matrix with one dimension corresponding to spatial information in the input sequence and the other corresponding to temporal information in the input sequence, such that the preprocessing layer implements at least one of:
i. a non-linear recurrent neural network (RNN) or a stack of non-linear RNNs;
ii. a linear RNN or a stack of linear RNNs where the recurrent linear transform is fixed;
iii. a convolution layer where the convolution operation involves weights that are either fixed or learned; or
iv. a convolution layer that implements a linear system by using the system's impulse response with the input vector in the Fourier domain can be considered to be a mathematical calculation,
b. defining at least one implicit-attention layer that processes the three output vectors reshaped into the three output matrices of the at least one preprocessing layer by:
i. taking an inner product between all pairs of rows in the first two output matrices so as to compute attention scores that model dependencies between row or column vectors in these matrices that represent temporal information from the original input sequence; and
ii. multiplying the resulting attention scores by the third output matrix to compute a final output vector that stores a compressed summary of all prior items in the input sequence; can be considered to be a mathematical calculation.
These limitations as drafted, are processes that, under broadest reasonable interpretation, covers the recitation of mathematical calculations which falls within the “Mathematical concepts” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
Step 2A Prong 2 Analysis: This judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements – “c. operating the resulting artificial neural network by using it to map a sequence of input vectors onto at least one final output vector to perform at least one task selected from the group consisting of pattern classification, signal processing, data representation and data generation.”. Thus, these elements in the claim are recited at a high level of generality such that they amount to no more than mere instructions to apply the exception using a generic computer component. Please see MPEP 2106.05(f). Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
The claim further recites: a. defining at least one preprocessing layer receiving an input sequence of vectors and producing, for each input vector, three output vectors; This limitation is a mere data gathering step and outputting and thus is an insignificant extra-solution activity. Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claim as a whole is directed to an abstract idea.
Step 2B Analysis: The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of utilizing a processor, a non-transitory computer readable medium, and a neural network to perform the steps of the claimed process amount to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Furthermore, the limitation of a. defining at least one preprocessing layer receiving an input sequence of vectors and producing, for each input vector, three output vectors is well-understood, routine, and conventional, as evidenced by MPEP §2106.05(d)(II)(I), “receiving or transmitting data over a network”. These limitations therefore remain insignificant extra-solution activity even upon reconsideration, and does not amount to significantly more. Even when considered in combination, these additional elements amount to mere instructions to apply the exception using generic computer components and insignificant extra-solution activity, which cannot provide an inventive concept. The claim is not patent eligible.
Regarding claim 2, the rejection of claim 1 is further incorporated, and further, the claim recites: wherein the linear recurrent transform in step a-ii is initialized randomly. This limitation amounts to more specifics of the judicial exception identified in the rejection of claim 1 above.
The claim does not include any additional elements that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible.
Regarding claim 3, the rejection of claim 1 is further incorporated, and further, the claim recites: wherein the linear recurrent transform in step a-ii is chosen the from following set: discrete or continuous Legendre Transform, Fourier Transform,Hadamard Transform, Haar Transform, Laplace Transform, Cosine Transform, Fourier- Stieltjes, Gelfand Transform, or Harley Transform. This limitation amounts to generally linking the judicial exception to a field of use or technological environment. Please see MPEP 2106.05(h).
The claim does not include any additional elements that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible.
Regarding claim 4, the rejection of claim 1 is further incorporated, and further, the claim recites: wherein any of the steps in a and b are followed by non-linearities. This limitation amounts to more specifics of the judicial exception identified in the rejection of claim 1 above.
The claim does not include any additional elements that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible.
Regarding claim 5, the rejection of claim 1 is further incorporated, and further, the claim recites: further comprising one or more skip-connections that pass neural network activities from one network layer to another downstream network layer while skipping one or more intermediate layers. This limitation amounts to more specifics of the judicial exception identified in the rejection of claim 1 above.
The claim does not include any additional elements that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible.
Regarding claim 6, the rejection of claim 1 is further incorporated, and further, the claim recites: wherein a single output matrix is produced by the preprocessing layer, and three copies of this output matrix are linearly or nonlinearly transformed before being provided as input to the implicit attention layer. This limitation amounts to additional mathematical calculations in addition to the judicial exception identified in the rejection of claim 1 above.
The claim does not include any additional elements that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible.
Regarding claim 7, the rejection of claim 1 is further incorporated, and further, the claim recites: wherein three copies of the input sequence of vectors are provided as input to the preprocessing layer. This limitation is an insignificant extra-solution activity and thus the judicial exception is not integrated into a practical application. The claim as a whole is directed to an abstract idea.
The claim does not include any additional elements that amount to significantly more than the judicial exception. This limitation is just a nominal or tangential addition to the claim, and is also well-understood, routine and conventional as evidenced by MPEP §2106.05(d)(II)(I), “receiving or transmitting data over a network”. This limitation therefore remains insignificant extra-solution activity even upon reconsideration, and does not amount to significantly more. Even when considered in combination, this additional element represents an insignificant extra-solution activity which cannot provide an inventive concept. The claim is not patent eligible.
Regarding claim 8, the rejection of claim 1 is further incorporated, and further, the claim recites: wherein a separate preprocessing layer is used to create each of the three output matrices. This limitation amounts to more specifics of the judicial exception identified in the rejection of claim 1 above.
The claim does not include any additional elements that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible.
Regarding claim 9, the rejection of claim 1 is further incorporated, and further, the claim recites: wherein the input sequence of vectors is passed through three independent linear or nonlinear transformations before being provided as input to the preprocessing layer. This limitation amounts to additional mathematical calculations in addition to the judicial exception identified in the rejection of claim 1 above.
The claim does not include any additional elements that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible.
Regarding claim 10, the rejection of claim 1 is further incorporated, and further, the claim recites: wherein the first of the output matrices of the preprocessing layer has a temporal dimensional with a length of one. This limitation amounts to more specifics of the judicial exception identified in the rejection of claim 1 above.
The claim does not include any additional elements that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible.
Regarding Claim 11, it recites features similar to claim 1 and is rejected for at least the same reasons therein.
Allowable Subject Matter
Claims 1-11 are objected to as being allowable over prior art if all outstanding rejections were withdrawn. None of the prior art, either alone or in combination, fairly discloses limitations of claims 1 and 11 in particular:
a. defining at least one preprocessing layer receiving an input sequence of vectors and producing, for each input vector, three output vectors; reshaping each output vector into an output matrix with one dimension corresponding to spatial information in the input sequence and the other corresponding to temporal information in the input sequence, such that the preprocessing layer implements at least one of:
i. a non-linear recurrent neural network (RNN) or a stack of non-linear RNNs;
ii. a linear RNN or a stack of linear RNNs where the recurrent linear transform is fixed;
iii. a convolution layer where the convolution operation involves weights that are either fixed or learned; or
iv. a convolution layer that implements a linear system by using the system's impulse response with the input vector in the Fourier domain;
b. defining at least one implicit-attention layer that processes the three output vectors reshaped into the three output matrices of the at least one preprocessing layer by:
i. taking an inner product between all pairs of rows in the first two output matrices so as to compute attention scores that model dependencies between row or column vectors in these matrices that represent temporal information from the original input sequence; and
ii. multiplying the resulting attention scores by the third output matrix to compute a final output vector that stores a compressed summary of all prior items in the input sequence; and,
c. operating the resulting artificial neural network by using it to map a sequence of input vectors onto at least one final output vector to perform at least one task selected from the group consisting of pattern classification, signal processing, data representation and data generation.
The closest prior art uncovered was Mahmud et al. (“Human Activity Recognition from Wearable Sensor Data Using Self-Attention”, cited by Applicant in the IDS filed 01/07/2026) which discloses a self attention neural network model that utilizes multiple attention mechanisms to generate higher dimensional feature representations for classification, however fails to explicitly disclose the specific steps of a)-c) as required by the claims.
Vinyals et al. (“US 11227206 B1”) discloses generating output sequences from input sequences using a neural network and uses attention values to generate an output sequence, however the prior art fails to explicitly disclose all of the specific steps of a)-c) as required by the claims.
Chowdhery et al. (“US 20230316055 A1”) discloses performing a machine learning task on a network input to generate a network output using an attention neural network, however the prior art fails to explicitly disclose all of the specific steps of a)-c) as required by the claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL H HOANG whose telephone number is (571)272-8491. The examiner can normally be reached Mon-Fri 8:30AM-4:30PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kakali Chaki can be reached at (571) 272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL H HOANG/PRIMARY EXAMINER, Art Unit 2122
Prosecution Insights