DETAILED ACTION
Response to Arguments
Claim objection. The objection to claim 15 is withdrawn based on amendment to claim 15.
Specification.
Applicant asserts that the specification discloses non-transitory artificial neural network structure (Remarks p. 7). Applicant points to paragraphs 76-77, 103 that the specification describes that various memories (machine-readable media) may be configured with weights activations and instructions that implement neural network structures (Remarks p. 7-8).
Examiner respectfully disagrees that these arguments support the question of antecedent basis for claim terminology. Neither of these paragraphs provide any discussion of the non-transitory nature of the artificial neural network structure itself. These paragraphs recite processing elements, memory, and controllers of processing elements configured by instructions stored in memory, but no reference at all is made to the structure of the artificial neural network itself being non-transitory.
Applicant further points to the specification describing non-volatile memories (remarks p. 9).
Examiner respectfully disagrees that these arguments support the question of antecedent basis for claim terminology. Non-volatile (persistent) memory is not related to non-transitory (non carrier wave).
35 USC 112(b).
Applicant asserts there is not ambiguity as to claims 9-20, and that introducing a claim by reciting an element in the preamble and then elaborating on the structure in the body of the claim (remarks p. 9).
Examiner respectfully disagrees. The use, in the preamble of multiple “computer system comprising”, wherein the first recites a “computer system comprising a non-transitory artificial neural network structure”, and the second reciting “the computer system comprising: one or more processors; the non-transitory artificial neural network artificial neural network structure …” not only renders it unclear where the preamble begins and ends, but also unclear what the relationship is between the non-transitory artificial neural network structure and the one or more processors.
35 USC 101. Applicant sets forth no new arguments as to the rejection under 35 USC 101. Examiner reasons set forth in the office action dated 02/13/26 apply equally to the present rejection.
Specification
The specification is objected to as failing to provide proper antecedent basis for the claimed subject matter. See 37 CFR 1.75(d)(1) and MPEP § 608.01(o). Correction of the following is required. Claim 9, and claim 19 recite, with further recitation in dependent claims: “A computer system comprising a non-transitory artificial neural network structure”. The specification discloses only non-transitory machine readable media comprising machine-executable instructions. See specification [0042]. There is no disclosure of a non-transitory artificial neural network structure.
Furthermore, the specification is objected to for failing to provide proper antecedent basis for “one or more machine-readable memories” as in claims 26 and 27 for the same reasons set forth above.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 9-20, and 26-27 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 9 recites “A computer system comprising a non-transitory artificial neural network structure, the computer system comprising: one or more processors; the non-transitory artificial neural network structure comprising…”. It is unclear what is meant by a non-transitory artificial neural network structure. The specification discloses only non-transitory machine readable media comprising machine-executable instructions. See specification [0042]. It is unclear whether the non-transitory artificial neural network structure refers to the structure comprising machine readable media, which is unclaimed, or whether there is some unknown structure of the artificial neural network that is non-transitory.
Furthermore the preamble recites two instances of the computer system comprising: “A computer system comprising a non-transitory artificial neural network structure” and “the computer system comprising: one or more processors”. This renders the claim indefinite, as it is unclear where the preamble begins and ends, and it is unclear what the relationship is between the non-transitory artificial neural network structure and the one or more processors. Claims 10-18 inherit the same deficiency as claim 9 based on dependence. Claim 19 recites substantially the same preamble limitations as claim 9 and is rejected for the same reasons. Claim 20 inherits the same deficiency as claim 19 based on dependence.
Claims 26 and 27 recite “at least one machine-readable memory configured to cause the one or more processors to implement an artificial neural network structure (claim 26), a transformer artificial neural network structure (claim 27)”. It is unclear how the memory causes the one or more processors to implement each structure. It is unclear if the memory on its own causes the implementation or if something stored on the memory such as machine-executable instructions as recited in [0042] cause the one or more processors to implement each structure.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 9-20, and 26-27 are rejected under 35 U.S.C. § 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
Regarding claim 9, under the Alice framework Step 1, claim 9 falls within the four statutory categories of patentable subject matter identified by 35 USC 101: a process, machine, manufacture or a composition of matter.
Under the Alice framework Step 2A prong 1, claim 9 recites mathematical concepts of mathematical calculations including raising a vector to the power of two, computing a maximum of a vector. Specifically, the claim recites the following mathematical calculations:
generate an unnormalized Softmax vector from the input vector by:
raising elements of the input vector to powers of two; and
computing an integer vector maximum of the input vector.
See for example [0040], which further describes these mathematical calculations in terms of equations. For these reasons claim 9 recites mathematical concepts.
Under the Alice framework Step 2A prong 2 analysis, the claim recites the following additional elements: a computer system comprising a non-transitory artificial neural network structure, the computer system comprising: one or more processors; the non-transitory artificial neural network structure comprising: one or more feed-forward layers; at least one of the Softmax layers configured to operate the one or more processor. The claim does no more than merely “apply it” in a computer system, comprising one or more processors. Furthermore, the computer system comprising a non-transitory artificial neural network structure comprising one or more feed-forward layers, and one or more softmax layers coupled to the one or more feed-forward layers merely generally link the use of the mathematical concepts to a particular technological environment or field of use, the field of use being an artificial neural network and the particular technological environment being the one or more feed forward layers and one or more software layers coupled to the one or more feed forward layers. The claim does not provide a specific structure or specifically limit these additional elements in a meaningful way beyond the generically recited feed-forward and SoftMax layers. For these reasons, claim 9 is not integrated into a practical application.
Under the Alice Framework Step 2B analysis, the claim considered individually and as an ordered combination does not include additional elements that are sufficient to amount to significantly more than the abstract idea. The claim does no more than merely generally link the use of the mathematical concepts to a particular technological environment or field of use. Furthermore artificial neural networks comprising one or more feed-forward layers; and one or more SoftMax layers coupled to the one or more feed-forward layers are well understood, routine, and conventional. See e.g., M. Gormley, Neural Networks, 10-601B Introduction to Machine Learning, Carnegie Mellon School of Computer Science, 2016 (hereinafter “Gormley”), which teaches an overview of neural network layer architectures including on ore more two feed forward layers, and one SoftMax layer (p. 58). See also applicant arguments p. 7, 2/24/25 describing a softmax layer as a common structural component of various types of neural networks. . For these reasons claim 9 does not amount to significantly more than the abstract idea.
Claims 10-11 are rejected for at least the reasons set forth with respect to claim 9. Under the Step 2A prong 1 analysis, claims 10-11 further mathematically limit claim 9. Under the Step 2A prong 2 and Step 2B analysis, claims 10-11 further includes the additional element of the one or more processors configured to operate to perform further mathematical calculations. This further limitation merely recites “apply it” in a computer.
Claim 12 is rejected for at least the reasons set forth with respect to claim 9. Under the Step 2A prong 1 analysis, claim 12 further mathematically limits claim 9. Under the Step 2A prong 2 and Step 2B analysis, claim 12 includes the following further additional elements: utilize a plurality of processing elements to perform further mathematical calculations. This further limitation merely recites “apply it” in a computer.
Claims 13-16 are rejected for at least the reasons set forth with respect to claim 12. Claims 13-16 merely further mathematically limit the mathematical concepts of claim 12. Claims 13-16 include no further additional elements that would require further analysis under Step 2A prong 2 and Step 2B beyond those recited in claim 12.
Claims 17 and 18 are rejected for at least the reasons set forth with respect to claims 9. Claims 17 and 18 further mathematically limits claim 1 and includes the following further additional elements: performing the mathematical calculations in a single execution loop.
Under the step 2A prong 2 analysis, performing mathematical calculations in an execution loop comprises an insignificant extra solution activity. Furthermore the execution of these mathematical calculations in a single execution loop is a direct result of the mathematical calculations, which only require one loop to perform the calculations. See figure 7A versus 7B and 7C, which include only one “for” loop to compute powers and sum of powers of two. For these reasons claims 17 and 18 are not integrated into a practical application.
Under the Step 2B analysis, the use of an execution loop is well understood, routine and conventional activity. See Patterson, Ch 2, p. 92 describing the use of loops. For these reasons, claim 17 and 18 does not amount to significantly more than the abstract idea.
Regarding claim 19, under the Alice framework Step 1, claim 19 falls within the four statutory categories of patentable subject matter identified by 35 USC 101: a process, machine, manufacture or a composition of matter.
Under the Alice framework Step 2A prong 1, claim 9 recites mathematical concepts of mathematical calculations including raising a vector to the power of two, computing a maximum of a vector. Specifically, the claim recites the following mathematical calculations:
generate an unnormalized Softmax vector from an input vector comprising x elements by:
raising element of the input vector to powers of two; and
computing an integer vector maximum of the input vector.
See for example [0040], which further describes these mathematical calculations in terms of equations. For these reasons claim 19 recites mathematical concepts.
Under the Alice framework Step 2A prong 2 analysis, the claim recites the following additional elements: a transformer artificial neural network structure in a computer system, the transformer artificial neural network structure comprising: a self-attention layer; and an encoder-decoder attention layer; each of the self-attention layer and the encoder-decoder attention layer comprising a SoftMax layer configured to operate the one or more processors. The claim does no more than merely generally link the use of the mathematical concepts to a particular technological environment or field of use, the field of use being a transformer artificial neural network and the particular technological environment being the self-attention layer, the encoder-decoder attention layer each comprising a SoftMax layer. The claim does not provide a specific structure or specifically limit these additional elements in a meaningful way beyond the generically recited layers. For these reasons, claim 19 is not integrated into a practical application.
Under the Alice Framework Step 2B analysis, the claim considered individually and as an ordered combination does not include additional elements that are sufficient to amount to significantly more than the abstract idea. The claim does no more than merely generally link the use of the mathematical concepts to a particular technological environment or field of use. Furthermore transformer artificial neural networks comprising a self-attention layer; and an encoder-decoder attention layer; each of the self-attention layer and the encoder-decoder attention layer comprising a SoftMax layer are well understood, routine, and conventional. See e.g., Z Hu et al, Lecture 16: Building Blocks of Deep Learning, overview of CNNs, RNNs, and attention, Carnegie Mellon University, 2019 (hereinafter “Hu”), and LP Morency, Tutorial on Multimodal Machine Learning, MultiComp Lab, Carnegie Mellon University, 2017 (hereinafter “Morency’) which teach overviews of neural networks including self-attention layers, encoder-decoder attention layers, and layers comprising SoftMax layers (Hu Attention Mechanisms, Transformer section), (Morency Autoencoder slide showing encoder-decoder layers, attention model for machine translation slide). See also Applicant arguments p.7, 2/24/25 describing the softmax layer as a common structural component. For these reasons claim 19 does not amount to significantly more than the abstract idea.
Claim 20 is rejected for at least the reasons set forth with respect to claims 19. Claims 10 further mathematically limits claim 19 and includes the following further additional elements: performing the mathematical calculations in a single execution loop.
Under the step 2A prong 2 analysis, performing mathematical calculations in an execution loop comprises an insignificant extra solution activity. Furthermore the execution of these mathematical calculations in a single execution loop is a direct result of the mathematical calculations, which only require one loop to perform the calculations. See figure 7A versus 7B and 7C, which include only one “for” loop to compute powers and sum of powers of two. For these reasons claim 20 is not integrated into a practical application.
Under the Step 2B analysis, the use of an execution loop is well understood, routine and conventional activity. See Patterson, Ch 2, p. 92 describing the use of loops. For these reasons, claim 20 does not amount to significantly more than the abstract idea.
Regarding claim 26, under the Alice framework Step 1, claim 26 falls within the four statutory categories of patentable subject matter identified by 35 USC 101: a process, machine, manufacture or a composition of matter.
Under the Alice framework Step 2A prong 1, claim 26 recites mathematical concepts of mathematical calculations including raising a vector to the power of two, computing a maximum of a vector. Specifically, the claim recites the following mathematical calculations:
generate an unnormalized Softmax vector from an input vector comprising x elements by:
raising element of the input vector to powers of two; and
computing an integer vector maximum of the input vector.
See for example [0040], which further describes these mathematical calculations in terms of equations. For these reasons claim 26 recites mathematical concepts.
Under the Alice framework Step 2A prong 2 analysis, the claim recites the following additional elements: a computer system comprising: one or more processors; at least one machine-readable memory configured to cause the one or more processors to implement an artificial neural network structure, the artificial neural network structure comprising: one or more feed-forward layers; and one or more Softmax layers coupled to the one or more feed-forward layers; at least one of the Softmax layers configured to operate the one or more processors. The claim does no more than merely generally link the use of the mathematical concepts to a particular technological environment and/or field of use, the field of use being an artificial neural network, and the particular technological environment being one or more feed-forward layers and one or more Softmax layers. The claim does not provide a specific structure or specifically limit these additional elements in a meaningful way beyond the generically recited layers. For these reasons, claim 26 is not integrated into a practical application.
Under the Alice Framework Step 2B analysis, the claim considered individually and as an ordered combination does not include additional elements that are sufficient to amount to significantly more than the abstract idea. The claim does no more than merely generally link the use of the mathematical concepts to a particular technological environment and/or field of use. Furthermore feed forward layers of an artificial neural network including a SoftMax layer are well understood, routine, and conventional. See e.g., Gormley. which teaches an overview of neural network layer architectures including on ore more two feed forward layers, and one SoftMax layer (p. 58). See also applicant arguments p. 7, 2/24/25 describing a softmax layer as a common structural component of various types of neural networks. For these reasons claim 26 does not amount to significantly more than the abstract idea.
Regarding claim 27, under the Alice framework Step 1, claim 27 falls within the four statutory categories of patentable subject matter identified by 35 USC 101: a process, machine, manufacture or a composition of matter.
Under the Alice framework Step 2A prong 1, claim 9 recites mathematical concepts of mathematical calculations including raising a vector to the power of two, computing a maximum of a vector. Specifically, the claim recites the following mathematical calculations:
generate an unnormalized Softmax vector from an input vector comprising x elements by:
raising element of the input vector to powers of two; and
computing an integer vector maximum of the input vector.
See for example [0040], which further describes these mathematical calculations in terms of equations. For these reasons claim 27 recites mathematical concepts.
Under the Alice framework Step 2A prong 2 analysis, the claim recites the following additional elements: a computer system comprising: one or more processors; one or more machine-readable memories configured to cause the one or more processors to implement a transformer artificial neural network structure comprising: a self-attention layer; and an encoder-decoder attention layer; each of the self-attention layer and the encoder-decoder attention layer comprising a SoftMax layer configured to operate the one or more processors. The claim does no more than merely generally link the use of the mathematical concepts to a particular technological environment or field of use, the field of use being a transformer artificial neural network and the particular technological environment being the self-attention layer, the encoder-decoder attention layer each comprising a SoftMax layer. The claim does not provide a specific structure or specifically limit these additional elements in a meaningful way beyond the generically recited layers. For these reasons, claim 27 is not integrated into a practical application.
Under the Alice Framework Step 2B analysis, the claim considered individually and as an ordered combination does not include additional elements that are sufficient to amount to significantly more than the abstract idea. The claim does no more than merely generally link the use of the mathematical concepts to a particular technological environment or field of use. Furthermore transformer artificial neural networks comprising a self-attention layer; and an encoder-decoder attention layer; each of the self-attention layer and the encoder-decoder attention layer comprising a SoftMax layer are well understood, routine, and conventional. See e.g.,Hu and Morency, which teach overviews of neural networks including self-attention layers, encoder-decoder attention layers, and layers comprising SoftMax layers (Hu Attention Mechanisms, Transformer section), (Morency Autoencoder slide showing encoder-decoder layers, attention model for machine translation slide). See also Applicant arguments p.7, 2/24/25 describing the softmax layer as a common structural component. For these reasons claim 27 does not amount to significantly more than the abstract idea.
Allowable Subject Matter
For the reasons set forth in the office action dated 01/25/24, claims 9-20, and 26-27 would be allowable if rewritten to overcome the rejections under 35 USC 101.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to EMILY E LAROCQUE whose telephone number is (469)295-9289. The examiner can normally be reached on 10:00am - 1200pm, 2:00pm - 8pm ET M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor Andrew Caldwell can be reached on 571-272-3702. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/EMILY E LAROCQUE/ Primary Examiner, Art Unit 2182
Prosecution Insights