DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is responsive to the original application filed on 9/29/2023 and the Remarks and Amendments filed on 7/16/2026. Acknowledgment is made with respect to a claim of priority to Japanese Application JP2022-212372 filed on 12/28/2022.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 3-8, 10-15, and 17-23 are rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more. The analysis of the claims will follow the 2019 Revised Patent Subject Matter Eligibility Guidance, 84 Fed. Reg. 50 (“2019 PEG”).
When considering subject matter eligibility under 35 U.S.C. 101, it must be determined whether the claim is directed to one of the four statutory categories of invention, i.e., process, machine, manufacture, or composition of matter (Step 1). If the claim does fall within one of the statutory categories, the second step in the analysis is to determine whether the claim is directed to a judicial exception (Step 2A). The Step 2A analysis is broken into two prongs. In the first prong (Step 2A, Prong 1), it is determined whether or not the claims recite a judicial exception (e.g., mathematical concepts, mental processes, certain methods of organizing human activity). If it is determined in Step 2A, Prong 1 that the claims recite a judicial exception, the analysis proceeds to the second prong (Step 2A, Prong 2), where it is determined whether or not the claims integrate the judicial exception into a practical application. If it is determined at step 2A, Prong 2 that the claims do not integrate the judicial exception into a practical application, the analysis proceeds to determining whether the claim is a patent-eligible application of the exception (Step 2B). If an abstract idea is present in the claim, any element or combination of elements in the claim must be sufficient to ensure that the claim integrates the judicial exception into a practical application, or else amounts to significantly more than the abstract idea itself.
Claim 1
Step 1: The claim recites a non-transitory computer-readable recording medium; therefore, it is directed to the statutory category of a manufacture.
Step 2A Prong 1: The claim recites, inter alia:
deleting an element included in at least one of a tensor QT and a tensor KT such that elements having a same index are left in the tensor QT and the tensor KT from among one or more elements included in the tensor QT included in a reduced Q layer in which one or more elements are reduced based on a first reduction ratio selected for the element of the Q layer and one or more elements included in the tensor KT included in a reduced K layer in which one or more elements are reduced based on a second reduction ratio selected independently of the first reduction ratio for the element of the k layer: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of deleting elements in tensor or matrices based on reduction ratios, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. For example, one can practically and mentally decide to prune an entry in a tensor or matrix based on an item in the tensor or matrix being irrelevant or unimportant per predetermined pruning ratios or thresholds.
the deleting comprises: calculating a logical product of a first index of an element included in the tensor QT except of a zero element in the tensor OT and a second index of an element included in the tensor KT except of a zero element in the tensor KT: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mathematical concept of calculating a product of two indices, which is performed through mathematical calculation as evidenced by paragraphs [0135-0141] of the originally filed specification.
deleting an element of an index not included in the logical product from the tensor OT or the tensor KT: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of deleting elements in tensor or matrices based on determined products, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. For example, one can practically and mentally decide to prune an entry in a tensor or matrix based on certain criteria.
Step 2A Prong 2: The claim does not recite any additional limitations which integrate the abstract idea into a practical application. Specifically, the additional elements consist of “a machine learning program for causing a computer to execute a process comprising” and “for an element of each of a Q layer and a K layer, the Q layer outputting a Query, the K layer outputting a Key, the Query and the Key being a result of an arithmetic operating process on an input tensor in an attention mechanism in a trained machine learning model of a neural network having the attention mechanism”.
The additional elements of “a machine learning program for causing a computer to execute a process comprising” amount to generic computer components used as a tool to perform an existing process. Thus, the additional elements amount to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
The additional element of “for an element of each of a Q layer and a K layer, the Q layer outputting a Query, the K layer outputting a Key, the Query and the Key being a result of an arithmetic operating process on an input tensor in an attention mechanism in a trained machine learning model of a neural network having the attention mechanism” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h).
Thus, even when viewed individually and as an ordered combination, these additional elements do not integrate the abstract idea into a practical application and the claim is thus directed to the abstract idea.
Step 2B: Finally, the claim taken as a whole does not contain an inventive concept which provides significantly more than the abstract idea.
The additional elements of “a machine learning program for causing a computer to execute a process comprising” amount to generic computer components used as a tool to perform an existing process. Thus, the additional elements amount to no more than a recitation of the words "apply it" (or an equivalent) or are more than mere instructions to implement an abstract idea or other exception on a computer (see MPEP § 2106.05(f)).
The additional element of “for an element of each of a Q layer and a K layer, the Q layer outputting a Query, the K layer outputting a Key, the Query and the Key being a result of an arithmetic operating process on an input tensor in an attention mechanism in a trained machine learning model of a neural network having the attention mechanism” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h).
Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 3
Step 1: A manufacture, as above.
Step 2A Prong 1: The claim recites, inter alia:
inserting a padding layer into a downstream side of the V layer, the V layer outputting a Value as a result of the arithmetic operation on the input tensor in the attention mechanism, the padding layer padding one or more elements of a tensor: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of inserting a padding layer or zero values into another layer, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
padding a tensor VT included in a reduced V layer in which one or more elements are reduced based on a third reduction ratio such that heads of the tensor VT have a same number of elements: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of inserting a padding layer or zero values into a tensor, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
Step 2A Prong 2, Step 2B: The additional element of “when the attention mechanism has a multi-head attention mechanism and each of the Q layer, the K layer, and a V layer outputs respective tensors of a plurality heads” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h). Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claim 4
Step 1: A manufacture, as above.
Step 2A Prong 1: The claim recites, inter alia:
deleting, from the tensor QT, the tensor KT, and the tensor VT, heads having a same index as a head in which all elements are zero among heads of the tensor QT, the tensor KT, and the tensor VT: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of deleting heads or information from tensors, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
Step 2A Prong 2, Step 2B: The claim does not recite any additional elements that are sufficient to integrate the judicial exceptions into a practical application or amount to significantly more than the judicial exception. As such, the claim is ineligible.
Claim 5
Step 1: A manufacture, as above.
Step 2A Prong 1: The claim recites, inter alia:
wherein the attention mechanism outputs a matrix product based on a tensor VT after the padding and the deletion of the head and a matrix product obtained by normalizing a matrix product of the tensor QT after the deleting of the element and the deleting of the heads and the tensor KT after the deleting of the element and the deleting of the head: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mathematical concept of determining or outputting a matrix product, which is performed through mathematical computation.
Step 2A Prong 2, Step 2B: The claim does not recite any additional elements that are sufficient to integrate the judicial exceptions into a practical application or amount to significantly more than the judicial exception. As such, the claim is ineligible.
Claim 6
Step 1: A manufacture, as above.
Step 2A Prong 1: The claim recites, inter alia:
the neural network outputs a result of concatenating elements of the matrix product outputted from the attention mechanism: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mathematical concept of concatenating elements of a matrix, which is performed through mathematical computation.
Step 2A Prong 2, Step 2B: The claim does not recite any additional elements that are sufficient to integrate the judicial exceptions into a practical application or amount to significantly more than the judicial exception. As such, the claim is ineligible.
Claim 7
Step 1: A manufacture, as above.
Step 2A Prong 1: The claim recites, inter alia:
inserts a zero matrix into a corresponding tensor to be input: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of inserting a zero matrix into a tensor, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
Step 2A Prong 2, Step 2B: The additional element of “wherein the padding layers are each a zero padding layer” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h). Taken alone or in combination, the additional elements of the claim do not provide an inventive concept and thus the claim is subject-matter ineligible.
Claims 8 and 10-14
Claims 8 and 10-14 recite a method (step 1: a process) to perform the steps of claims 1 and 3-7, respectively, without any additional elements that integrate the abstract ideas into a practical application or provide significantly more than the abstract idea by itself, and are thus rejected for the same reasons set forth in the rejection of claims 1 and 3-7, respectively.
Claims 15 and 17-20
Claims 15 and 17-20 recite an apparatus (step 1: a machine) using a processor and memory to perform the steps of claims 1 and 3-6, respectively, which by MPEP 2106.05(f) (“apply it”) cannot integrate an abstract idea into a practical application or provide significantly more than the abstract idea by itself, and are thus rejected for the same reasons set forth in the rejection of claims 1 and 3-6, respectively.
Claim 21
Step 1: A manufacture, as above.
Step 2A Prong 1: The claim recites, inter alia:
deleting the element included in the at least one of the tensor QT and the tensor KT such that the elements having the same index are left in the tensor QT and the tensor KT to enable a matrix product operation between the tensor QT and the tensor KT in the attention mechanism: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of deleting elements in tensor or matrices, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. For example, one can practically and mentally decide to prune an entry in a tensor or matrix based on certain criteria.
Step 2A Prong 2, Step 2B: The claim does not recite any additional elements that are sufficient to integrate the judicial exceptions into a practical application or amount to significantly more than the judicial exception. As such, the claim is ineligible.
Claim 22
Step 1: A process, as above.
Step 2A Prong 1: The claim recites, inter alia:
deleting the element included in the at least one of the tensor QT and the tensor KT such that the elements having the same index are left in the tensor QT and the tensor KT to enable a matrix product operation between the tensor QT and the tensor KT in the attention mechanism: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of deleting elements in tensor or matrices, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. For example, one can practically and mentally decide to prune an entry in a tensor or matrix based on certain criteria.
Step 2A Prong 2, Step 2B: The claim does not recite any additional elements that are sufficient to integrate the judicial exceptions into a practical application or amount to significantly more than the judicial exception. As such, the claim is ineligible.
Claim 23
Step 1: A machine, as above.
Step 2A Prong 1: The claim recites, inter alia:
deleting the element included in the at least one of the tensor QT and the tensor KT such that the elements having the same index are left in the tensor QT and the tensor KT to enable a matrix product operation between the tensor QT and the tensor KT in the attention mechanism: Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of deleting elements in tensor or matrices, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. For example, one can practically and mentally decide to prune an entry in a tensor or matrix based on certain criteria.
Step 2A Prong 2, Step 2B: The claim does not recite any additional elements that are sufficient to integrate the judicial exceptions into a practical application or amount to significantly more than the judicial exception. As such, the claim is ineligible.
Response to Arguments
Applicant's arguments and amendments, filed on 7/16/2026, with respect to the objection to the specification/title have been fully considered and are persuasive. The objection to the specification/title is withdrawn.
Applicant's arguments and amendments, filed on 7/16/2026, with respect to the 35 USC §112(b) rejection of the pending claims have been fully considered and are persuasive. The 35 USC §112(b) rejection of the pending claims is withdrawn.
Applicant's arguments and amendments, filed on 7/16/2026, with respect to the 35 USC §101 rejection of the pending claims have been fully considered and are not persuasive.
With respect to Step 2A, Prong One, Applicant argues “the operations of the noted features involve a unique technical configuration. This limitation defines a concrete, structured programmatic execution where a computer calculates the logical product of indices derived from the tensors via specific rules and programmatically deletes elements to align tensors. It does not claim a generic concept of mathematical evaluation or human thought, but rather a specialized computing process”. Examiner respectfully disagrees.
Applicant has failed to provide any evidence for the assertions that the “deleting” limitations of the independent claims are not directed towards abstract ideas in the form of mental processes or mathematical concepts. The allegation that the limitation is “a specialized computing process” does not disprove the characterization of the limitation as an abstract idea.
As stated in the rejection above, the limitation deleting an element included in at least one of a tensor QT and a tensor KT such that elements having a same index are left in the tensor QT and the tensor KT from among one or more elements included in the tensor QT included in a reduced Q layer in which one or more elements are reduced based on a first reduction ratio selected for the element of the Q layer and one or more elements included in the tensor KT included in a reduced K layer in which one or more elements are reduced based on a second reduction ratio selected independently of the first reduction ratio for the element of the k layer, under its broadest reasonable interpretation in light of the specification, encompasses the mental process of deleting elements in tensor or matrices based on reduction ratios, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. For example, one can practically and mentally decide to prune an entry in a tensor or matrix based on an item in the tensor or matrix being irrelevant or unimportant per predetermined pruning ratios or thresholds. The limitation the deleting comprises: calculating a logical product of a first index of an element included in the tensor QT except of a zero element in the tensor OT and a second index of an element included in the tensor KT except of a zero element in the tensor KT, under its broadest reasonable interpretation in light of the specification, encompasses the mathematical concept of calculating a product of two indices, which is performed through mathematical calculation as evidenced by paragraphs [0135-0141] of the originally filed specification. Last, the limitation deleting an element of an index not included in the logical product from the tensor OT or the tensor KT, under its broadest reasonable interpretation in light of the specification, encompasses the mental process of deleting elements in tensor or matrices based on determined products, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper. For example, one can practically and mentally decide to prune an entry in a tensor or matrix based on certain criteria. Applicant has failed to provide any evidence to the contrary.
With respect to Step 2A, Prong Two, Applicant argues “Independent claims 1, 8, and 15 directly improve this computer data processing and machine learning framework. By calculating the logical product of indices between the independently reduced tensors QT and KT, and programmatically deleting elements of an index not included in the logical product to dynamically synchronize the tensors such that elements having a same index are left in both tensors, independent claims 1, 8, and 15 solve a technical problem unique to artificial intelligence deployment”. Examiner respectfully disagrees.
It appears that any alleged technical improvement or integration into a practical application is reflected in the abstract ideas of the claims – i.e. the “calculating” and “deleting” limitations of the independent claims. Abstract ideas alone cannot reflect a technical improvement. See MPEP §2106.05(a) (“It is important to note, the judicial exception alone cannot provide the improvement. The improvement can be provided by one or more additional elements”). Applicant has failed to identify any additional elements in the claim language beyond the identified abstract ideas that reflect a technical improvement, integrate the abstract ideas into a practical application, or provide significantly more than the abstract ideas.
With respect to Step 2B, Applicant argues “claims 1, 8, and 15 recite the additional elements that amount to significantly more than the judicial exception by maximizing pruning capabilities via independent reduction ratios while dynamically reconciling the structural data disalignment at the index level via the cross-tensor logical product operation”. Examiner respectfully disagrees.
First, it is not clear as to what “additional elements” in the claim language Applicant refers to when asserting that these unknown additional elements maximize “pruning capabilities via independent reduction ratios while dynamically reconciling the structural data disalignment at the index level via the cross-tensor logical product operation”. Second, and as stated above, it appears that any alleged technical improvement is reflected in the abstract ideas of the claims – i.e. the “calculating” and “deleting” limitations of the independent claims. Abstract ideas alone cannot reflect a technical improvement. See MPEP §2106.05(a). Applicant has failed to identify any additional elements in the claim language beyond the identified abstract ideas that reflect a technical improvement, integrate the abstract ideas into a practical application, or provide significantly more than the abstract ideas.
Accordingly, Applicant’s arguments and amendments are not persuasive, and the 35 USC §101 rejection of the pending claims is maintained.
Applicant's arguments and amendments, filed on 7/16/2026, with respect to the 35 USC §102(a)(1) and 35 USC § 103 rejection of the pending claims have been fully considered and are persuasive.
The closest prior art of record Wang et al., “SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruning”, discloses an efficient algorithm-architecture co-design that leverages token sparsity, head sparsity, and quantization opportunities to reduce the attention computation and memory access, but fails to explicitly disclose deleting an element included in at least one of a tensor QT and a tensor KT such that elements having a same index are left in the tensor QT and the tensor KT from among one or more elements included in the tensor QT included in a reduced Q layer in which one or more elements are reduced based on a first reduction ratio selected for the element of the Q layer and one or more elements included in the tensor KT included in a reduced K layer in which one or more elements are reduced based on a second reduction ratio selected independently of the first reduction ratio for the element of the K layer, the deleting comprises: calculating a logical product of a first index of an element included in the tensor QT except of a zero element in the tensor OT and a second index of an element included in the tensor KT except of a zero element in the tensor KT, and deleting an element of an index not included in the logical product from the tensor OT or the tensor KT, all taught in the context of the remaining claim limitations and when considered as a whole, as claimed.
Further, Michel et al., “Are Sixteen Heads Really Better than One?”, discloses examining greedy algorithms for pruning down models and multi-attention mechanisms, but fails to explicitly disclose deleting an element included in at least one of a tensor QT and a tensor KT such that elements having a same index are left in the tensor QT and the tensor KT from among one or more elements included in the tensor QT included in a reduced Q layer in which one or more elements are reduced based on a first reduction ratio selected for the element of the Q layer and one or more elements included in the tensor KT included in a reduced K layer in which one or more elements are reduced based on a second reduction ratio selected independently of the first reduction ratio for the element of the K layer, the deleting comprises: calculating a logical product of a first index of an element included in the tensor QT except of a zero element in the tensor OT and a second index of an element included in the tensor KT except of a zero element in the tensor KT, and deleting an element of an index not included in the logical product from the tensor OT or the tensor KT, all taught in the context of the remaining claim limitations and when considered as a whole, as claimed.
Accordingly, the 35 USC §102(a)(1) and 35 USC § 103 rejection of the pending claims is withdrawn.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Brent Hoover whose telephone number is (303)297-4403. The examiner can normally be reached Monday - Friday 9-5 MST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abdullah Kawsar can be reached on 571-270-3169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BRENT JOHNSTON HOOVER/Primary Examiner, Art Unit 2127
