DETAILED ACTION
The present application, filed on 18/28/2023 is being examined under the AIA first inventor to file provisions.
The following is a FINAL Office Action in response to Applicant’s amendments filed on 7/17/2026.
a. Claims 1-3, 6-9, 12-15, 18-21, 24 are amended
b. Claims 4-5, 10-11, 16-17, 22-23, 25 are cancelled
Overall, claims 1-3, 6-9, 12-15, 18-21, 24 are pending and have been considered below.
Claim Rejections - 35 USC § 101
35 USC 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-3, 6-9, 12-15, 18-21, 24 are rejected under 35 USC 101 because the claimed invention is not directed to patent eligible subject matter. The claimed matter is directed to a judicial exception, i.e. an abstract idea, not integrated into a practical application, and without significantly more.
Per Step 1 of the multi-step eligibility analysis, claims 106 are directed to a computer implemented method, claims 7-12 are directed to a system, claims 13-18 are directed to a system and claims 19-24 are directed to computer executable instructions stored on a non-transitory storage medium.
Thus, on its face, each independent claim and the associated dependent claims are directed to a statutory category of invention.
[INDEPENDENT CLAIMS]
Per Step 2A.1. Independent claim 1, (which is representative of independent claims 7, 13, 19) is rejected under 35 USC 101 because the independent claim is directed to an abstract idea, a judicial exception, without reciting additional elements that integrate the judicial exception into a practical application.
The limitations of the independent claim 1 (which is representative of independent claims 7, 13, 19) recite an abstract idea, shown in bold below:
[A] An apparatus comprising: one or more processors; and one or more memories coupled with the one or more processors and storing instructions
[B] tokenizing an input text into a sequence of tokens, the input text being received at a large language model (LLM) comprising a transformer-based architecture;
[C] generating, based on the sequence of tokens, a set of three-dimensional (3D) key (K) tensors and a set of 3D value (V) tensors for an attention layer of the LLM, a sequence length corresponding to a length of the sequence of tokens;
[D] bundling a set of 3D K tensors into a single four-dimensional K tensor associated with the attention layer of a neural network model;
[E] bundling set of 3D V tensors into a single 4D V tensor associated with the attention layer;
[F] processing, via one or more application processors, the single 4D K tensor and the single 4D V tensor; for the attention layer,
[G] the attention layer receiving: the single 4D K tensor in place of the set of 3D K tensors; and the single 4D V tensor in place of the set of 3D V tensors; and
[H] executing the LLM based on processing the single 4D K tensor and the single 4D V tensor.
Independent claim 1 (which is representative of independent claims 7, 13, 19) recites: tokenizing an input text into a sequence of tokens and generating three dimensional 3D V and 3D V tensors([B], [C]) ; bundling a set of key tensors and a set of V tensors ([D], [E]); processing the single key tensor and the single value tensor and replacing 3D with 4D tensors ([F], [G]); and executing a neural network model ([H]), which, based on the claim language and in view of the application disclosure, represents a process aimed at: executing a large language model (LLM) by bundling 4D K and 4D V tensors (a mathematical operation).
The steps [C] through [H] represent mathematical operations and embody a combination that, under its broadest reasonable interpretation, covers performance of limitations expressing mathematical concepts like mathematical relationships, mathematical calculations. These fall under the Mathematical Concepts. i.e., mathematical relationships, mathematical formulas or equations, or mathematical calculations grouping of abstract ideas (see MPEP 2106.04(a)(2) I).
Accordingly, it is concluded that independent claim 1 (which is representative of independent claims 7, 13, 19) recites an abstract idea that corresponds to a judicial exception.
[INDEPENDENT CLAIMS – Additional Elements]
Per Step 2A.2. The identified abstract idea is not integrated into a practical application because the additional elements in the independent claims only amount to instructions to apply the judicial exception to a computer, or are a general link to a technological environment (see MPEP 2106.05(f); MPEP 2106.05(h)).
For example, the added elements “one or more processors,” and “one or more memories” recite computing elements at a high level of generality, generally linking the use of a judicial exception to a particular technological environment (see MPEP 2106.05(h)), or merely using a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
These additional elements of the independent claims do not preclude from carrying out the identified abstract idea executing a large language model (LLM) by bundling 4D K and 4D V tensors (a mathematical operation), and do not serve to integrate the identified abstract idea into a practical application.
Therefore, the additional claim elements of independent claim 1, (which is representative of independent claims 7, 13, 19), evaluated individually, as well as a whole, as an ordered combination, do not integrate the identified abstract idea into a practical application and the claims are directed to the recited judicial exception.
Per Step 2B. Independent claim 1 (which is representative of claims independent 7, 13, 19) does not include additional elements that are sufficient to amount to significantly more than the judicial exception because, when the independent claim is reevaluated as a whole, as an ordered combination under the considerations of Step 2B, the outcome is the same like under Step 2A.2.
Overall, it is concluded that independent claims 1, 7, 13, 19 are deemed ineligible.
[DEPENDENT CLAIMS]
Dependent claim 2, which is representative of dependent claims 8, 14, 20, recites:
transferring the processed single 4D K tensor and the processed single 4D V tensor to a specialized processor for executing the LLM.
When considered individually, these added claim elements further elaborate on the abstract idea identified in the independent claims, because the dependent claim continues to recite the identified abstract idea: executing a large language model (LLM) by bundling 4D K and 4D V tensors (a mathematical operation). The elements in this dependent claim are comparable to receiving/transmitting data, processing data, storing results or transmitting data that serves merely to implement the abstract idea using computing components for performing computer functions (corresponding to the words “apply it” or an equivalent), or merely uses a computer as a tool to perform the identified abstract idea. Thus, it is concluded that these claim elements do not integrate the identified abstract idea (executing a large language model (LLM) by bundling 4D K and 4D V tensors (a mathematical operation)) into a practical application (see MPEP 2106.05(f)(2)). Thus, the dependent claim elements are not directed to any specific improvements of the independent claims and do not practically or significantly alter how the identified abstract idea would be performed.
Therefore, dependent claim 2 (which is representative of dependent claims 8, 14, 20) is deemed ineligible.
Dependent claims 3, 6, which are representative of dependent claims 9, 12, 15, 18, 21, 24, respectively, recite:
wherein the processed single K tensor and the processed single V tensor are transferred via a high-speed communication protocol.
wherein the neural network model is a large language model (LLM).
wherein each of the single K tensor and the single V tensor is a four-dimensional (4D) tensor.
wherein the single K tensor has a shape of [number of heads, 1, depth, sequence length]; and the single V tensor has a shape of [number of heads, 1, sequence length, depth].
These further elements in the dependent claims do not perform any claimed method steps. They describe the nature, structure and/or content of other claim elements – the single K tensor; the neural network; the single K tensor and the single V tensor – and as such, cannot change the nature of the identified abstract idea (executing a large language model (LLM) by bundling 4D K and 4D V tensors (a mathematical operation)), from a judicial exception into eligible subject matter, because they do not represent significantly more (see MPEP 2106.07). The nature, form or structure of the other claim elements themselves do not practically or significantly alter how the identified abstract idea would be performed and do not provide more than a general link to a technological environment.
Therefore, dependent claims 3, 6, which are representative of dependent claims 9, 12, 15, 18, 21, 24 respectively, are deemed ineligible.
When the dependent claims are considered as a whole, as an ordered combination, the claim elements noted above appear to merely apply the abstract concept to a technical environment in a very general sense. The most significant elements, which form the abstract concept, are set forth in the independent claims. The fact that the computing devices and the dependent claims are facilitating the abstract concept is not enough to confer statutory subject matter eligibility, since their individual and combined significance do not transform the identified abstract concept at the core of the claimed invention into eligible subject matter. Therefore, it is concluded that the dependent claims of the instant application, considered individually, or as a as a whole, as an ordered combination, do not amount to significantly more (see MPEP 2106.07(a)II).
In sum, claims 1-3, 6-9, 12-15, 18-21, 24 are rejected under 35 USC 101 as being directed to non-statutory subject matter.
The prior art made of record and not relied upon which, however, is considered pertinent to applicant's disclosure:
US 20230359697 A1 Jiang; Chengquan et al. TENSOR PROCESSING A method is proposed for tensor processing. A first tensor is obtained to be applied attention, the first tensor representing a batch of inputs with variable sequence lengths. The first tensor is divided into a plurality of matrices based on the number of inputs of the batch and the number of heads of the attention, wherein a matrix of the plurality of matrices has a dimension corresponding to the sequence length of an input of the batch and a dimension corresponding to a head size of the attention. A second tensor is generated by applying the attention to the plurality of matrices respectively based on a grouped matrix multiplication, wherein a resulting matrix of the grouped matrix multiplication comprises a plurality of tiles each of which is computed by a set of threads with a shared memory. Therefore, the computations on useless tokens are avoided.
US 20240273887 A1 Wu; Chih-Wei et al. NEURAL NETWORK SYSTEM AND SIGNAL PROCESSING METHOD A neural network system and a signal processing method are provided. The neural network system includes at least one processing unit and a neural network module. The signal processing method includes: inputting a neural network input to the neural network module by the processing unit to generate an input at a previous layer of each convolutional transformer layer; performing pointwise convolution on the input by a key embedding layer based on key convolutional kernels to output a key tensor; performing convolution on the input by a value embedding layer based on value convolutional kernels to output a value tensor; performing a convolution on the cascading tensor of a first tensor and the key tensor by an attention embedding layer based on attention convolution kernels to output an attention tensor; and outputting an output tensor based on the attention tensor and the value tensor by an output module.
US 20180341479 A1 Temam; Olivier et al. ACCESSING DATA IN MULTI-DIMENSIONAL TENSORS USING ADDERS Methods, systems, and apparatus, including an apparatus for accessing a N-dimensional tensor, the apparatus including, for each dimension of the N-dimensional tensor, a partial address offset value element that stores a partial address offset value for the dimension based at least on an initial value for the dimension, a step value for the dimension, and a number of iterations of a loop for the dimension. The apparatus includes a hardware adder and a processor. The processor obtains an instruction to access a particular element of the N-dimensional tensor. The N-dimensional tensor has multiple elements arranged across each of the N dimensions, where N is an integer that is equal to or greater than one. The processor determines, using the partial address offset value elements and the hardware adder, an address of the particular element and outputs data indicating the determined address for accessing the particular element of the N-dimensional tensor.
US 20230371831 A1 Cao; Jun et al. METHOD AND APPARATUS FOR PREDICTING BLOOD PRESSURE BY FUSING CALIBRATED PHOTOPLETHYSMOGRAPHIC SIGNAL DATA A method for predicting blood pressure by fusing calibrated photoplethysmography (PPG) signal data includes obtaining real-time PPG signal data, calibrated PPG signal data, calibrated diastolic blood pressure data, and systolic blood pressure data; using a CNN+ANN model to calculate the relative blood pressure data, and generating an ANN output tensor, according to the calibrated diastolic blood pressure and systolic blood pressure data, and the ANN output tensor, performing blood pressure data calculation and generating a blood pressure tensor; if the predicted type information is a first type, performing a mean value calculation according to the blood pressure tensor, and generating diastolic blood pressure prediction data and systolic blood pressure prediction data; if the predicted type information is a second type, performing data extraction processing on the blood pressure tensor, and generating a diastolic blood pressure prediction data sequence and a systolic blood pressure prediction data sequence.
US 11514370 B1 Yu; Gyeongin et al. Selective batching for inference system for transformer-based generation tasks An inference system applies a machine-learning transformer model to a batch of requests with variable input length or variable target length or variable internal state length by selectively batching a subset of operations in the transformer model but processing requests in the batch individually for a subset of operations in the transformer model. In one embodiment, the operation to be processed individually is an attention operation of an encoder or a decoder of the transformer model. By selective batching, the inference system can allow batching operations to be performed for a batch of requests with variable input or target length or internal state length to utilize the parallel computation capabilities of hardware accelerators while preventing unnecessary computations that occur for workarounds that restrain the data of a batch of requests to a same length
US 20230177401 A1 Yu; Gyeongin et al. Selective Batching for Inference System for Transformer-Based Generation Tasks An inference system applies a machine-learning transformer model to a batch of requests with variable input length or variable target length or variable internal state length by selectively batching a subset of operations in the transformer model but processing requests in the batch individually for a subset of operations in the transformer model. In one embodiment, the operation to be processed individually is an attention operation of an encoder or a decoder of the transformer model. By selective batching, the inference system can allow batching operations to be performed for a batch of requests with variable input or target length or internal state length to utilize the parallel computation capabilities of hardware accelerators while preventing unnecessary computations that occur for workarounds that restrain the data of a batch of requests to a same length.
US 12579564 B1 Yang; Shan et al. Image-based complementary item recommendations A two-stage learning framework provides image-based complementary item recommendations. After receiving an image of a real-world scene depicting at least one object, the first stage generates feature embeddings for the real-world scene and the at least one object. The second stage generates a predicted feature embedding for at least one predicted object from the feature embeddings obtained from the first stage, generates a category embedding for the at least one predicted object from the predicted feature embedding, generates a recommended feature embedding for at least one recommended object from the predicted feature embedding and the category embedding, and outputs an identifier of the at least one recommended object, wherein the at least one recommended object is complementary to the at least one object.
US 11442775 B1 Yu; Gyeongin et al. Dynamic batching for inference system for transformer-based generation tasks An inference system applies a machine-learning transformer model to a batch of requests with variable input length or variable target length or variable internal state length by selectively batching a subset of operations in the transformer model but processing requests in the batch individually for a subset of operations in the transformer model. In one embodiment, the operation to be processed individually is an attention operation of an encoder or a decoder of the transformer model. By selective batching, the inference system can allow batching operations to be performed for a batch of requests with variable input or target length or internal state length to utilize the parallel computation capabilities of hardware accelerators while preventing unnecessary computations that occur for workarounds that restrain the data of a batch of requests to a same length.
US 11836520 B2 Yu; Gyeongin et al. Dynamic batching for inference system for transformer-based generation tasks An inference system applies a machine-learning transformer model to a batch of requests with variable input length or variable target length or variable internal sate length by selectively batching a subset of operations in the transformer model but processing requests in the batch individually for a subset of operations in the transformer model. In one embodiment, the operation to be processed individually is an attention operation of an encoder or a decoder of the transformer model. By selective batching, the inference system can allow batching operations to be performed for a batch of requests with variable input or target length or internal state length to utilize the parallel computation capabilities of hardware accelerators while preventing unnecessary computations that occur for workarounds that restrain the data of a batch of requests to a same length.
Response to Amendments/Arguments
Applicant’s submitted remarks and arguments have been fully considered.
Applicant disagrees with the Office Action conclusions and asserts that the presented claims fully comply with the requirements of 35 U.S.C. § 101 regrading judicial exceptions. Further, Applicant is of the opinion that the prior art fails to teach Applicant’s invention.
Examiner respectfully disagrees with the former.
With respect to Applicant’s Remarks as to the claims being rejected under 35 USC § 101.
Applicant submits:
a. The pending claims are not directed to an abstract idea.
b. The identified abstract idea is integrated into a practical application.
c. The pending claims amount to significantly more.
Furthermore, Applicant asserts that the Office has failed to meet its burden to identify the abstract idea and to establish that the identified abstract idea is not integrated into a practical application and that the pending claims do not amount to significantly more.
Examiner responds – The arguments have been considered in light of Applicants’ amendments to the claims. The arguments ARE NOT PERSUASIVE. Therefore, the rejection is maintained.
The pending claims, as a whole, are directed to an abstract idea not integrated into a practical application. This is because (1) they do not effect improvements to the functioning of a computer, or to any other technology or technical field (see MPEP 2106.05 (a)); (2) they do not apply or use the abstract idea to effect a particular treatment or prophylaxis for a disease or a medical condition (see the Vanda memo); (3) they do not apply the abstract idea with, or by use of, a particular machine (see MPEP 2106.05 (b)); (4) they do not effect a transformation or reduction of a particular article to a different state or thing (see MPEP 2106.05 (c)); (5) they do not apply or use the abstract idea in some other meaningful way beyond generally linking the use of the identified abstract idea to a particular technological environment, such that the claim as a whole is more than a drafting effort designated to monopolize the exception (see MPEP 2106.05 (e) and the Vanda memo).
In addition, the pending claims do not amount to significantly more than the abstract idea itself.
As such, the pending claims, when considered as a whole, are directed to an abstract idea not integrated into a practical application and not amounting to significantly more.
More specific:
Applicant submits “This pipeline is not practically performable in the human mind … ”
Examiner has carefully considered, but doesn’t find Applicant’s arguments persuasive.
The eligibility analysis in the instant office action does not make such an allegation.
Thus, the rejection is proper and has been maintained.
Applicant submits “This pipeline … is not a mathematical relationship, formula, or calculation.”
Examiner has carefully considered, but doesn’t find Applicant’s arguments persuasive.
The eligibility analysis in the instant office action has concluded that: “tokenizing an input …a set of 3D K and 3D V tensors … bundling the set of 3D K and 3D V tensors into 4D K and 4D V tensors … processing the generated 4D K and 4D V tensors … executing a LLM” is directed to a mathematical concept, particularly to mathematical relationships, mathematical calculations.
Thus, the rejection is proper and has been maintained.
Applicant submits “As discussed during the interview, the claimed arrangement improves LLM processing because it changes the data structures supplied to the attention layer and reduces the number of tensor inputs and outputs handled during transformer execution.”
Examiner has carefully considered, but doesn’t find Applicant’s arguments persuasive.
MPEP 2106.04(d)(1) discloses:
An important consideration to evaluate when determining whether the claim as a whole integrates a judicial exception into a practical application is whether the claimed invention improves the functioning of a computer or other technology .... In short, first the specification should be evaluated to determine if the disclosure provides sufficient details such that one of ordinary skill in the art would recognize the claimed invention as providing an improvement. The specification need not explicitly set forth the improvement, but it must describe the invention such that the improvement would be apparent to one of ordinary skill in the art .... Second, if the specification sets forth an improvement in technology. the claim must be evaluated to ensure that the claim itself reflects the disclosed improvement. (Emphasis added)
That is, the claimed invention may integrate the judicial exception into a practical application by demonstrating that it improves the relevant existing technology although it may not be an improvement over well-understood, routine, conventional activity. (Emphasis added)
Thus, the rejection is proper and has been maintained.
Applicant submits “The additional elements impose meaningful limits on the claim scope and are not merely linking the exception to a technological environment or using a computer as a tool to perform the abstract idea.”
Examiner has carefully considered, but doesn’t find Applicant’s arguments persuasive.
The issue at hand is not if the limits (or limitations) are meaningful, but if those limits are applied “in some other meaningful way …” (see MPEP 2106.05(e))
Thus, the rejection is proper and has been maintained.
Applicant submits “Furthermore, assuming arguendo that the claims are directed to an abstract idea, the claims recite significantly more than the judicial exception.”
Examiner has carefully considered, but doesn’t find Applicant’s arguments persuasive.
The eligibility analysis in the instant office action has determined at step 2B:
Per Step 2B. Independent claim 1 (which is representative of claims independent 7, 13, 19) does not include additional elements that are sufficient to amount to significantly more than the judicial exception because, when the independent claim is reevaluated as a whole, as an ordered combination under the considerations of Step 2B, the outcome is the same like under Step 2A.2.
Overall, it is concluded that independent claims 1, 7, 13, 19 are deemed ineligible.
Thus, the rejection is proper and has been maintained.
Applicant submits “Action provides no evidence, such as citations to publications, patents, or other documentation, that the claimed attention layer input replacement was well-understood, routine, or conventional at the time of filing.”
Examiner has carefully considered, but doesn’t find Applicant’s arguments persuasive.
It appears that applicant makes reference to the provisions of the Berkheimer Memo.
In essence, the Berkheimer Memo requires that, if an Examiner finds that certain claim feature are “well-known, routine or conventional,” the conclusion has to be accompanied by factual proof. Because the eligibility analysis in the instant Office Action has not arrived at such a conclusion, no factual proof is required (see MPEP 2106.05(d); MPEP 2106.07(a); MPEP 2106.04(a)(2)).
Thus, the rejection is proper and has been maintained.
Applicant submits “Under the 2019 Revised Patent Subject Matter Eligibility Guidance, the Examiner bears the burden of establishing that additional elements are well-understood, routine, and conventional with supporting evidence. No such evidence has been provided.”
Examiner has carefully considered, but doesn’t find Applicant’s arguments persuasive.
See response immediately above.
Thus, the rejection is proper and has been maintained.
It follows from the above that there are no meaningful limitations in the claims that transform the judicial exception into a patent eligible application such that the claims amount to significantly more than the judicial exception itself. Therefore, the rejection under 35 U.S.C. § 101 is maintained.
With respect to Applicant’s Remarks as to the claims being rejected under 35 USC § 103.
The rejection is withdrawn, as a result of the amendments. The identified prior art does not disclose:
generating, based on the sequence of tokens, a set of three-dimensional (3D) key (K) tensors and a set of 3D value (V) tensors for an attention layer of the LLM, a sequence length corresponding to a length of the sequence of tokens;
processing, via one or more application processors, the single 4D K tensor and the single 4D V tensor; for the attention layer,
the attention layer receiving: the single 4D K tensor in place of the set of 3D K tensors; and the single 4D V tensor in place of the set of 3D V tensors;
Examiner has reviewed and considered all of Applicant’s remarks. The rejection is maintained, necessitated by the fact that the rejection of the claims under 35 USC § 101 has not been overcome.
Conclusion
THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Inquiries
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Radu Andrei whose telephone number is 313.446.4948. The examiner can normally be reached on Monday – Friday 8:30am – 5pm EST. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John Hayes can be reached at 571.272.6708. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http:/www.uspto.gov/interviewpractice.
As disclosed in MPEP 502.03, communications via Internet e-mail are at the discretion of the applicant. Without a written authorization by applicant in place, the USPTO will not respond via Internet e-mail to any Internet correspondence which contains information subject to the confidentiality requirement as set forth in 35 U.S.C. 122. A paper copy of such correspondence will be placed in the appropriate patent application. The following is a sample authorization form which may be used by applicant:
“Recognizing that Internet communications are not secure, I hereby authorize the USPTO to communicate with me concerning any subject matter of this application by electronic mail. I understand that a copy of these communications will be made of record in the application file.”
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Status information for published applications may be obtained from Patent Center information webpage. Status information for unpublished applications is available to registered users through Patent Center information webpage only.
To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov.
Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (in USA or CANADA) or 571-272-1000.
Any response to this action should be mailed to:
Commissioner of Patents and Trademarks
P.O. Box 1450
Alexandria, VA 22313-1450
or faxed to 571-273-8300
/Radu Andrei/
Primary Examiner, AU 3697
Prosecution Insights