SYSTEMS AND METHODS FOR STATIC CACHED DECODING

§101 §103

DETAILED ACTION Notice of Pre-AIA or AIA Status The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . 2. This action is in response to the amendment filed on 07/22/2026. Claims 1-30 are pending in the case. This action is Final. Applicant Response 3. In Applicant’s response dated 07/22/2026, Applicant amended Claims 1, 4, 8- 9, 11-12, 23, and 26 and argued against all objections and rejections previously set forth in the Office Action dated 05/07/2026. Examiner Comments 4. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. Claim Rejections - 35 USC § 103 The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. Claims 1-30 are rejected under 35 U.S.C. 103 as being unpatentable over Lia (Pub. No.: US 20240281393 A1, Pub. Date: 2024-08-22) in view of Yu (Pub. No.: US 20230176903 A1, Pub. Date: 2023-06-08) in further view of Shazeer (NPL: Title: Fast Transformer Decoding: One Write-Head is All You Need; Dae 11/07/2019, arXiv:1911.02150v1 [cs.NE]) Regarding independent Claim 1, Lai teaches an apparatus for cached decoding (see Lai: Fig.8, [0090], “logical architecture for loading input data and storing output data of tensor computations”), the apparatus comprising: at least one memory; and at least one processor coupled to the at least one memory (see Lai: Fig.8, [0090], describing memory 850, processor 840), and configured to: receive an input token (see Lai: Fig.14, [0125], “Step 1410, where a controller of a machine-learning accelerator may receive a token indicating a data chunk becomes available in a first circular buffer of a pre-determined size from the direct memory access component.” See also Fig.5B, [0060], “an example processing for calculating embeddings from input embeddings at a self-attention layer.” ); apply a projection tensor to the input token to generate a feature tensor (see Lai: Fig.3, [0084], “the cluster activation smart DMA 706 and the cluster weight smart DMA 704 may be communicably coupled to an activation DMA 705 and a weight DMA 703, such as those illustrated in FIG. 7A, over activation DMA bus 714 and weight DMA bus 712, respectively, to provide the appropriate weights and input features to each tensor processor unit 726 in each cycle. In the example tensor processor cluster 724, each of the four tensor processor units 726A-D may operate on one-quarter of the input features allocated to tensor processor cluster 724 by the compiler, as provided by the cluster activation smart DMA 706.”) and process at least the feature tensor and at least one previous feature tensor using at least one attention calculation to generate an output token (see Lai: Fig.5C, [0063], “multi-headed self-attention computation. A first flow 570 represents a traditional multi-headed self-attention, while a second flow 580 shows an efficient variant called Fast Attention. Fast Attention implements the attention between query, key, and value embeddings in different orders. A first difference between a self-attention network and a CNN network may be that the self-attention network (for both traditional multi-headed self-attention and Fast Attention) comprises batch matrix-matrix product (bmm) operators that perform General Matrix Multiplication (GEMM) between two runtime-generated activation tensors, instead of between an activation tensor with off-line generated weight tensor.”) and the at least one previous feature tensor retrieved from a buffer (see Lai: Fig,1, [0094], “the controllers 830 may cause the one or more data chunks to be retrieved from the one or more generated addresses in the circular buffer to internal memories of one or more tensor processors 840”) and the at least one previous feature tensor previously calculated based on application of the projection tensor to a previous input token (see Lai: Fig,11c, [0099], “After generating the first address, the second address out 1030 would be calculated to be 3 by 2+1, where 2 is the current address 1010 at the end of the first address generation and +1 is an incremental value for the second address. The address wrapping engine 1040 may wrap the second address_out to 0 by 3−3. The third address_out 1030 would be calculated to be 1, by 0+1. The third address_out does not need to be wrapped. Therefore, the generated addresses 1050 corresponding to the third, fourth, and fifth rows of the input tensor at time t+2 would be 2, 0, and 1. The controller 830 may cause the three rows of the input tensor from the generated addresses 1050 to be loaded to the internal buffers of the one or more tensor processors 840.”) Lai does not teach the system wherein: the at least one previous feature tensor retrieved from a buffer for use in a current attention iteration without re-computation of the at least one previous feature tens the at least one previous feature tensor previously calculated during a previous attention iteration based on application of the projection tensor to a previous input token. an inference graph associated with the at least one attention calculation remains static across at least the previous attention iteration and the current attention iteration. generate an output based on the output token. However Yu teaches the system wherein: at least one previous feature tensor retrieved from a buffer for use in a current attention iteration without re-computation of the at least one previous feature tensor (see Yu: Fig.2B, [0050], “The self-attention block 225 is applied to the query tensor, the key cache tensor, and the value cache tensor for the current iteration to generate the attention output tensor. The attention linear block 230 is applied to the attention output tensor to generate an output tensor. The add block 235 combines the inputs and the output tensor from the attention linear block 230 to generate a combined tensor.’) at least one previous feature tensor previously calculated during a previous attention iteration based on application of the projection tensor to a previous input token (see Yu: Fig.2B, [0044], “The split block 220 is coupled to receive the output tensor from the QKV operation block 215 and split the output tensor into a query tensor, a key tensor, a value tensor for the current iteration, in the case that these tensors are not generated separately. The query tensor includes the queries for the batch of requests, the key tensor includes the keys for the batch of requests, and the value tensor includes the values for the batch of requests for the current iteration.”) generate an output based on the output token (see Yu: Fig.1, [0026], “The inference system 130 repeatedly applies a transformer model for one or more iterations to generate a sequence of output tokens (e.g., response to query). The output tokens are converted to output text as a response to the request.”) Because both Lai and Yu are in the same/similar field of endeavor of optimizing machine learning model, accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify the teaching of Lai to include the system that use in a current attention iteration without re-computation of the at least one previous feature tensor and a previous attention iteration based on application of the projection tensor to a previous input token and generate an output based on the output token as taught by Yu. One would have been motivated to make such a combination to improve latency and throughput, especially when the number of parameters of the transformer model is significantly large ( see Yu: [0004]) Lai and Yu does not teach the system wherein: an inference graph associated with the at least one attention calculation remains static across at least the previous attention iteration and the current attention iteration. However, Shazeer teaches the system wherein: an inference graph associated with the at least one attention calculation remains static across at least the previous attention iteration and the current attention iteration (see Shazeer: Pg.7, Sec.4.3, parag.10, “We ran incremental greedy inference on a batch of 1024 sequences (128 per core) using a source-sequence length of 128 tokens and a target sequence length of 128. 1 For the baseline model, the encoder part of the model took 222ms and each incremental step of the decoder took 47ms. Dividing” … Pg.7, Fn.1, “Due to system limitations requiring fixed shapes, we used padding and masking in our decoder-self-attention implementation. The memory tensors were thus padded to the maximum length (128), or to the window-size (32) in the case of local attention. Each decoding step thus took the same amount of time. An alternative implementation of incrementally growing the tensors could save time near the beginning of the sequence.”) Because both Lai, Yu and Shazeer are in the same/similar field of endeavor of optimizing machine learning model, accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify the teaching of Lai to include the system an inference graph associated with the at least one attention calculation remains static across at least the previous attention iteration and the current attention iteration as taught by Shazeer. One would have been motivated to make such a combination to provide resulting models can indeed be much faster to decode, and incur only minor quality degradation from the baseline.( see Abstract) Regarding Claim 2, As shown above, Lai, Yu and Shazeer teaches all the limitations of Claim 1. Lai further teach the apparatus the system wherein: the feature tensor is a key feature tensor, wherein the projection tensor is a key projection tensor, and wherein the buffer is a key buffer (see Lai: Fig.5B, [0060], “key embedding 555A corresponding to the input embedding 535A may be calculated by multiplying the input embedding 535A with WK 550. A value embedding 565A corresponding to the input embedding 535A may be calculated by multiplying the input embedding 535A with WV 560. Likewise, a query embedding 545B, a key embedding 555B, and a value embedding 565B corresponding to an input embedding 535B may be calculated by multiplying the input embedding 535B with WQ 540, WK 550, and WV 560, respectively.”) Regarding Claim 3, As shown above, Lai, Yu and Shazeer teaches all the limitations of Claim 1. Lai further teach the apparatus the system wherein: the feature tensor is a value feature tensor, wherein the projection tensor is a value projection tensor, and wherein the buffer is a value buffer (see Lai: Fig.8, [0090], “the size of the local memory 850 may not be large enough for some computations. At a high level, the systems and methods disclosed herein try to virtualize a memory space for the layer input and a memory space for the layer output by implementing them as circular buffers. A circular buffer as an input buffer and a circular buffer as an output buffer may be created. Each circular buffer may be of a pre-determined size. The controller 830 associated with the tensor processor 840 may wrap a generated address when the generated address goes beyond an upper or a lower limit of an address space associated with the circular buffer.”) Regarding Claim 4, As shown above, Lai, Yu and Shazeer teaches all the limitations of Claim 1. Lai further teach the apparatus the system wherein: the at least one processor is configured output the output (see Lai: Fig.8, [0098], “The controller 830 may cause the one or more tensor processors 840 to store the one or more output data chunks of the computations to the one or more addresses in the second circular buffer. Although this disclosure describes storing output of tensor computations in a circular buffer in a particular manner, this disclosure contemplates storing output of tensor computations in a circular buffer in any suitable manner.”) Regarding Claim 5, As shown above, Lai, Yu and Shazeer teaches all the limitations of Claim 1. Lai further teach the apparatus the system wherein: the at least one processor is configured to: store the feature tensor in the buffer (see Lai: Fig.8, [0090], “the size of the local memory 850 may not be large enough for some computations. At a high level, the systems and methods disclosed herein try to virtualize a memory space for the layer input and a memory space for the layer output by implementing them as circular buffers. A circular buffer as an input buffer and a circular buffer as an output buffer may be created. Each circular buffer may be of a pre-determined size. The controller 830 associated with the tensor processor 840 may wrap a generated address when the generated address goes beyond an upper or a lower limit of an address space associated with the circular buffer.”) Regarding Claim 6, As shown above, Lai, Yu and Shazeer teaches all the limitations of Claim 5. Lai further teach the apparatus processor is configured to: overwrite an invalid portion of the buffer with the feature tensor to store the feature tensor in the buffer (see Lai: Fig.10B, [0090], “. When the address_out is greater than address_max, the address_out is wrapped by subtracting the size of the circular buffer from the address_out. When the address_out is less than address_min, the address_out is wrapped by adding the size of the circular buffer to the address_out. Although this disclosure describes wrapping addresses for a circular buffer in a particular manner, this disclosure contemplates wrapping addresses for a circular buffer in any.”) Regarding Claim 7, As shown above, Lai, Yu and Shazeer teaches all the limitations of Claim 5. Lai further teach the apparatus processor is configured to: discard an oldest feature tensor from the buffer before storing the feature tensor in the buffer, wherein the oldest feature tensor is an oldest one of a plurality of feature tensors stored in the buffer, the plurality of feature tensors including the at least one previous feature tensor (see Lai: Fig.8, [0094], “The weights for the kernel may be already loaded. As the local memory 850 is not large enough for storing the whole N-by-N input tensor and N−2-by-N−2 output tensor, circular buffers are utilized for the input tensor and the output tensor. FIGS. 11A-11D illustrates the circular buffer for the input tensor. As the size of the circular buffer for the input tensor is 3, addr_min is 0 and addr_max is 2. As illustrated in FIG. 11A, the direct memory access component 820 may initialize the token counter associated with the circular buffer with −2 as the first two rows should be loaded without causing any convolution operation.”) Regarding Claim 8, As shown above, Lai, Yu and Shazeer teaches all the limitations of Claim 1. Lai further teach the apparatus processor is configured to: receive a second input token; apply the projection tensor to the second input token to generate a second feature tensor; and process at least the second feature tensor, the feature tensor, and the at least one previous feature tensor using the at least one attention calculation to generate a second output token, the feature tensor and the at least one previous feature tensor retrieved from the buffer (see Lai: Fig.5B, [0060], “key embedding 555A corresponding to the input embedding 535A may be calculated by multiplying the input embedding 535A with WK 550. A value embedding 565A corresponding to the input embedding 535A may be calculated by multiplying the input embedding 535A with WV 560. Likewise, a query embedding 545B, a key embedding 555B, and a value embedding 565B corresponding to an input embedding 535B may be calculated by multiplying the input embedding 535B with WQ 540, WK 550, and WV 560, respectively.”), wherein the output is based on the output token and the second output token (see Lai: Fig.4B, [0057], “CNN in which an output feature map 424 is generated based on an input feature map 412 in a UNet-type neural network. This type of neural network may involve high resolution input and/or output feature maps and a relatively small number of input and/or output channels.”) Regarding Claim 9, As shown above, Lai, Yu and Shazeer teaches all the limitations of Claim 8. Lai further teach the apparatus processor is configured to: output the output (see Lai: Fig.7, [0072], “the egress component may be configured to read a second block of data from a third address of the data buffer, process the second block of data with an egress modification function, and store the second block to a fourth address of the destination memory at an iteration of the loop among the n-dimensional loops.”) Regarding Claim 10, As shown above, Lai, Yu and Shazeer teaches all the limitations of Claim 1. Lai further teach the apparatus processor is configured to: retrieve the at least one previous feature tensor from the buffer (see Lai: Fig.7, [0082], “cluster-level controller 730 may use the synchronizers to generate a token indicating that tensor processor units 726 have completed the common series of operations and that the tensor data was processed. In one embodiment, cluster-level controller 730 may send the token to activation DMA controller 716 such that activation DMA controller 716 may instruct cluster activation smart DMA 706 to retrieve additional tensor data from data buffer 707 to distribute to tensor processor units 726 for further processing in lockstep the input embedding 535B with WQ 540, WK 550, and WV 560, respectively.”) Regarding Claim 11, As shown above, Lai, Yu and Shazeer teaches all the limitations of Claim 5. Lai further teach the apparatus wherein: the input token is based on an encoder output of an encoder, wherein the at least one attention calculation and the projection tensor are part of a decoder (see Lai: Fig.7D, [0083], “custer-level controller 730 receives tensor instructions (e.g., coarse-grained tensor instructions) over instruction bus 710. Each coarse-grained tensor instruction sent to a tensor processor cluster 724 may encode information usable by the tensor processor cluster 724 to perform a multi-cycle operation corresponding to a part of a single neural network layer..”) Regarding Claim 12, As shown above, Lai, Yu and Shazeer teaches all the limitations of Claim 11. Lai further teach the apparatus further comprising: the decoder wherein a decoder output of the decoder includes the output and is based on the output token (see Lai: Fig.15, [0129], “processor 1502 includes hardware for executing instructions, such as those making up a computer program. As an example and not by way of limitation, to execute instructions, processor 1502 may retrieve (or fetch) the instructions from an internal register, an internal cache, memory 1504, or storage 1506; decode and execute them; and then write one or more results to an internal register, an internal cache, memory 1504, or storage 1506. I.”) Regarding Claim 13, As shown above, Lai, Yu and Shazeer teaches all the limitations of Claim 11. Lai further teach the apparatus wherein: an input of the encoder includes a first string of text, wherein an output of the decoder includes a second string of text that is based on the first string of text (see Lai: Fig.7D, [0083], “custer-level controller 730 receives tensor instructions (e.g., coarse-grained tensor instructions) over instruction bus 710. Each coarse-grained tensor instruction sent to a tensor processor cluster 724 may encode information usable by the tensor processor cluster 724 to perform a multi-cycle operation corresponding to a part of a single neural network layer.”) Regarding Claim 14, As shown above, Lai, Yu and Shazeer teaches all the limitations of Claim 1. Lai further teach the apparatus herein processor is configured to: the at least one attention calculation receives three inputs, the three inputs including a query input and a key input and a value input, and wherein one of the three inputs includes at least the feature tensor and the at least one previous feature tensor (see Lai: Fig.5B, [0060], “an example processing for calculating embeddings from input embeddings at a self-attention layer. Each self-attention layer may maintain three matrices: WQ 540, WK 550, and WV 560. A query embedding 545A corresponding to an input embedding 535A may be calculated by multiplying the input embedding 535A with WQ 540. A key embedding 555A corresponding to the input embedding 535A may be calculated by multiplying the input embedding 535A with WK 550.”) Regarding Claim 15, As shown above, Lai, Yu and Shazeer teaches all the limitations of Claim 1. Lai further teach the apparatus wherein: the at least one attention calculation includes a scaling function that uses a scaling factor dk (see Lai: Fig.5B, [0061], “After calculating query embeddings 545A, 545B, and 545C, key embeddings 555A, 555B, and 555C, and value embeddings 565A, 565B, and 565C corresponding to input embeddings 535A, 535B, and 535C, the self-attention layer may calculate self-attention scores for all the possible pairs of input embeddings.”) Regarding Claim 16, As shown above, Lai, Yu and Shazeer teaches all the limitations of Claim 1. Lai further teach the apparatus wherein : the at least one attention calculation includes a mask configured to confine an attention span (see Lai: Fig.5C, [0063], “wo example flows for multi-headed self-attention computation. A first flow 570 represents a traditional multi-headed self-attention, while a second flow 580 shows an efficient variant called Fast Attention. Fast Attention implements the attention between query, key, and value embeddings in different orders. A first difference between a self-attention network and a CNN network may be that the self-attention network (for both traditional multi-headed self-attention and Fast Attention) comprises batch matrix-matrix product (bmm) operators that perform General Matrix Multiplication (GEMM) between two runtime-generated activation tensors, instead of between an activation tensor with off-line generated weight tensor..”) Regarding Claim 17, As shown above, Lai, Yu and Shazeer teaches all the limitations of Claim 1. Lai further teach the apparatus wherein: the mask is dependent on an iteration of the at least one attention calculation (see Lai: Fig.5A, [0059], “The self-attention layer 512 of the first encoder 510 may produce output embeddings 515A, 515B, and 515C, which would be processed by the feed forward layer 514 of the first encoder 510. The output of the feed forward layer 514 may be provided to the self-attention layer of a second encoder (not shown in FIG. 5A) as input. As the encoding component illustrated in FIG. 5A comprises N encoders, the Nth encoder 520 may be the last encoder of the encoding component. The Nth encoder 520 may take output embeddings of an N−1st encoder as input. The self-attention layer 522 of the 520 may produce embeddings 525A, 525B, and 525C by processing the output embeddings of the N−1st encoder (not shown in FIG. 5A). The embeddings 525A, 525B, and 525C may be processed through the feed forward layer 524 of the Nth encoder 520. Output embeddings of the feed forward layer 524 may be provided to the decoding component of the Transformer architecture.”) Regarding Claim 18, As shown above, Lai, Yu and Shazeer teaches all the limitations of Claim 1. Lai further teach the apparatus wherein: at least one attention calculation includes a SoftMax function configured to normalize at least one weight value (see Lai: Fig.5C, [0063], “A first difference between a self-attention network and a CNN network may be that the self-attention network (for both traditional multi-headed self-attention and Fast Attention) comprises batch matrix-matrix product (bmm) operators that perform General Matrix Multiplication (GEMM) between two runtime-generated activation tensors, instead of between an activation tensor with off-line generated weight tensor. Another difference between the self-attention network and the CNN network may be that various normalization operators including softmax operators and layer normalization (L2-N) operators with runtime-generated scaling factors instead of batch normalizations with offline-generated scaling factors.”) Regarding Claim 19, As shown above, Lai, Yu and Shazeer teaches all the limitations of Claim 1. Lai further teach the apparatus wherein: an inference graph associated with the at least one attention calculation is static (see Lai: Fig.1, [0041], “Construction of a neural network model, or a machine-learning model in general, may include a learning stage, which may also be referred to as a training stage, and an inference stage, which may also be referred to as an operational, execution, or service stage. In the learning stage, the neural network may be trained for a specific purpose and may be provided with a set of training examples, including training inputs and training outputs provided as input-output training pairs, and optionally including a set of validation examples to test the progress of the training. Regarding Claim 20, As shown above, Lai, Yu and Shazeer teaches all the limitations of Claim 5. Lai further teach the apparatus wherein: initialize the buffer, wherein the buffer is sized according to a first dimension and a second dimension, wherein the first dimension of the buffer is based on a maximum context length, wherein the second dimension of the buffer is based on a size of the input token (see Lai: Fig.3A, [0047], “each 2D kernel of filter 300-m is applied in a convolution operation over the elements in a respective channel of input feature map 304. For example, a first 2D kernel of filter 300-m provides the weights that are multiplied by respective values of the elements in an r×s sized portion 302-1 of the elements of a first channel of input feature map 304, a second 2D kernel of filter 300-m provides the weights that are multiplied by respective values of the elements in an r×s sized portion 302-2 of the elements of a second channel of input feature map 304, and so on, such that a final 2D kernel of filter 300-m provides the weights that are multiplied by respective values of the elements in an r×s sized portion 302-3 of the elements of the last channel of input feature map 304.”) Regarding Claim 21, As shown above, Lai, Yu and Shazeer teaches all the limitations of Claim 1. Lai further teach the apparatus processor is configured to: maintain a counter tracking a number of feature tensors cached in the buffer (see Lai: Fig.5C, [0064], “The ML accelerators described herein employ a multi-level control architecture designed to optimally exploit parallelism provided by tensor processor units in the ML accelerator. These machine learning accelerators may include one or more tensor processor clusters, each of which may include multiple tensor processor units.”) Regarding Claim 22, As shown above, Lai, Yu and Shazeer teaches all the limitations of Claim 1. Lai further teach the apparatus processor is configured to: maintain a counter tracking a number of iterations of the at least one attention calculation (see Lai: Fig.9B, [0092], “the controller 830 may determine that the value of the token counter is zero. In response to the determination, the controller 830 may cause one or more 3×3 convolution operations on the three rows of the tensor to be performed by one or more tensor processors 840. Although this disclosure describes suppressing causing computations when a value of a token counter is in a particular range in a particular manner, this disclosure contemplates suppressing causing computations when a value of a token counter is in a particular range in any suitable manner.”) Regarding independent Claim 23, Claim 23 is directed to a method claim and has similar/same claim limitations as claim 1 and is rejected under the same rationale. Regarding Claim 24, Claim 24 is directed to a method claim and has similar/same claim limitations as claim 2 and is rejected under the same rationale. Regarding Claim 25, Claim 25 is directed to a method claim and has similar/same claim limitations as claim 3 and is rejected under the same rationale. Regarding Claim 26, Claim 26 is directed to a method claim and has similar/same claim limitations as claim 4 and is rejected under the same rationale. Regarding Claim 27, Claim 27 is directed to a method claim and has similar/same claim limitations as claim 5 and is rejected under the same rationale. Regarding Claim 28, Claim 28 is directed to a method claim and has similar/same claim limitations as claim 6 and is rejected under the same rationale. Regarding Claim 29, Claim 29 is directed to a method claim and has similar/same claim limitations as claim 7 and is rejected under the same rationale. Regarding Claim 30, Claim 27 is directed to a method claim and has similar/same claim limitations as claim 10 and is rejected under the same rationale Response to Arguments Claim Rejections - 35 U.S.C. § 101, Regarding the 35 U.S.C. 101 rejection for being directed non-statutory subject matter has been sustained based on applicant amendments and. Therefore, the 35 U.S.C. 101 rejection has been sustained. Claim Rejections - 35 U.S.C. § 103, Applicant’s arguments with respect to claim amendments have been considered but are moot considering the new combination of references being used in the current rejection. The new combination of references was necessitated by Applicant’s claim amendments. Therefore, the claims are rejected under the new combination of references as indicated above. Conclusion The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. PGPUB NUMBER: INVENTOR-INFORMATION: TITLE / DESCRIPTION US 10846096 B1 Chung; Kiuk Title: Batch Processing Of Requests For Trained Machine Learning Model Description: This disclosure generally relates to accelerators for machine learning models and, more particularly, to circular buffers for input and output of tensor computations. US 20250124105 A1 Jha, Gopi Krishna Title: Efficient token pruning in transformer-based neural networks Description: Deep neural networks (DNNs) are used extensively for a variety of artificial intelligence applications ranging from computer vision to speech recognition and natural language processing due to their ability to achieve high accuracy. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action. Any inquiry concerning this communication or earlier communications from the examiner should be directed to ZELALEM W SHALU whose telephone number is (571)272-3003. The examiner can normally be reached M- F 0800am- 0500pm. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Cesar Paula can be reached at (571) 272-4128. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /Zelalem Shalu/Examiner, Art Unit 2145 /CESAR B PAULA/Supervisory Patent Examiner, Art Unit 2145