DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
This Office Action is in response to the application filed on 06/20/2025. Claims 1-20 have been examined.
Priority
Receipt is acknowledged of certified copies of papers submitted under 35 U.S.C. 119(a)- (d), which have been placed of record in the file.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 06/20/2025, and 02/23/2026 filed in accordance with the provisions of 37 CFR 1.97. Accordingly, it is being considered by the examiner.
Specification
The specification has not been reviewed in detail to determine the presence of all possible minor errors. Applicant's attention is directed to the specification, and applicant is requested to make any necessary corrections of which applicant is aware.
Allowable Subject Matter
Claims 4 and corresponding electronic device claim 14 are objected to as depending from a rejected base claim. However, if rewritten in independent form to include all of the limitations of the base claim and any intervening claims, claims 4 and 14 would be allowable.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C.
102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction
of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status
The following is a quotation of 35 U.S.C. 103 which forms the basis for all
obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention
is not identically disclosed as set forth in section 102, if the differences between the claimed
invention and the prior art are such that the claimed invention as a whole would have been obvious
before the effective filing date of the claimed invention to a person having ordinary skill in the art
to which the claimed invention pertains. Patentability shall not be negated by the manner in which
the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35
U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness
Claims 1, 5, 7-11, 15, and 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over LIU YUHAN ET AL: “CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Serving” Proceeding of the ACM/IEEE 27TH INTERNATIONAL CONFERENCE ON MODEL DRIVEN ENGINEERING LANGUAGES AND SYSYTEMS, ACM, New York, NY, USA, date 4 August 2024, pages 38-56 hereinafter “Liu” in view of Howard et al (US-20150245067-A1) hereinafter “Howard” further in view of KOYUNCU et al. (US-20240267568-A1) hereinafter “Koyuncu”.
Regarding Claim 1 Liu-Howard-Koyuncu
Liu discloses 1. A compression method implemented by at least one hardware processor, (Liu, page 44, section 6, “implement CacheGen with about 2K lines of code in Python, and about 1K lines of CUDA kernel code, based on PyTorch v2.0 and CUDA 12.0.” GPU based encoding and decoding.) comprising:
acquiring attention data generated in a layer of a deep learning model based on an attention mechanism; (Liu, Abstract “CacheGen uses a custom tensor encoder, leveraging KV cache’s distributional properties to encode a KV cache into more compact bitstream representations”
converting a data format of the attention data into a converted data format, (Liu, page, 43, section 5.2, “After partitioning the tokens into groups, CacheGen uses quantization to reduce the precision of elements (floating points) in a KV cache so that they can be represented by fewer bits.”) . . . and
generating first compressed data by inputting . . . attention data to an encoder model . . . (Liu, page 43, section 5.2, “After quantizing the KV cache into discrete symbols, CacheGen uses arithmetic coding [135] (AC) to losslessly compress the delta tensors and anchor tensors of a context into bitstreams.”)
Liu does not expressly disclose
wherein a bit pattern of the attention data is maintained in converting the data format of the attention data into the converted data format;
realigning, based on data characteristics of input data of the deep learning model, the attention data in the converted data format;
the realigned…. based on an artificial neural network.
However, in the same field of endeavor HowardKim discloses more explicitly the following:
wherein a bit pattern of the attention data is maintained in converting the data format of the attention data into the converted data format; (Howard, [0036] “A bit vector 706 has the same number of bits as the number of coefficients in the transform coefficient list, and there is a one-to-one correspondence between coefficients in the coefficient list and bits in the single entity or bit vector. The bit vector thus represents a significance map for the one-dimensional list 704 of transform coefficients. Setting each bit in the bit vector where the corresponding coefficient in the coefficient list is zero fills the bit vector. The bit vector is then reinterpreted as an integer 708. An arithmetic coder 710 encodes the integer 708, with the context being identified as the "bit vector" context 712. The arithmetic coder outputs bits to a bitstream 714. The arithmetic coder 710 is as described above and illustrated in FIG. 5.”)
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Liu with Howard to convert the data format of the attention data into converted format while maintaining the bit pattern of the attention data during the conversion, as taught by Howard, in order to improve coding and decoding by enabling efficient arithmetic coding of the converted data (Howard, [0015]- [0016])
Liu-Howard does not expressly disclose
realigning, based on data characteristics of input data of the deep learning model, the attention data in the converted data format;
the realigned…. based on an artificial neural network.
However, in the same field of endeavor Koyuncu discloses more explicitly the following:
realigning, based on data characteristics of input data of the deep learning model, the attention data in the converted data format; (Koyuncu, [0205] “An arrangement 830 of the plurality of segments is processed by one or more layers of a neural network. Such an arrangement may be predefined… in the spatial and/or channel direction. An arrangement of a first exemplary embodiment may include a reshaping of the latent tensor into a sequential form…” [0210] “the segments may be arranged in a predefined order, wherein segments with a same spatial coordinate….” [0258] “A processing circuitry within any of these exemplary devices is configured to separate the latent tensor into a plurality of segments in the spatial dimensions, each segment including at least one latent tensor element, to process an arrangement of the plurality of segments by one or more layers of a neural network, including at least one attention layer, and to obtain a probability model for the entropy encoding of a current element of the latent tensor based on the processed plurality of segments.”)
the realigned…. based on an artificial neural network. (Koyuncu, [0205] “An arrangement 830 of the plurality of segments is processed by one or more layers of a neural network. Such an arrangement may be predefined…A first exemplary embodiment may include a reshaping of the latent tensor into a sequential form…”)
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Liu-Howard with Koyuncu to realign, based on data characteristics of input data of the deep learning model, the attention data in the converted data format, as taught by Koyuncu, in order to support larger latent tensors while reducing memory requirements and/or the amount of processing. (Koyuncu, ¶[0021])
Regarding Claims 5 and 15 Liu-Howard-Koyuncu
Liu-Howard-Koyuncu discloses 5. The compression method of claim 1, wherein realigning the attention data (Koyuncu, [0058] “segments, which have different spatial coordinates, are arranged consecutively in a predefined order.) in the converted data format (Liu, page, 43 “After partitioning the tokens into groups, CacheGen uses quantization to reduce the precision of elements (floating points) in a KV cache so that they can be represented by fewer bits.”) comprises:
in response to the input data of the deep learning model being determined as image data, (Koyuncu, [0050] “obtaining a latent tensor by processing the image data with an autoencoding convolutional neural network….”) realigning, (Koyuncu, [0058] “segments, which have different spatial coordinates, are arranged consecutively in a predefined order.) based on spatial information on a relationship between pixels of the image data, the attention data in the converted data format. (Koyuncu, [0052] “…method is provided for entropy decoding of a latent tensor, comprising: initializing the latent tensor with zeroes; separating the latent tensor into a plurality of segments in the spatial dimensions, each segment including at least one latent tensor element; processing an arrangement of the plurality of segments by one or more layers of a neural network, including at least one attention layer; and obtaining a probability model for the entropy decoding of a current element of the latent tensor based on the processed plurality of segments.” [0053] “The method considers spatial correlations in the latent tensor and spatial adaptivity for the implicit entropy estimation…”)
Regarding Claims 7 and 17 Liu-Howard-Koyuncu
Liu-Howard-Koyuncu discloses 7. The compression method of claim 1,
wherein the deep learning model comprises a transformer model (Koyuncu, [0187] “the attention-mechanism is the so-called transformer model....” ) and
acquiring the attention data comprises acquiring output data of any of a multi-head self- attention layer, a multi-head cross attention layer, and a feedforward network of the transformer model. (Koyuncu, [0070] “the at least one attention layer in the neural network is a multi-head attention layer” [00187] “A transformer model applies an attention layer followed by a feed-forward neural network. Two exemplary implementations of a transformer block are shown in Figs. 5a and 5b.”)
Regarding Claims 8 and 18 Liu-Howard-Koyuncu
Liu-Howard-Koyuncu discloses 8. The compression method of claim 1, further comprising:
generating second compressed data by performing entropy encoding on the first compressed data. (Koyuncu, [0235] “The entropy model 770 produces an estimation of the statistical properties of the quantized latent tensor y. The entropy encoder 370 may use these statistical properties to create a bitstream representation 371 of the tensor y.”)
Regarding Claims 9 and 19 Liu-Howard-Koyuncu
Liu-Howard-Koyuncu discloses 9. The compression method of claim 1, further comprising:
storing the first compressed data in a memory. (Koyuncu, [0284] “A video encoding device may encode and store data to memory, and/or a video decoding device may retrieve and decode data from memory.”)
Regarding Claims 10-20 Liu-Howard-Koyuncu
The independent claim 10 recites a limitation that are substantially the similar to those of independent claim 1, except that claim 10 is directed to decoder rather than a encoder. It is well established in the art that video compression systems comprise complementary components, namely encoder (compressor) and a decoder (decompressor), which perform reciprocal operations. The encoder compresses source data to reduce the bit rate foe transmission or storage, while the decoder reconstructs the data from the compressed bitstream by performing a corresponding inverse process.
Regarding Claim 11 Liu-Howard-Koyuncu
Liu-Howard-Koyuncu discloses 11. An electronic device, comprising: at least one processor including processing circuitry (Liu, page 46, 7.1 Hardware Setting “…two Intel(R) Xeon(R) Gold 6130 CPUs.”); and
a memory storing instructions, wherein the instructions, when executed by the at least one processor, (Howard, claim 8. “ a computer-readable storage medium storing a plurality of instructions which, when executed by the processor, cause the processor to perform operations,….”) cause the electronic device to:
The remaining limitations of independent claim 11 recite subject matter substantially similar to that of independent claim 1. Accordingly, the reasoning and analysis provided with respect to claim 1 apply equally to claim 11.
Claims 12-13 and 15-20 recite corresponding subject matter in electronic device form. Accordingly, the analysis set forth above with respect to corresponding method claims 2-3 and 5-10 applies equally to claims 12-13, and 15-20.
Claim Rejections - 35 USC § 103
Claims 2 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Liu-Howard-Koyuncu in view of Song et al (US-20230206025-A1) hereinafter “Song”.
Regarding Claims 2 and 12 Liu-Howard-Koyuncu-Song
Liu-Howard-Koyuncu discloses 2. The compression method of claim 1,
wherein converting the data format of the attention data into the converted data format (Liu, page, 43 “After partitioning the tokens into groups, CacheGen uses quantization to reduce the precision of elements (floating points) in a KV cache so that they can be represented by fewer bits.”)
Liu-Howard-Koyuncu does not discloses
. . . comprises converting the data format of the attention data into a same format as a format predetermined as required by the encoder model.
However, in the same field of endeavor Song discloses more explicitly the following:
comprises converting the data format of the attention data into a same format as a format predetermined as required by the encoder model. (Song teaches converting data into the same format required by the encoder model. Specifically, song teaches converting heterogenous variables into embedding values having an embedding dimension through an embedding layer (¶[0050]); the resulting embedding value E(X.sub.i)∈R.sup.(n+1)×d is used to input to the learning layer (¶[0055]); the transformer block uses that embedding value as its input (¶[0056]); and the transformer block follows the structure of an encoder of an existing transformer model. (¶[0074]);
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Liu-Howard-Koyuncu with Song to convert the data format of the attention data into a same format as a format predetermined as required by the encoder model, as taught by Song in order to enable the transformer block to process embedding values in the format required by the encoder model for efficient attention for efficient attention processing. (Song, ¶ ¶ [0050]-[0051])
Claim Rejections - 35 USC § 103
Claims 3 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Liu-Howard-Koyuncu in view of Tseng et al (US-20250371329-A1) hereinafter “Tseng”.
Regarding Claims 3 and 13 Liu-Howard-Koyuncu-Tseng
Liu-Howard-Koyuncu discloses 3. The compression method of claim 1, wherein converting the data format of the attention data into the converted data format (Liu, page, 43 “After partitioning the tokens into groups, CacheGen uses quantization to reduce the precision of elements (floating points) in a KV cache so that they can be represented by fewer bits.”) comprises . . .
Liu-Howard-Koyuncu does not expressly disclose
converting the data format of the attention data into any of byte units and integer data.
However, in the same field of endeavor Tseng discloses more explicitly the following:
converting the data format of the attention data into any of byte units and integer data. (Tseng, [0027] “…the weights within the multi-head attention operator 10a2 are quantized to an integer 4-bit integer data format (“int4”) … the weights…may be quantized to a low precision (such as, an integer 4-bit integer data format)”.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Liu-Howard-Koyuncu with Tseng to convert the data format of the attention data into any of byte units and integer data in order “to provide optimal balance between model accuracy and computational efficiency (latency) during model inference.” (Tseng, [0029])
Claim Rejections - 35 USC § 103
Claims 6 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Liu-Howard-Koyuncu in view of Mohan et al (US-20250056036-A1) hereinafter “Mohan”.
Regarding Claims 6 and 16 Liu-Howard-Koyuncu-Mohan
Liu-Howard-Koyuncu discloses 6. The compression method of claim 1, wherein realigning the attention data (Koyuncu, [0053] “segments, which have different spatial coordinates, are arranged consecutively in a predefined order.) in the converted data format (Liu, page, 43 “After partitioning the tokens into groups, CacheGen uses quantization to reduce the precision of elements (floating points) in a KV cache so that they can be represented by fewer bits.”) comprises:
in response to the input data of the deep learning model being determined as video data (Koyuncu, Fig, 16 “Video encoder 20” [0002] “…artificial intelligence (Al)-based video or picture compression technologies, and in particular, to context modelling using an attention layer within a neural network to process elements of a latent tensor.”), realigning, (Koyuncu, [0205] “An arrangement 830 of the plurality of segments is processed by one or more layers of a neural network. Such an arrangement may be predefined…A first exemplary embodiment may include a reshaping of the latent tensor into a sequential form…” . . . the attention data in the converted data format. (Liu, page, 43 “After partitioning the tokens into groups, CacheGen uses quantization to reduce the precision of elements (floating points) in a KV cache so that they can be represented by fewer bits.”)
Liu-Howard-Koyuncu does not expressly disclose
based on temporal information on a relationship between frames of the video data,
However, in the same field of endeavor Mohan discloses more explicitly the following:
based on temporal information on a relationship between frames of the video data, (Mohan, [0135] “Long-term dependencies between frames may be considered in video encoding and/or decoding. Learning the dependencies between frames at multiple temporal points may be helpful for encoding smooth and/or complex scene transitions. {circumflex over (x)}.sub.ref may include information (e.g., encoded information) of previous frame(s). RNNs (e.g., such as long short-term memory (LSTMs)) may help overcome the difficulty in encoding information of previous frames (e.g., using {circumflex over (x)}.sub.ref).” [0136]) “encoding of temporal dependencies between frames; incorporating attentions; or modeling attentions (e.g., with spatial attributes)”)
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Liu-Howard-Koyuncu with Mohan to perform the realignment based on temporal information on a relationship between frames of the video data, as taught by Mohan, because learning the dependencies between frames at multiple temporal points is helpful for encoding smooth and/or complex scene transitions, thereby improving the efficiency of video coding. (Mohan, [0136])
Conclusion
Any comments considered necessary by applicant must be submitted no later than the payment of the issue fee and, to avoid processing delays, should preferably accompany the issue fee. Such submissions should be clearly labeled “Comments on Statement of Reasons for Allowance.”
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ASTEWAYE GETTU ZEWEDE whose telephone number is (703)756-1441. The examiner can normally be reached Mo-Fr 8:30 am to 5:30 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, William Vaughn can be reached at (571)272-3922. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ASTEWAYE GETTU ZEWEDE/Examiner, Art Unit 2481 /WILLIAM C VAUGHN JR/Supervisory Patent Examiner, Art Unit 2481
Prosecution Insights