DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Preliminary Amendment
The preliminary amendment filed 2/11/2025 has been entered. Claims 1-21 have been canceled. Claims 22-42 have been added.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 22-42 are rejected under 35 U.S.C. 103 as being unpatentable over Moritz et al. (US 2022/0310070 A1), hereinafter “Moritz”, and in view of Yu et al. (US 2024/0403722 A1), hereinafter “Yu”.
As per claims 22, 41-42, Moritz teaches a method for processing streaming data comprising:
“obtaining input data representative of a frame of streaming data” at [0006], [0044]-[0045];
(Moritz teaches obtaining input sequence correspond to sequence of frames extracted from a speech utterance that contains a sequence of speech sound events)
“identifying a query transformation, a key transformation, and a value transformation based on the input data for the frame of streaming data” at [0006];
(Moritz teaches transforming the frames of such an input sequence to a sequence of key frames, values frames and query frames)
“determining a dot product of the retrieved one or more query frames and frames in the key buffer to determine a set of weights” at [0060], [0064]-[0065];
(Moritz teaches inputs of the softmax function is the dot-product score between query 302 and the keys 304A-304D, which are utilized to determine the attention scores. Each of the corresponding values 306A, 306B, 306C and 306D are weighted according to the normalized attention scores, i.e., each of the values 306A, 306B, 306C, and 306D is multiplied with the normalized attention scores)
“determining a weighted sum between the set of weights and frames in the value buffer” at [0060], [0064]-[0065];
(Moritz teaches the weighted values 306A-306D are summed up)
“utilizing the weighted sum to generate a streaming attention vector, wherein the streaming attention vector is usable by a network to generate a prediction associated with the streaming data” at [0060], [0064]-[0065].
(Moritz teaches the dilated self-attention module 208 determines an output vector, such as an attention value 310 based on a summation of the weighted values 306A-306D)
Moritz does not explicitly teach “updating a query buffer, a key buffer, and a value buffer based on the identified query, key, and value transformations, such that the query buffer, the key buffer, and the value buffer are each configured to store corresponding query, key and value transformations associated with previous frames of streaming data and the frame of streaming data; retrieving one or more query frames from the updated query buffer to be used to process the input data” as claimed.
However, Yu teaches a method for caching query, key and value data of a transformer model including the steps of:
“updating a query buffer, a key buffer, and a value buffer based on the identified query, key, and value transformations, such that the query buffer, the key buffer, and the value buffer are each configured to store corresponding query, key and value transformations associated with previous frames of streaming data and the frame of streaming data” at [0042] and Fig. 2A;
(Yu teaches receiving the output tensor from the QKV operation block and splitting the output tensor into a query tensor, a key tensor, a value tensor for the current iteration, maintaining a key cache tensor for caching the keys generated at previous iterations and the current iteration, and a value cache tensor for caching the values that were generated at previous iterations and the current iterations. The inference system adds the key tensor for the current iteration to the key cache tensor and the value tensor for the current iteration to the value cache tensor)
“retrieving one or more query frames from the updated query buffer to be used to process the input data” at [0043] and Fig. 2A.
(Yu teaches the self-attention block is coupled to receive the query tensor, the key cache tensor, and the value cache tensor as the input tensors, and generate an output tensor including attention outputs for requests in the batch)
Thus, it would have been obvious to one of ordinary skill in the art to combine Yu with Moritz by using a cache to store query, key and value data generated at a previous iterations and current iteration, allowing access to previous iteration data in order to generate attention output tensor for requests as a batch operation, as suggested by Yu at [0043].
As per claim 23, Moritz and Yu teach the method of claim 22 discussed above. Moritz also teaches: wherein “the network is a transformer network” at [0067].
As per claim 24, Moritz and Yu teach the method of claim 22 discussed above. Moritz also teaches: wherein “the streaming data is streaming audio data” at [0044], [0112].
As per claim 25, Moritz and Yu teach the method of claim 22 discussed above. Moritz also teaches: wherein “the prediction associated with the streaming data comprises a prediction of speech emotion associated with the streaming data” at [0110]-[0116].
As per claim 26, Moritz and Yu teach the method of claim 22 discussed above. Moritz also teaches: wherein “the prediction associated with the streaming data comprises identification of one or more feature useful for provision to one or more downstream machine learning models” at [0110]-[0116].
As per claim 27, Moritz and Yu teach the method of claim 26 discussed above. Moritz also teaches: wherein “the one or more features comprise identification of one or more speakers associated with the streaming data” at [0110]-[0116].
As per claim 28, Moritz and Yu teach the method of claim 22 discussed above. Moritz also teaches: wherein “the prediction associated with the streaming data comprises classification of one or more words or phonemes of the streaming data” at [0110]-[0116].
As per claim 29, Moritz and Yu teach the method of claim 22 discussed above. Yu also teaches: wherein “at least one of the query buffer, the key buffer, or the value buffer is a circular buffer” at [0042]-[0043], [0091].
As per claim 30, Moritz and Yu teach the method of claim 22 discussed above. Yu also teaches: wherein “updating the query buffer comprises: appending a current query frame based on the query transformation to the query buffer; and discarding an oldest query frame in the query buffer” at [0042]-[0043], [0091].
As per claim 31, Moritz and Yu teach the method of claim 30 discussed above. Yu also teaches: wherein “the retrieved one or more query frames correspond to the discarded oldest query frame” at [0042]-[0043], [0091].
As per claim 32, Moritz and Yu teach the method of claim 22 discussed above. Yu also teaches: wherein “the network comprises a plurality of layers, and wherein updating the query buffer comprises, for a first layer of the plurality of layers: appending a current query frame based on the query transformation to the query buffer; replacing a plurality of query frames in the query buffer with a plurality of lookahead query frames corresponding to future times; and discarding an oldest query frame in the query buffer” at [0042]-[0043], [0091].
As per claim 33, Moritz and Yu teach the method of claim 32 discussed above. Yu also teaches: wherein “the retrieved one or more query frames used to process the input block by the first layer of the plurality of layers comprise the current query frame and the plurality of lookahead query frames” at [0042]-[0043], [0091].
As per claim 34, Moritz and Yu teach the method of claim 32 discussed above. Yu also teaches: wherein “the plurality of lookahead query frames comprises two lookahead frames” at [0042]-[0048].
As per claim 35, Moritz and Yu teach the method of claim 32 discussed above. Yu also teaches: wherein “the retrieved one or more query frames used to process the input block by each of the plurality of layers other than the first layer is passed to a given layer by a preceding layer” at [0042]-[0048].
As per claim 36, Moritz and Yu teach the method of claim 32 discussed above. Yu also teaches: wherein “updating the key buffer and updating the value buffer comprise: appending a current key frame based on the key transformation to the key buffer and discarding an oldest key frame from the key buffer; appending a current value frame based on the value transformation to the value buffer; and discarding an oldest value frame from the value buffer” at [0042]-[0048], [0091].
As per claim 37, Moritz and Yu teach the method of claim 32 discussed above. Yu also teaches: wherein “updating the key buffer and updating the value buffer comprise: appending a current key frame to the key buffer; replacing a plurality of key frames in the key buffer with a plurality of lookahead key frames based on the key transformation; discarding an oldest key frame from the key buffer; appending a current value frame to the value buffer; replacing a plurality of value frames in the value buffer with a plurality of lookahead value frames based on the value transformation; and discarding an oldest value frame from the value buffer” at [0042]-[0048], [0091].
As per claim 38, Moritz and Yu teach the method of claim 32 discussed above. Yu also teaches: wherein “the network was trained by: performing an initial training using a first version of the network that does not utilize the query buffer, the key buffer, and the value buffer to store a subset of query frames, key frames, and value frames; and performing a subsequent training that modifies weights associated with the network” at [0042]-[0048], [0091].
As per claim 39, Moritz and Yu teach the method of claim 38 discussed above. Yu also teaches: wherein “the subsequent training is performed using a second version of the network that includes the query buffer, the key buffer, and the value buffer, and wherein performing the subsequent training comprises performing backpropagation using derivatives derived from the second version of the network” at [0031]-[0048] and Figs. 3-5.
As per claim 40, Moritz and Yu teach the method of claim 38 discussed above. Moritz also teaches: wherein “the subsequent training is performed using the first version of the network, and wherein performing the subsequent training comprises: providing, data given block of training data, a series of time shifted segments to the first version of the network to generate a corresponding series of predicted outputs; aggregating the series of predicted outputs; determining a loss based on the aggregated series of predicted outputs; and updating weights associated with the first version of the network based on the loss” at [0063]-[0076].
Conclusion
Examiner's Note: Examiner has cited particular columns and line numbers in the references applied to the claims above for the convenience of the applicant. Although the specified citations are representative of the teachings of the art and are applied to specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested from the applicant in preparing responses, to fully consider the references in entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the Examiner.
In the case of amending the Claimed invention, Applicant is respectfully requested to indicate the portion(s) of the specification which dictate(s) the structure relied on for proper interpretation and also to verify and ascertain the metes and bounds of the claimed invention.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KHANH B PHAM whose telephone number is (571)272-4116. The examiner can normally be reached Monday - Friday, 8am to 4pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sanjiv Shah can be reached at (571)272-4098. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KHANH B PHAM/Primary Examiner, Art Unit 2166
January 27, 2026