DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 7/17/2026 has been entered.
Response to Amendment
Claims 1, 10, 11, and 18 are amended. Claims 1-5, 7-16, and 18-20 are presented for examination.
Response to Arguments
Rejection under 35 U.S.C. 101
Applicant’s arguments have been fully considered and are persuasive. The amended independent claims 1 and 11 recites, in part, generating a batched prompt as input to an LLM by configuring the batched prompt as a pre-fill tokenized input including overlaps in context of terms in which one or more self-attention values for tokens associated with each query does not require recalculation for every query within the batch prompt. This is a process that cannot be performed mentally and provides an improvement to the field of LLMs by allowing for LLMs to more efficiently process batched prompts by reducing the amount of calculations needed for queries in a batched prompt.
Rejection under 35 U.S.C. 103
Applicant’s arguments have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 9-14, 16, and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Lin et al. (US 20250077844 A1; hereinafter referred to as Lin) in view of Radmilac et al. (US 20240412040 A1; hereinafter referred to as Radmilac) and Phanishayee et al. (US 12602270 B1; hereinafter referred to as Phanishayee).
Regarding claim 1, Lin discloses: one or more processors having access to one or more memories ([0070] The computer system 500 also includes memory 503 in electronic communication with the processor 501), wherein the one or more processors are configured to... generate the batched prompt from the plurality of LLM queries ([0023] multiple input prompts are included or otherwise combined within a batch prompt. In one or more embodiments described herein, the batch prompt is a combination of a task (or multiple similar tasks) with multiple data inputs) and one or more prompt- engineering inputs… ([0022] an input prompt refers to a task (e.g., a task specification) and an associated data input (e.g., a data specification) including data to be processed or labeled by the LLM in accordance with a context that takes into account a knowledge base, the task, and any other data provided to the LLM in connection with the input query);
provide the one or more prompt-engineering inputs and the batched prompt to an LLM… ([0039] a plurality of batch permutations 134a-c (e.g., including the batch permutations 134a-c and/or the original batch prompt 124 as shown in FIG. 2A) maybe provided as inputs to the LLM 108. The LLM 108 may analyze the batch data set 130a-c of each of the batch permutations 134 with respect to the task 126);
receive an output from the LLM that is based on the batched prompt ([0049] The output generator 118 may consider the outputs for each of the data inputs from across the plurality permutation output sets 140a-c to determine a batched output set 150 (or batch prompt output) including a set of final outputs 146. As mentioned above, the batch output set 150 may be a set of outputs determined to be responsive to the batch prompt 124 and/or responsive to the plurality of input prompts 122a-c used in generating the batch prompt 124);
generate, from the output, a plurality of responses corresponding to the plurality of users ([0042] the LLM 108mayevaluateeachofthe data inputs 128with respect to the task 126 and may determine a corresponding answer, result, response, or other output as the output 144 for that data input 128);
and provide a corresponding response, from the plurality of responses, to each of the plurality of users ([0051] the output generator 118 may generate the batch output set 150 corresponding with the answers, results, responses, or other outputs for the task 126 as applied to each of the input data instances represented within the batch prompt 124, which, as discussed above, may include a final output 146 for each of the input prompts 122a-c that are used in generating the batch prompt 124).
Lin does not explicitly, but Radmilac teaches: identify a plurality of LLM queries from a plurality of users for inclusion as part of a batched prompt… ([0181] a batch corresponds to one or more requests received from a plurality of users);
wherein the one or more prompt-engineering inputs identifies the batched prompt as including queries from a plurality of users… ([0178] Systems also identify at least one batching criterion (e.g., batching criteria 726) for a particular node (e.g., node f5) of the different processing nodes (act 1120). Some examples of batching criteria include waiting to transmit a batch of processing requests until a minimum number of processing requests are received, based on a maximum number of processing requests that a particular node can process, waiting for a predetermined amount of time between receiving batching requests, or other batching criterion. The batching criterion can determine that a batched prompt has multiple processing requests, and therefore multiple user queries.);
Lin and Radmilac are considered analogous in the field of large language models. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Lin to combine the teachings of Radmilac because doing so would allow for multiple users to provide queries in order to create a batch prompt for input to an LLM, which would improve user experience and LLM efficiency (Radmilac [0140] By implementing systems according to such embodiments, the user experience is improved by allowing a user to submit data prompts and retrieve moderated outputs in a streamlined manner since all of the underlying models and data dependencies between models are abstracted away from the user interface).
The combination of Lin and Radmilac does not explicitly, but Phanishayee teaches: wherein the batched prompt is configured as a pre-fill tokenized input ([col 6, lines 46-56] Prompt pipelines are compute dominated and typically have small batch sizes (e.g., number of prompts). Each prompt pipeline has one or more pipeline stages which correspond to different layers in the generative transformer model. Each prompt pipeline maintains a prompt compute cache (e.g., KV cache) which receives the computed prompt values as the prompt is processed through the prompt pipeline. Once a prompt has been processed through the last stage of the pipeline, cache streaming is used to stream the prompt cache to the token cache for a token processing pipeline. Generating a KV cache from an input prompt indicates a prefill phase) with overlaps in context of terms ([col 4, lines 40-45] Once the query vector, key vector, and value vector have been determined, these vectors are used to calculate “attention scores” which represent the relevance of each word in an input sequence. More specifically, the attention scores indicate the similarity between the query and each key in the sequence, resulting in a weight for each token) in which one or more self-attention values for tokens associated with each query ([col 4, lines 28-36] Generative transformer models may use a caching mechanism to store computed values which are generated at various layers and/or stages of the transformer model that can be used in subsequent steps. For example, the transformer has a self-attention mechanism which involves calculating three vectors: a query vector, a key vector, and a value vector by multiplying the input embedding with weight matrices that are learned during training) does not require individual recalculation for every query within the batch prompt ([col 4, lines 49-57] The output (i.e., the token having the highest probability) of the current token prediction iteration is used as the query for the next iteration. The keys and values are derived from the input prompt embedding and embeddings of previous token predictions. The keys and values can therefore be cached, referred to as KV caching, and provided to the transformer during the next iteration so that the keys and values for preceding tokens do not have to be recomputed).
Lin, Radmilac, and Phanishayee are considered analogous in the field of large language models. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Lin and Radmilac to combine the teachings of Phanishayee because doing so would allow for an LLM model to incorporate a prefill phase including KV caching, leading to more efficient and quicker batched prompt processing using caching and self-attention mechanisms (Phanishayee [col 4-5, lines 58-3] Without KV caching, the latency per token (i.e., processing time per token) would increase as the number of tokens that need to be processed per iteration increases. This is due to the fact that for each iteration the keys and values of all preceding tokens must be computed. KV caching enables the keys and values of each iteration to be saved and reused in the next iteration which significantly reduces the computational requirements for the transformer. With KV caching the processing time per token remains almost constant, which is a significant reduction in latency relative to the case where KV caching is not used. The use of KV caching enables generative model serving to be stateful and distributed).
Regarding claim 2, the combination of Lin, Radmilac, and Phanishayee teaches: the system of claim 1. Lin further teaches: wherein the one or more processors are further configured to determine that the plurality of LLM queries share one or more batching parameters ([0033] The batch prompt generation system 110 may group the input prompts 122a-n within respective groupings. For example, the batch manager112 may receive a plurality of input prompts 122a-n and categorize them based on the tasks 126a-n that are included. For instance, the batch manager 112 may categorize the input prompts 122a-n within a first grouping according to a (e.g., same or similar) first task and a second grouping according to a (e.g., identical or similar) second task. The batch manager 112maygrouptheinputprompts 122a-n in accordance with any of a variety of criteria or characteristics of the input prompts 122a-n).
Regarding claim 3, the combination of Lin, Radmilac, and Phanishayee teaches: the system of claim 2. Radmilac further teaches: wherein the one or more batching parameters comprise at least one of a temporal parameter, a spatial parameter, and a subject-matter parameter ([0178] Some examples of batching criteria include waiting to transmit a batch of processing requests until a minimum number of processing requests are received, based on a maximum number of processing requests that a particular node can process, waiting for a predetermined amount of time between receiving batching requests, or other batching criterion).
Lin, Radmilac, and Phanishayee are considered analogous in the field of large language models. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Lin, Radmilac, and Phanishayee to further combine the teachings of Radmilac because doing so would improve the processing of batch prompts by utilizing batching parameters to increase efficiency and user experience (Radmilac [0124-0125] system identifies one or more batching criteria associated with node f5. For example, as shown in FIG. 20, batching criteria 726 includes a node maximum 728, time elapsed since last request 730, or other 732 batching criteria… Larger batches are hardware-efficient but result in increased latency. Thus, setting a maximum batch size ensures that batches are not so large as to increase the latency to an unacceptable level such that it degrades the user experience, particularly in streaming applications).
Regarding claim 4, the combination of Lin, Radmilac, and Phanishayee teaches: the system of claim 3. Radmilac further teaches: wherein the temporal parameter comprises the plurality of LLM queries ([0121] each model graph instance is generated fora different data processing request from one or more different users or in response to receiving different data prompts) having timestamps that are within a predefined amount of time from one another ([0182] the batching criteria are based on a minimum or maximum number of data processing requests and/or based on a maximum wait time between received data processing requests. In some instances, the maximum wait time is less than a millisecond, or less than a few milliseconds, depending on the model processing specifications).
Regarding claim 9, the combination of Lin, Radmilac, and Phanishayee teaches: the system of claim 1. Radmilac further teaches: wherein the one or more processors are configured to identify the plurality of LLM queries to be combined based on a determination that an LLM- query threshold has been reached ([0125] The node maximum 728 indicates the maximum number of data processing requests that a node can process as a single input. This is the threshold maximum queue size of requests for the batch. In particular, the node maximum 728 dictates a maximum number of data processing requests that can be included in a batch queue or cache assigned to a particular node before the enqueued requests are dispatched to the node for processing).
Regarding claim 10, the combination of Lin, Radmilac, and Phanishayee teaches: the system of claim 1. Phanishayee further teaches: wherein the one or more processors are further configured to provide the batched prompt as pre-fill tokenized input… ([col 6, lines 46-56] wherein the one or more processors are further configured to provide the batched prompt as pre-fill tokenized input).
Lin further teaches: and wherein the pre-fill tokenized input identifies overlap between one or more terms within the plurality of LLM queries ([0033] the batch manager 112 may categorize the input prompts 122a-n within a first grouping according to a (e.g., same or similar) first task and a second grouping according to a (e.g., identical or similar) second task. The batch manager 112 may group the input prompts 122a-n in accordance with any of a variety of criteria or characteristics of the input prompts 122a-n).
Regarding claim 11, it recites similar limitations as claim 1 and therefore is rejected similarly.
Regarding claim 12, it recites similar limitations as claim 2 and therefore is rejected similarly.
Regarding claim 13, it recites similar limitations as claim 3 and therefore is rejected similarly.
Regarding claim 14, it recites similar limitations as claim 4 and therefore is rejected similarly.
Regarding claim 16, the combination of Lin, Radmilac, and Phanishayee teaches: the method of claim 11. Radmilac further teaches: wherein the one or more prompt- engineering inputs identifies the batched prompt as including the plurality of LLM queries ([0178] Systems also identify at least one batching criterion (e.g., batching criteria 726) for a particular node (e.g., node f5) of the different processing nodes (act 1120). Some examples of batching criteria include waiting to transmit a batch of processing requests until a minimum number of processing requests are received, based on a maximum number of processing requests that a particular node can process, waiting for a predetermined amount of time between receiving batching requests, or other batching criterion. The batching criterion can determine that a batched prompt has multiple processing requests, and therefore multiple user queries.).
Regarding claim 19, it recites similar limitations as claim 9 and therefore is rejected similarly.
Regarding claim 20, it recites similar limitations as claim 10 and therefore is rejected similarly.
Claims 5 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Lin in view of Radmilac and Phanishayee, as applied to claims 1-4, 9-14, 16, and 19-20 above, and further in view of Lafon et al. (US 20250068741 A1; hereinafter referred to as Lafon).
Regarding claim 5, the combination of Lin, Radmilac, and Phanishayee teaches: the system of claim 3. The combination of Lin, Radmilac, and Phanishayee does not explicitly, but Lafon teaches: wherein the spatial parameter comprises the plurality of LLM queries ([0017] Prompts or user inputs or user queries which are intended to misguide generative Al models in the different ways described herein are usually made up of base prompt) having location metadata within a predefined region ([0039] the geographical location can be identified from the metadata of the user query 122).
Lin, Radmilac, Phanishayee, and Lafon are considered analogous in the field of large language models. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Lin, Radmilac, and Phanishayee to combine the teachings of Lafon because doing so would allow for output of an LLM to be modified based on the location associated with a user query, leading to more relevant responses for users (Lafon [0039] At 710, the content regulations for the geographical location are accessed from the regulator knowledgebase 188. At 712 it is determined if the model query response complies with the content regulation of the geographical location).
Regarding claim 15, it recites similar limitations as claim 5 and therefore is rejected similarly.
Claims 7-8 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Lin in view of Radmilac and Phanishayee, as applied to claims 1-4, 9-14, 16, and 19-20 above, and further in view of Bharadwaj et al. (US 20240273345 A1; hereinafter referred to as Bharadwaj).
Regarding claim 7, the combination of Lin, Radmilac, and Phanishayee teaches: the system of claim 1. The combination of Lin, Radmilac, and Phanishayee does not explicitly, but Bharadwaj teaches: wherein the one or more prompt engineering inputs comprise one or more segmentation-related instructions for the LLM to structure the output in a manner that allows for segmentation of the output in connection with each of the plurality of users ([0129] with regard to response module selection and/or construction the one or more responses to the first input query generated using the selected or constructed response modules may be tailored to a user segment, for instance, by using prompting optimized for that user segment, adapter models finetuned for generating outputs for that user segment, and so on).
Lin, Radmilac, Phanishayee, and Bharadwaj are considered analogous in the field of large language models. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Lin, Radmilac, and Phanishayee to combine the teachings of Bharadwaj because doing so would allow for specific outputs from the LLM to be directed to particular users, improving user experience (Bharadwaj [0198] identifying one or more population groups/user segments to serve the selected or constructed response modules at step 906, the method may proceed to step 908, wherein step 908 comprises serving the response modules to the identified population groups/user segments. After serving the response modules to the identified population groups/user segments at step 908, the method 900 may proceed to step 910, wherein step 910 comprises collecting metrics associated with the served modules).
Regarding claim 8, the combination of Lin, Radmilac, Phanishayee, and Bharadwaj teaches: the system of claim 7. Bharadwaj further teaches: wherein the segmentation instructions ([0013] the plurality of machine learning models in each respective response module comprises each of a foundational language model) comprise instructions allow for portions of the output to be associated with one or more user identifiers from the plurality of users ([0098] one or more response modules maybe selected or constructed based on the input query characteristics, including the prompt, user/user identifier, data associated with the user/user identifier, and user preference metrics associated with the response modules. The selected or constructed one or more modules maybe used to generate one or more responses to the input query).
Regarding claim 18, it recites similar limitations as claim 8 and therefore is rejected similarly.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Nathan Tengbumroong whose telephone number is (703)756-1725. The examiner can normally be reached Monday - Friday, 11:30 am - 8:00 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hai Phan can be reached at 571-272-6338. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NATHAN TENGBUMROONG/Examiner, Art Unit 2654
/HAI PHAN/Supervisory Patent Examiner, Art Unit 2654
