DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims 1-20 remain pending and are ready for examination.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 06/26/2025, was filed. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1:
Independent claim 1 recites a method, independent, and independent claim 10 recites a computing device. Therefore, step 1 is satisfied for claims 1-20.
Step 2A Prong One:
The claim(s) recite(s) mental process steps of:
dividing the input prompt into a plurality of prefill chunks; (this step recite abstract mental processes that can be performed by the human mind or practicably with pen and paper. MPEP § 2106.04(a)(2)(II). The concept of dividing data is a mental process (e.g., observations, evaluations, judgments, and opinions) that is applied and performed in a computing environment—i.e., an abstract idea. See MPEP § 2106.04(a)(2)(I]); see also Elec. Power Grp., 830 F.3d at 1354 (“[A]nalyzing information by steps people go through in their minds, or by mathematical algorithms, without more, [are] essentially mental processes within the abstract-idea category.”’). ).
creating a plurality of hybrid batches, wherein each hybrid batch includes a prefill chunk and at least one decode; (this step recite abstract mental processes that can be performed by the human mind or practicably with pen and paper. MPEP § 2106.04(a)(2)(II). The concept of creating a plurality of hybrid batches is a mental process (e.g., observations, evaluations, judgments, and opinions) that is applied and performed in a computing environment—i.e., an abstract idea. See MPEP § 2106.04(a)(2)(I]); see also Elec. Power Grp., 830 F.3d at 1354 (“[A]nalyzing information by steps people go through in their minds, or by mathematical algorithms, without more, [are] essentially mental processes within the abstract-idea category.”’). ).
Step 2A Prong Two:
The claim/s recites the combination of the additional elements, the additional elements in the claim are:
receiving, at a large language model (LLM), an input prompt for LLM inference; (claims 1 and 10)
providing the plurality of hybrid batches to a processing unit for processing the LLM inference. (claims 1 and 10)
a memory to store data and instructions; (claim 10)
a processor operable to communicate with the memory (claim 10)
the bold elements above are directed to mere insignificant extra-solution activity. See MPEP 2106.04(d)(I) and 2106.05(g). The act of transmitting data based on the abstract idea fails to integrate the judicial exception into a practical application as it does not differ from those actions that have previously been held to be extra-solution activity, such as “presenting offers to potential customers and gathering statistics generated based on the testing about how potential customers responded to the offers; the statistics are then used to calculate an optimized price”, “selecting information, based on types of information and availability of information in a power-grid environment, for collection, analysis and display”, and “requiring a request from a user to view an advertisement and restricting public access.” The judicial exception is not integrated into a practical application because the remaining additional elements amount to nothing more than generic components recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using generic computer components. See MPEP 2106.04(d)(I) and 2106.05(f).
Step 2B:
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above, the additional elements amount to nothing more than mere instructions to apply the exception using generic computer component(s) and insignificant extra-solution activity. These cannot provide an inventive concept, and thus the claims are patent-ineligible.
Claims 2-10 and 11-20 directed to the same abstract idea without significantly more. The claims either recite an additional insignificant extra-solution activity OR recite an additional mental process to evaluate and judge using pen and paper. There are no additional elements recited in these claims that integrates the abstract idea into a practical application or amounts to significantly more than the abstract idea. Therefore, the claims are rejected under the same abstract idea as claim 1 or 10.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Holmes, “DeepSpeed-FastGen: High-throughput Text Generation for LLMs via MII and DeepSpeed-Inference”, 2024 (Hereinafter “Holmes”).
Holmes teaches:
1. A method, comprising:
receiving, at a large language model (LLM), an input prompt for LLM inference (see page 2, section 2, wherein addressing standard LLM text generation workloads which involve processing user-provided text during a prompt processing phase);
dividing the input prompt into a plurality of prefill chunks (see page. 5 section 3.2, wherein the Dynamic SplitFuse strategy which dictates that long prompts are decomposed into much smaller chunks and scheduled across multiple forward passes);
creating a plurality of hybrid batches, wherein each hybrid batch includes a prefill chunk and at least one decode (see page. 5 section 3.2, wherein the Dynamic SplitFuse performs the dynamic composition of fixed-sized batches composed of both generation and prompt token); and
providing the plurality of hybrid batches to a processing unit for processing the LLM inference (see page. 5 section 3.2).
2. The method of claim 1, further comprising: receiving a prefill chunk size; and using the prefill chunk size to divide the input prompt into the plurality of prefill chunks, wherein each prefill chunk is equal to the prefill chunk size (see page 5 section 3.2, wherein DeepSpeed-FastGen utilizes Dynamic SplitFuse to run at a consistent forward size).
3. The method of claim 2, wherein the prefill chunk size is selected for the LLM and the processing unit using an expected prefill to decode ratio and an expected prefill and decode time for an application (see page.4 section 3.1.2, wherein evaluate how a model’s throughput response to changing the number of token in the forward pass).
4. The method of claim 2, wherein the prefill chunk size is determined by selecting the prefill chunk size at a prefill chunk size threshold for a minimum input prompt size where a prefill throughput on the processing unit is constant (see page.4 section 3.1.2, wherein after a certain token threshold, the model becomes bound by compute and sees near-constant throughput).
5. The method of claim 2, wherein the prefill chunk size is selected in response to analyzing a prefill throughput of various chunk sizes for expected workloads using the LLM on the processing unit and the prefill chunk size is provided to the LLM as a configuration parameter (see page.4 section 3.1.2 and page 5 section 3.2, wherein evaluate how a model’s throughput response to changing the number of token in the forward pass).
6. The method of claim 1, further comprising: determining a size for each hybrid batch based on a prefill chunk size and a maximum decode batch size for a number of decodes to include in each hybrid batch, wherein the size for each hybrid batch is uniform (see page. 5 section 3.2, wherein Dynamic SplitFuse relies on the dynamic composition of fixed-sized batches).
7. The method of claim 6, wherein the maximum decode batch size is determined based on available processing unit memory, a parameter requirement for the LLM, and a maximum sequence length supported by the LLM (see page. 2 section 2.1, wherein concurrency in LLMs is historically limited by memory fragmentation caused by large KV-caches).
8. The method of claim 1, wherein the processing unit is one of a central processing unit (CPU), a graphics processing unit (GPU), or an Application Specific Integrated Circuit (ASIC) (see page 6 section 4.1.2,).
9. The method of claim 1, further comprising: Providing the plurality of hybrid batches to a plurality of graphics processing units (GPUs) for processing the LLM inference, where each GPU of the plurality of GPUs processes a portion of the LLM inference (see page 7 and fig. 4, wherein the system scales by distributing workloads across a plurality of GPUs, such as testing the Llama 2 70B model across 4 A100-80GB GPUs).
10. The method of claim 9, further comprising: using pipeline parallelism or tensor parallelism to schedule the plurality of hybrid batches across the plurality of GPUs (see page 7 and fig. 4, wherein using tensor parallelism across 4 A100-80GB GPUs).
Claims 11-20 are rejected under the same rationale as claims 1-10.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MAHER N ALGIBHAH whose telephone number is (571)272-0718. The examiner can normally be reached on Monday-Thursday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Aleksandr Kerzhner can be reached on (571) 270-1760. The fax phone number for the organization where this application or proceeding is assigned is 571-273-1264.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MAHER N ALGIBHAH/Primary Examiner , Art Unit 2165
Prosecution Insights