Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103 is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Priority
Acknowledgment is made of applicant's claim for domestic priority based on US provisional application 63565551 filed on 03/15/2024 and non-provisional application 18/641001 filed on 04/19/2024.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
(a) NOVELTY; PRIOR ART.—A person shall be entitled to a patent unless—
(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention; or
(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
PNG
media_image1.png
18
19
media_image1.png
Greyscale
(b) EXCEPTIONS.—
(1) DISCLOSURES MADE 1 YEAR OR LESS BEFORE THE EFFECTIVE FILING DATE OF THE CLAIMED INVENTION.—A disclosure made 1 year or less before the effective filing date of a claimed invention shall not be prior art to the claimed invention under subsection (a)(1) if—
(A) the disclosure was made by the inventor or joint inventor or by another who obtained the subject matter disclosed directly or indirectly from the inventor or a joint inventor; or
(B) the subject matter disclosed had, before such disclosure, been publicly disclosed by the inventor or a joint inventor or another who obtained the subject matter disclosed directly or indirectly from the inventor or a joint inventor.
(2) DISCLOSURES APPEARING IN APPLICATIONS AND PATENTS.—A disclosure shall not be prior art to a claimed invention under subsection (a)(2) if—
(A) the subject matter disclosed was obtained directly or indirectly from the inventor or a joint inventor;
(B) the subject matter disclosed had, before such subject matter was effectively filed under subsection (a)(2), been publicly disclosed by the inventor or a joint inventor or another who obtained the subject matter disclosed directly or indirectly from the inventor or a joint inventor; or
(C) the subject matter disclosed and the claimed invention, not later than the effective filing date of the claimed invention, were owned by the same person or subject to an obligation of assignment to the same person.
Claims 2-4, 7-8, 10-12, 15-16, and 18-20 are rejected under 35 USC 102(a)(2) as being anticipated by Singh et al. (US 2024/0296315 A1).
Regarding Claims 2, 10, and 18, Singh discloses a system (Fig. 12), comprising:
one or more processors (¶114, processing unit 820 comprises processors); and
a non-transitory computer readable storage medium having instructions encoded thereon that, when executed by the one or more processors (¶114, system bus 821 couples system memory 830 to processing unit 820; ¶116, RAM 832 in system memory 830 contains program modules accessible to processing unit 820), cause the one or more processors to perform steps comprising:
receiving a request from an application for AI-based processing (¶32, call API 106 to submit generative AI requests to generative AI models 142-144, determine a priority for the received generative AI requests, and enter the requests in one or more priority queues);
selecting an LLM from a plurality of candidate LLMs based on the LLM having at least a threshold GPU consumption criterion for performing the AI-based processing (¶¶41-42, parse the requests into individual parts to identify one or more generative AI models / type of generative AI model that is being called to service the request (per Fig. 1, Generative Artificial Intelligence (AI) Model (LLM, GPT) API 106; i.e., generative AI models are LLMs); ¶44, to identify the target model for servicing the requests, evaluate tokens in the prompt to determine the likely length of the requested generation, indicate the processing load that the requested generation will place on the target generative AI model, evaluate the current available capacity (¶78, processing resources / capacity for different GPU clusters running the requested generative AI model type) for different generative AI models of the requested generative AI model type and the generative load (per ¶77, amount of time that one or more GPUs may be needed to execute the model) that will be placed on the AI model type if it is chosen as the target generative AI model to service this generative AI request; i.e., each generative AI model has a current GPU processing capacity as threshold GPU consumption criterion to determine which generative AI model can handle the generative load required by the generative AI request; ¶79, based upon the generative load corresponding to this generative AI request and the available capacity identified, identify a target generative AI model or model cluster for serving this generative AI request);
selecting one or more GPU types to be used for deploying the selected LLM (¶20, each type of generative AI model has a dedicated set of GPU resources allocated to it; i.e., select the dedicated set of GPU resources allocated to the identified target generative AI model or model cluster);
allocating one or more GPUs of the selected type from a cloud service provider for deploying the selected LLM (Fig. 11 and ¶108, cloud computer architecture 590; ¶111, Generative AI model layer 104 and layer 107 with AI model execution layer 108 comprising GPU Management System 148 and GPUs 150 of Fig. 1 are located in cloud 592; i.e., dedicated GPU resources allocated to the identified target generative AI model or model cluster are from the cloud);
deploying the selected LLM onto the allocated one or more GPU (in view of ¶27 and ¶45, target model calling system 222 calls the target generative AI model or model cluster in the generative AI model layer 104; per ¶20, scaling a pool of GPU resources among a plurality of different types of generative AI models; ¶33 and Fig. 1, GPU management system to scale the number of (capacity of) GPUs in the production pool 108);
executing the request using the deployed LLM to generate an output (¶40 and ¶80, once the target generative AI model or model cluster has been identified, route the generative AI request to the target generative AI model); and
providing the output to the application (¶40 and ¶45, response processor receives response from the target generative AI model and passes that response back to the requesting entity).
Further regarding claim 10, Singh discloses a non-transitory computer readable storage medium having instructions encoded thereon that, when executed by one or more processors, cause the one or more processors to perform steps of claim 2 and functions of claim 18 (¶116, RAM 832 in system memory 830 contains program modules accessible to processing unit 820).
Regarding Claims 3, 11, and 19, Singh discloses wherein the selecting of the LLM is further based on: applying a classification model to the request to determine a type of the request (¶44 and ¶79, target model identifier 212 identifies a particular target generative AI model or cluster where the request is to be routed; see Fig. 2, Target model identifier 212 comprising token evaluator 214, generative load identifier 216, capacity evaluator 218, and other 220 evaluating the request to identify the target generative AI model or cluster); and selecting the LLM based on the determined type (¶45, target model calling system 222 calls the target generative AI model or model cluster in the generative AI model layer 104).
Regarding Claims 4, 12, and 20, Singh discloses wherein the selecting of the LLM further comprises: comparing performance metrics of the plurality of candidate LLMs based on historical request data (¶46, collect data corresponding to prompt (i.e., generative AI request) and response for processing and storage (i.e., historical prompts and responses); ¶58 and ¶89, model evaluation processor 294 evaluated prompt and responses using evaluation metrics generated by evaluation metric generator 332 to compare the performance of different generative AI models or different types of generative AI models to determine a different generative AI model or a different type of generative AI model may perform better than the currently selected generative AI model; ¶90, Model evaluation processor 294 determine whether other types of generative AI models should be used instead of the model type currently specified); and selecting the LLM based on the comparison (¶45, target model calling system 222 then calls the target generative AI model or model cluster in generative AI model layer 104).
Regarding Claims 7 and 15, Singh discloses wherein allocating the one or more GPUs of the selected type comprises: determining an input token count for the request (¶41, parse the request into individual requests and generate tokens based upon the parsed words in the request; ¶73, parsing system identifies different parts of the prompt such as tokens and parses them (tokens) out so that they can be treated separated; i.e., parse out / count tokens of the prompt); and
determining an output token count for the output generated by the selected LLM (¶44, evaluate the tokens in the prompt to determine the likely length of the requested generation; per ¶77, likely length of the generation measured in terms of the number of tokens in the requested AI generation).
Regarding Claims 8 and 16, Singh discloses receiving feedback from a user or application regarding the output generated by the selected LLM (¶58, model evaluation processor 294 compared the performance of different generative AI models so that developer or user can determine when to switch toa different generative AI model which may perform better than the currently selected generative AI model); and
in response to receiving a subsequent request from the application, selecting a different LLM based on the feedback (¶90, model evaluation processor 294 determines other types of generative AI models should be sued instead of the model type currently specified).
Claim Rejections - 35 USC § 103
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 103 that form the basis for the rejections under this section made in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 5-6, 13-14, and 21 are rejected under 35 USC 103 for being unpatentable over Singh et al. (US 2024/0296315 A1) in view of Xie (CN 112286644 A, see IP.com translation).
Regarding Claims 5, 13, and 21, Singh does not disclose wherein the selecting of the one or more GPU types is based on historical GPU utilization patterns for each of a plurality of candidate GPU types for the selected LLM.
Xie discloses selecting or deploying AI Inference Models (p. 5, “The AI inference is just based on a training model to complete a series of inference calculations of technologies such as image recognition, character recognition, voice recognition, etc., and it can be seen that the calling of an AI inference class service container (hereinafter referred to as "AI inference container") depends on GPU computing power” and “…improve the deployment efficiency of an AI (AI) inference container and improve the utilization rate of the GPU computing power in the related technology”) based on historical GPU utilization patterns for each of a plurality of candidate GPU types for the selected AI inference model (p. 8, “Step S100: and automatically testing the AI inference container to obtain the historical maximum operation index… where the historical maximum operation index includes the maximum delay, the maximum QPS, and the historical maximum GPU usage rate (i.e., maxGPU)… Step S200: and automatically determining the calculation power quota of the container according to the historical maximum operation index…”; step S300, detecting GPU of the container in real time and real-time operation indexes of the container (e.g., real-time GPU usage) to consider whether the allocated computation power quota is sufficient; i.e., considering AI inference model / container with a detected GPU and real-time GPU usage being sufficient or insufficient based on AI inference model / container’s historical maximum GPU usage rate, determine whether to select or deploy the AI inference model / container).
It would’ve been obvious to one ordinarily skilled in the art before the effective filing date of the invention to select the one or more GPU types based on historical GPU utilization patterns for each of the plurality of candidate GPU types for the selected LLM / AI inference model – container to improve the deployment efficiency of the selected LLM / AI inference model – container and improve the utilization rate of the GPU computing power (Xie, p. 5, “Disclosure of Invention”).
Regarding Claims 6 and 14, Singh does not disclose wherein allocating the one or more GPUs comprises of the selected type: dividing a GPU into a plurality of virtual GPUs; and deploying multiple LLMs, including the selected LLM, concurrently on the plurality of virtual GPUs.
Xie teaches allocating the one or more GPUs comprises of the selected type: dividing a GPU into a plurality of virtual GPUs (p. 8, step S300, allocated GPU resources may be resources of one or more complete GPU cards or a virtualized GPU computation power slice such as a computation power slice of 0.1 GPU cards); and deploying multiple LLMs, including the selected LLM, concurrently on the plurality of virtual GPUs (p. 8, when real-time GPU usage is above 90%, automatically increment the number of container instances (i.e., multiple AI inference models such as LLMs of Singh; i.e., consider whether computation power quota allocated based on historical maximum GPU usage rate is sufficient for the real-time GPU usage (when real-time GPU usage is above 90%) to deploy multiple instances of the AI inference models / containers or LLMs of Singh).
It would’ve been obvious to one ordinarily skilled in the art before the effective filing date of the invention to allocating the one or more GPUs comprises of the selected type by dividing a GPU into a plurality of virtual GPUs and deploying multiple LLMs, including the selected LLM, concurrently on the plurality of virtual GPUs in order to improve the deployment efficiency of the selected LLM / AI inference model – container and improve the utilization rate of the GPU computing power (Xie, p. 5, “Disclosure of Invention”).
Claims 9 and 17 are rejected under 35 USC 103 for being unpatentable over Singh et al. (US 2024/0296315 A1) in view of Bhupati et al. (US 2024/0378396 A1).
Regarding Claims 7 and 17, Singh does not disclose determining whether the request should be restructured based on the input token count; responsive to determining that the request should be restructured, reconstructing the request into a new form with a reduced input token count.
Bhupati teaches determining whether a request prompt to a LLM should be restructured based on input token count and responsive to determining that the request should be restructured, reconstructing the request into a new form with a reduced input token count (¶31, given a LLM’s token limit, reducing or optimizing the token count of the prompt; i.e., determine whether token count of the prompt exceeds the LLM’s token limit and optimizing token count of the prompt to improve LLM performance).
It would’ve been obvious to one ordinarily skilled in the art before the effective filing date of the invention to determining whether the request should be restructured based on the input token count, reconstructing the request into a new form with a reduced input token count, and sending the reconstructed request to the selected LLM (Singh, ¶80, route the generative AI request to the target generative AI model) in order to improve the performance of the LLM without overwhelming / exceeding the LLM’s token limit (Bhupati, ¶31).
Conclusion
Prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
CN 106710596 B discloses a server configured with a plurality of GPUs, each GPU is configured with a preset neural network (p. 7 of the IP.com translation).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to examiner Richard Z. Zhu whose telephone number is 571-270-1587 or examiner’s supervisor Hai Phan whose telephone number is 571-272-6338. Examiner Richard Zhu can normally be reached on M-Th, 0730:1700.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RICHARD Z ZHU/Primary Examiner, Art Unit 2654 08/07/2026
