Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-5, 7, 10-13, 15-19 are rejected under 35 U.S.C. 103 as being unpatentable over Shen et al. (Shen), Non-Patent Literature, “HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face”, Published in December 2023, arXiv, 27Pages, in view of Groh et al. (Groh), Non-Patent Literature, “Non-Patent Literature, “GGNN: Graph-Based GPU Nearest Neighbor Search”, Published in January 2023, IEEE, 13Pages, in further view of Zhang et al. (Zhang), Non-Patent Literature, “A Survey on Learning to Reject”, Published in February 2023, IEEE, 31Pages.
As to independent Claim 1,
Shen teaches the input query comprising a request for a recommended model and a recommended prompt template for use with the recommended model (Shen, Pg2, First and Second Bullets, "Task Planning: Using ChatGPT to analyze the requests of users to understand their intention, and disassemble them into possible solvable tasks",
"Model Selection: To solve the planned tasks, ChatGPT selects expert models that are hosted on Hugging Face based on model descriptions",
Pg1, Abstract, Lines8-10, “we present HuggingGPT, an LLM-powered agent that leverages LLMs (e.g., ChatGPT) to connect various AI models in machine learning communities (e.g., Hugging Face) to solve AI tasks”,
Pg5, Table1,
PNG
media_image1.png
947
724
media_image1.png
Greyscale
, wherein Shen discloses that chatGPT takes in the requests or queries of users, which then the system or HuggingGPT architecture analyzes to find best AI model to perform the task along with the prompt template to be feed into that selected model(in the Table1, Model Selection part), which is functionally equivalent to the claimed invention.)
Shen further teaches constructing a response to the input query, the response comprising a model-prompt template pair selected (Shen, Pg7, Section3.4, Response Generation, Lines1-4, "After all task executions are completed, HuggingGPT needs to generate the final responses. As shown in Table 1, HuggingGPT integrates all the information from the previous three stages (task planning, model selection, and task execution) into a concise summary in this stage, including the list of planned tasks, the selected models for the tasks, and the inference results of the models"
Pg5, Table1 above, wherein Shen discloses that the response integrates all the information used in the three stages and Model Selection stage includes selecting the best model and its prompt, rendering it functionally equivalent to the claimed invention.)
However, Shen does not teach
computing, using a trained embedding model, a query embedding representing an input query
selecting a set of nearest neighbor embeddings with a distance less than a threshold distance from the query embedding in an embedding space;
predicting, using a trained confidence predictor model, a confidence value of each model-prompt template pair represented by a nearest neighbor embedding in the set of nearest neighbor embeddings, the predicting resulting in a set of predicted confidence values;
using the predicted confidence values.
From the same field of endeavor, Groh teaches computing, using a trained embedding model, a query embedding representing an input query (Groh, Pg270, Section 4.1, Lines2-4, "for a query q in Rd, the simple greedy downhill search with backtracking from Algorithm1 is executed",
Pg270, Right Column, Paragraph4, Lines5-8, "The distance to the query point is computed in parallel (dist). Every thread reads its respective vector element(s) from memory and computes the element-wise result",
Pg267, Right Column, Lines1-8, "for newly embedded vectors of neural networks similar to DEEP1B... feature matching in embeddings", wherein Groh discloses that the vectors or the queries are embedded using neural network (the corresponding trained embedding model) as they are written as q in Rd, meaning they are already embedded queries using a black-box model, rendering it functionally equivalent to the claimed invention of computing query embeddings using a trained embedder.)
Groh further teaches selecting a set of nearest neighbor embeddings with a distance less than a threshold distance from the query embedding in an embedding space (Groh, Pg270, Section 4.1, Lines2-4, "for a query q in Rd, the simple greedy downhill search with backtracking from Algorithm1 is executed",
Pg269, Section3.1, Lines3-4, "For the sake of simplicity, here, we assume an Euclidean space
PNG
media_image2.png
26
159
media_image2.png
Greyscale
and Euclidean distances
PNG
media_image3.png
27
65
media_image3.png
Greyscale
",
Pg270, Right Column, Paragraph4, Lines5-8, "The distance to the query point is computed in parallel (dist). Every thread reads its respective vector element(s) from memory and computes the element-wise result",
Pg270, Section4.1, Paragraph2, Lines1-6, "Until there are no more unexplored points in the priority queue, all neighbors N_a of the closest not yet visited point a are being fetched to the cache... Our cache structure manages a sorted list of the k closest points (best) to the query which are observed during the traversal",
Pg271, Equation2,
PNG
media_image4.png
58
343
media_image4.png
Greyscale
, wherein Groh teaches about computing Euclidean distances between the queries q to their k nearest neighbors which the searching for its nearest neighbors will be terminated by the condition of Equation 2, where the search terminates once the distance falls below the slack factor r (the corresponding threshold distance), rendering it functionally equivalent to the claimed invention.)
Shen and Groh are analogous to the claimed invention as they are from the same field of endeavor of artificial intelligence systems utilizing deep learning neural network embeddings and automated multi-modal model execution. Therefore, it would have been obvious to one of ordinary skills in the art, before the effective filing date, to combine LLM-powered task planning and model orchestration of Shen with GPU-accelerated graph-based approximate nearest neighbor (ANN) search of Groh. The motivation is as recited by Groh (Groh, Pg267, Abstract, Lines1-2, "Approximate nearest neighbor (ANN) search in high dimensions is an integral part of several computer vision systems and gains importance in deep learning with explicit memory representations") such that high-dimensional embedding vectors or memory representations accumulated while Shen processes diverse models and multimodal data can be rapidly searched and indexed on-the-fly using Groh's ultra-fast GPU graph search, thereby significantly enhancing overall data retrieval speed and processing efficiency across the entire LLM agent system.
The combination of Shen and Groh teaches the embeddings representing the pairs of model-prompt templates and their k nearest neighbors that have similar traits. However, the combination fails to teach predicting, using a trained confidence predictor model, a confidence value of each model-prompt template pair represented by a nearest neighbor embedding in the set of nearest neighbor embeddings, the predicting resulting in a set of predicted confidence values.
From the same field of endeavor, Zhang teaches this limitation (Zhang, Pg189, Section 4), Lines1-3, "Distance-Based Confidence: Distances between examples can be used effectively to define confidence due to their close relationship to similarity"
Pg189, Right Column, Lines4-7, "Mandelbaum and Weinshall define the confidence as a local density based on the Euclidean distance in the penultimate layer between the test point and its k nearest neighbors"
Pg197, Right Column, Section 3), Paragraph2, Lines8-12, "an auxiliary model (named ConfidNet) [40] is used to learn TCP(true class probability) on the training set. ConfidNet is built on top of the feature representation of the base model, by using some additional dense layers and a sigmoid activation to regress TCP under a l2 loss function"
Pg197, Right Column, Section 3), Paragraph3, Lines8-9, "Therefore, an auxiliary confidence model is trained to estimate this distance on the training set", wherein Zhang explicitly discloses computing the confidence values by computing the distances to the k nearest neighbors using an auxiliary model (the corresponding trained confidence predictor model), thus using the nearest neighbors found by Shen and Groh to compute the confidence value renders it functionally equivalent to the claimed invention.)
Zhang further teaches using the predicted confidence values (Zhang, Pg186, Right Column, Paragraph2, Lines7-9, "If p_i (the confidence function) is smaller than some threshold, it should be unreliable and then rejected; otherwise, it is accepted as a confident prediction", wherein Zhang explicitly discloses only embedded queries above the threshold will be selected, rendering it functionally equivalent to the claimed invention.)
Shen, Groh and Zhang are analogous to the claimed invention as they are from the same field of endeavor of artificial intelligence systems utilizing deep learning neural network embeddings, automated model orchestration, and reliable decision-making. Therefore, it would have been obvious to one of ordinary skills in the art, before the effective filing date, to combine LLM-powered task planning and model orchestration of Shen, GPU-accelerated graph-based approximate nearest neighbor (ANN) search of Groh with confidence calibration and rejection mechanism of Zhang. The motivation as recited by Zhang (Zhang, Pg185, Introduction, Lines3-6, "in real-world environments, intelligent systems must not only be accurate but also need to be aware of when they are likely to be incorrect and then refuse to make a decision") such that during Shen's autonomous task planning and execution, candidate model routes or vector search matches retrieved via Groh's high-dimensional index can be evaluated for prediction uncertainty on-the-fly and rejected if uncalibrated or out-of-distribution, thereby preventing overconfident execution errors and ensuring safe, robust, and reliable decision-making across the multi-modal agent architecture.
As to dependent Claim 2,
The combination of Shen, Groh and Zhang teaches, as mentioned above, all the limitations of Claim 1. It teaches about the overall architecture of creating responses comprising recommended AI model and its prompt template in response to the user’s input query, which is embedded into the embedding space to find nearest neighbors that shares the similar traits so the confidence values can be computed to choose the most suitable model-prompt pair.
Shen, as mentioned above and in Claim 1, teaches a model-prompt template pair. However, Shen does not teach the computer-implemented method of claim 1, further comprising:
training a confidence predictor model to predict a confidence.
From the same field of endeavor, Zhang teaches this limitation (Zhang, Pg186, Figure1, Line1-2, "confidence: further learn a score function g for measuring the probabilistic correctness of the prediction"
Pg186, Right Column, Paragraph2, Lines 4-6, "As shown in the bottom of Fig. 1(a), the objective is to further learn a confidence function p_i = g(f, xi, yi) to measure the possibility of the prediction being correct"
Pg197, Right Column, Section 3), Paragraph2, Lines8-12, "an auxiliary model (named ConfidNet) [40] is used to learn TCP on the training set. ConfidNet is built on top of the feature representation of the base model, by using some additional dense layers and a sigmoid activation to regress TCP under a l2 loss function"
Pg197, Right Column, Section 3), Paragraph3, Lines8-9, "Therefore, an auxiliary confidence model is trained to estimate this distance on the training set", wherein Zhang explicitly discloses the auxiliary model is trained with the training set to make a correct prediction or confidence, rendering it functionally equivalent to the claimed invention if the training set is Shen’s model-prompt template pairs.)
Shen and Zhang are analogous to the claimed invention as they are from the same field of endeavor of artificial intelligence decision-making and automated machine learning model selection frameworks. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the multi-model and prompt selection pipeline for task execution of Shen with the teaching of training a separate auxiliary confidence predictor model to estimate prediction reliability of Zhang. The motivation as recited by Zhang (Zhang, Pg186, Right Column, Paragraph2, Lines 1-4, "However, no matter how accurate the model is, prediction errors are still inevitable; therefore, we need another function to measure the probabilistic correctness of the prediction") such that the integration of the two would quantitatively evaluate the predicted reliability of each candidate model-prompt template pair prior to task execution. Thereby avoiding costly execution failures and improving overall system task success rates.
As to dependent Claim 3,
The combination of Shen, Groh and Zhang teaches, as mentioned above, all the limitations of Claim 1. It teaches about the overall architecture of creating responses comprising recommended AI model and its prompt template in response to the user’s input query, which is embedded into the embedding space to find nearest neighbors that shares the similar traits so the confidence values can be computed to choose the most suitable model-prompt pair.
Shen further teaches the computer-implemented method of claim 1, wherein the input query comprises an example of a task the recommended model and the recommended prompt template are being selected to perform (Shen, Pg6, Demonstration-based Parsing, Lines1-3, "To better understand the intention and criteria for task planning, HuggingGPT incorporates multiple demonstrations in the prompt. Each demonstration consists of a user request and its corresponding output, which represents the expected sequence of parsed tasks"
Pg1, Abstract, Lines8-10, “we present HuggingGPT, an LLM-powered agent that leverages LLMs (e.g., ChatGPT) to connect various AI models in machine learning communities (e.g., Hugging Face) to solve AI tasks”,
Pg8, Lines1-3, "In this case, the user’s request included three tasks: detecting the pose of a person in an example image, generating a new image based on that pose and specified text, and creating a speech describing the image", wherein Shen explicitly discloses that the input query comprises tasks to be performed by the selected AI model and its prompt, rendering it functionally equivalent to the claimed invention.)
As to dependent Claim 4,
The combination of Shen, Groh and Zhang teaches, as mentioned above, all the limitations of Claim 1. It teaches about the overall architecture of creating responses comprising recommended AI model and its prompt template in response to the user’s input query, which is embedded into the embedding space to find nearest neighbors that shares the similar traits so the confidence values can be computed to choose the most suitable model-prompt pair.
Shen further teaches the computer-implemented method of claim 1, wherein the input query comprises a query requesting a response from the recommended model using the recommended prompt template (Shen, Pg4, Section3, Lines3-5, "Given a user request, our HuggingGPT, which adopts an LLM as the controller, will automatically deploy the whole workflow, thereby coordinating and executing the expert models to fulfill the target"
Pg7, Section3.4, Lines1-10, "After all task executions are completed, HuggingGPT needs to generate the final responses...Moreover, instead of simply aggregating the results, LLM generates responses that actively respond to user requests...", wherein Shen explicitly discloses the architecture of intaking user's queries or tasks to perform, which then generates the actual responses to the queries using the selected model and the prompt, rendering it functionally equivalent to the claimed invention.)
As to dependent Claim 5,
The combination of Shen, Groh and Zhang teaches, as mentioned above, all the limitations of Claim 1. It teaches about the overall architecture of creating responses comprising recommended AI model and its prompt template in response to the user’s input query, which is embedded into the embedding space to find nearest neighbors that shares the similar traits so the confidence values can be computed to choose the most suitable model-prompt pair.
As mentioned above, each model-prompt pair has a confidence value and Zhang further teaches the computer-implemented method of claim 1, wherein the model-prompt template pair selected using the predicted confidence values comprises the model-prompt template pair with a highest average predicted confidence value (Zhang, Pg186, Right Column, Paragraph2, Lines7-9, "If p_i (the confidence function) is smaller than some threshold, it should be unreliable and then rejected; otherwise, it is accepted as a confident prediction"
Pg192, Left Column, Section 2), Equations 28-30,
PNG
media_image5.png
144
408
media_image5.png
Greyscale
, "The single-value uncalibrated confidence p_i is then define as the maximum value of the class-specific scores, and the prediction y_i is the class with the maximum score", wherein Zhang explicitly discloses in Equation 28 that pi1, pi2, … represents the list of the predicted confidence values where pi_hat represents the highest value among them. Then Zhang picks yi_hat which is the corresponding pair to that highest confidence value, rendering it functionally equivalent to the claimed invention of selecting a pair that has the highest confidence value as a response.)
Shen and Zhang are analogous to the claimed invention as they are from the same field of endeavor of artificial intelligence decision-making and automated machine learning model selection frameworks. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the multi-model and prompt selection pipeline for task execution of Shen with the teaching of training a separate auxiliary confidence predictor model to estimate prediction reliability of Zhang. The motivation as recited by Zhang (Zhang, Pg186, Right Column, Paragraph2, Lines 1-4, "However, no matter how accurate the model is, prediction errors are still inevitable; therefore, we need another function to measure the probabilistic correctness of the prediction") such that the integration of the two would quantitatively evaluate the predicted reliability of each candidate model-prompt template pair prior to task execution. Thereby avoiding costly execution failures and improving overall system task success rates.
As to independent Claim 7,
it is a product claim that contains similar limitations of Claim 1 and thus rejected under the same rationale.
As to dependent Claim 10,
it is a product claim that contains similar limitations of Claim 2 and thus rejected under the same rationale.
As to dependent Claim 11,
it is a product claim that contains similar limitations of Claim 3 and thus rejected under the same rationale.
As to dependent Claim 12,
it is a product claim that contains similar limitations of Claim 4 and thus rejected under the same rationale.
As to dependent Claim 13,
it is a product claim that contains similar limitations of Claim 5 and thus rejected under the same rationale.
As to independent Claim 15,
it is a system claim that contains similar limitations of Claim 1 and thus rejected under the same rationale.
As to dependent Claim 16,
it is a system claim that contains similar limitations of Claim 2 and thus rejected under the same rationale.
As to dependent Claim 17,
it is a system claim that contains similar limitations of Claim 3 and thus rejected under the same rationale.
As to dependent Claim 18,
it is a system claim that contains similar limitations of Claim 4 and thus rejected under the same rationale.
As to dependent Claim 19,
it is a system claim that contains similar limitations of Claim 5 and thus rejected under the same rationale.
Claims 6, 14, 20 are rejected under 35 U.S.C. 103 as being unpatentable over Shen, Groh and Zhang as mentioned in Claim 1, in further view of Jain et al. (Jain), Non-Patent Literature, “Lightweight reranking for language model generations”, Published in Jan 2024, arXiv, 18Pages.
As to dependent Claim 6,
The combination of Shen, Groh and Zhang teaches, as mentioned above, all the limitations of Claim 1. It teaches about the overall architecture of creating responses comprising recommended AI model and its prompt template in response to the user’s input query, which is embedded into the embedding space to find nearest neighbors that shares the similar traits so the confidence values can be computed to choose the most suitable model-prompt pair.
As mentioned above and from Claim 1, the combination teaches the model-prompt template pair selected using the predicted confidence values (Zhang, Pg186, Right Column, Paragraph2, Lines7-9, "If p_i (the confidence function) is smaller than some threshold, it should be unreliable and then rejected; otherwise, it is accepted as a confident prediction",
Shen, Pg7, Section3.4, Response Generation, Lines1-4, "After all task executions are completed, HuggingGPT needs to generate the final responses. As shown in Table 1, HuggingGPT integrates all the information from the previous three stages (task planning, model selection, and task execution) into a concise summary in this stage, including the list of planned tasks, the selected models for the tasks, and the inference results of the models",
Shen, Pg5, Table1 above,
Groh, Pg270, Section 4.1, Lines2-4, "for a query q in Rd, the simple greedy downhill search with backtracking from Algorithm1 is executed",
Groh, Pg269, Section3.1, Lines3-4, "For the sake of simplicity, here, we assume an Euclidean space
PNG
media_image2.png
26
159
media_image2.png
Greyscale
and Euclidean distances
PNG
media_image3.png
27
65
media_image3.png
Greyscale
".)
Shen further teaches the computer-implemented method of claim 1, further comprising:
constructing a second response to the input query (Shen, Pg5, Table 1 above, wherein Shen’s HuggingGPT pipeline shows that it first goes through the Model Selection part which provides pairs of candidate models and prompts to be used for the models (the corresponding first response). And the pipeline also has the Response Generation part, (the corresponding second response), where it is the actual response or the given task completed according to the input query, rendering it functionally equivalent to the claimed invention of where the first response comprises a model-prompt pair and the second response comprises the actual task completed.)
Shen, however, does not teach the following limitations:
generating a plurality of candidate responses to the input query
computing a plurality of similarity scores between pairs of candidate responses in the plurality of candidate responses
a candidate response with a highest average similarity score.
From the same field of endeavor, Jain teaches generating a plurality of candidate responses to the input query (Jain, Pg1, Right Column, Lines12-16, "In this approach, the underlying model is not touched – we instead take multiple samples from the model and select the best one post-facto using a reranking method"
Pg1, Introduction, Lines11-15, "Multiple output samplings for the same input can produce certain generations which are of substantially higher quality than the quality of the average generation from the model", wherein Jain explicitly discloses using the same input to generate multiple samples (the corresponding plurality of candidate responses), which they can be used to compute the pairwise similarity metrics, rendering it functionally equivalent to the claimed invention.)
Jain further teaches computing a plurality of similarity scores between pairs of candidate responses in the plurality of candidate responses (Jain, Pg4, Section 3, Paragraph2, Lines1-4, "Once we have our similarity function, we can define a generalized self-consistency score GSC_sim(i) for each generation I, given by
PNG
media_image6.png
38
239
media_image6.png
Greyscale
",
Pg1, Abstract, Lines10-13, "our approach relies on easy to compute pairwise statistics between the generations that have minimal compute overhead",
Pg2, Right Column, Second Bullet, Lines1-2, "As our method is based on pairwise similarity between generations", wherein Jain explicitly discloses computing pairwise statistics (the corresponding scores) using the function, where Sim(i,j) represents the similarity between response i and response j, rendering it functionally equivalent to the claimed invention of computing similarity scores for the candidates mentioned above.)
Jain further teaches a candidate response with a highest average similarity score (Jain, Pg3, Right Column, Last paragraph, Lines11-14, "We then try to identify a generation such that the i average pairwise fractional agreement for that generation with all other generations is maximized"
Pg4, Theorem 2.2, Lines5-7, "selecting the generation with the highest average fractional agreement with other generations will pick g"
Pg5, Right Column, Paragraph2, Lines1-3, "Finally, to rank the generations, we employ argmax GSC_sim(i) where Sim can take the form of UCS, WUCS, or Consensus-UCS", wherein Jain explicitly discloses of choosing the generation (the corresponding response) that has the highest average similarity score, rendering it functionally equivalent to the claimed invention.)
Shen, Groh, Zhang and Jain are analogous to the claimed invention as they are from the same field of endeavor of artificial intelligence, large language models, and automated response generation systems. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine and modify the multi-stage task execution framework and model selection process of Shen, GPU-accelerated graph nearest neighbor search of Groh, confidence calibration and reject option mechanism of Zhang with the generation of a plurality of candidate responses and selection based on pairwise similarity scores of Jain. The motivation as recited by Jain (Jain, Pg1, Abstract, Lines3-13, “Reranking and selecting the best generation from the sampled set is a popular way of obtaining strong gains in generation quality… relies on easy to compute pairwise statistics between the generations that have minimal compute overhead” and Pg1, Introduction, Lines11-15, "Multiple output samplings for the same input can produce certain generations which are of substantially higher quality than the quality of the average generation from the model") such that modification of the execution stage of Shen to generate a plurality of candidate responses exhibiting the highest average pairwise similarity score, thereby significantly enhancing output accuracy and consistency without incurring excessive computational overhead.
As to dependent Claim 14,
it is a product claim that contains similar limitations of Claim 6 and thus rejected under the same rationale.
As to dependent Claim 20,
it is a system claim that contains similar limitations of Claim 6 and thus rejected under the same rationale.
Claims 8, 9 are rejected under 35 U.S.C. 103 as being unpatentable over Shen, Groh and Zhang as mentioned in Claim 1, in further view of Banipal et al. (Banipal), US-Patent Application Publication, US-2022/0335302-A1, Published in October 2022.
As to dependent Claim 8,
The combination of Shen, Groh and Zhang teaches, as mentioned above, all the limitations of Claim 7. It teaches about the overall architecture of creating responses comprising recommended AI model and its prompt template in response to the user’s input query, which is embedded into the embedding space to find nearest neighbors that shares the similar traits so the confidence values can be computed to choose the most suitable model-prompt pair.
However, the combination does not teach the computer program product of claim 7, wherein the stored program instructions are stored in a computer readable storage device in a data processing system, and wherein the stored program instructions are transferred over a network from a remote data processing system.
From the same field of endeavor, Banipal teaches this limitation (Banipal, Pg11, Claim15, "The computer program product of claim 14, wherein the stored program instructions are stored in a computer readable storage device in a data processing system, and wherein the stored program instructions are transferred over a network from a remote data processing system.")
Shen, Groh, Zhang and Banipal are analogous to the claimed invention as they are from the same field of endeavor of AI and machine learning-based decision-making and data processing systems. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine LLM-powered task planning and multi-model orchestration of Shen, GPU-accelerated hierarchical graph search and fast nearest-neighbor indexing of Groh, confidence calibration and predictive rejection mechanisms of Zhang with decision-tree question sequence prediction, confidence deviation tracking, and generative custom path assembly of Banipal. The motivation is as recited by Banipal (Banipal, Pg1, Paragraph16, Lines4-11, “estimating the configuration of assets needed to support a cloud application is difficult. For example, a complex platform, such as an integration platform, offers a large variety of hardware and software configuration options. It can be difficult for users to determine the optimal configuration based on usage requirements, sometimes resulting in undersized or oversized platforms”) such that when interactive user inputs or projected usage parameters deviate from predicted pathways or remain ambiguous, the system accurately derives the user’s true intent via confidence signal verification and dynamic query generation, thereby automatically recommending an optimal resource setup without under-sizing or oversizing cloud computing platforms.
As to dependent Claim 9,
The combination of Shen, Groh and Zhang teaches, as mentioned above, all the limitations of Claim 7. It teaches about the overall architecture of creating responses comprising recommended AI model and its prompt template in response to the user’s input query, which is embedded into the embedding space to find nearest neighbors that shares the similar traits so the confidence values can be computed to choose the most suitable model-prompt pair.
However, the combination does not teach the computer program product of claim 7, wherein the stored program instructions are stored in a computer readable storage device in a server data processing system, and wherein the stored program instructions are downloaded in response to a request over a network to a remote data processing system for use in a computer readable storage device associated with the remote data processing system, further comprising:
program instructions to meter use of the program instructions associated with the request; and
program instructions to generate an invoice based on the metered use.
From the same field of endeavor, Banipal teaches this limitation (Banipal, Pg11, Claim16, "16. The computer program product of claim 14, wherein the stored program instructions are stored in a computer readable storage device in a server data processing system, and wherein the stored program instructions are downloaded in response to a request over a network to a remote data processing system for use in a computer readable storage device associated with the remote data processing system, further comprising: program instructions to meter use of the program instructions associated with the request; and program instructions to generate an invoice based on the metered use.")
Shen, Groh, Zhang and Banipal are analogous to the claimed invention as they are from the same field of endeavor of AI and machine learning-based decision-making and data processing systems. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine LLM-powered task planning and multi-model orchestration of Shen, GPU-accelerated hierarchical graph search and fast nearest-neighbor indexing of Groh, confidence calibration and predictive rejection mechanisms of Zhang with decision-tree question sequence prediction, confidence deviation tracking, and generative custom path assembly of Banipal. The motivation is as recited by Banipal (Banipal, Pg1, Paragraph16, Lines4-11, “estimating the configuration of assets needed to support a cloud application is difficult. For example, a complex platform, such as an integration platform, offers a large variety of hardware and software configuration options. It can be difficult for users to determine the optimal configuration based on usage requirements, sometimes resulting in undersized or oversized platforms”) such that when interactive user inputs or projected usage parameters deviate from predicted pathways or remain ambiguous, the system accurately derives the user’s true intent via confidence signal verification and dynamic query generation, thereby automatically recommending an optimal resource setup without under-sizing or oversizing cloud computing platforms.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Shnitzer et al. Non-Patent Literature, “Large Language Model Routing with Benchmark Datasets”, Published in September 2023, arXiv, 18 Pages
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DONG YOON JUNG whose telephone number is (571)270-0198. The examiner can normally be reached 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Cesar Paula can be reached at (571) 272-4128. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DONG YOON JUNG/ Examiner, Art Unit 2145
/CESAR B PAULA/ Supervisory Patent Examiner, Art Unit 2145
Prosecution Insights