Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Election/Restrictions
Applicant’s election without traverse of Group 4 (claims 1-4,8-10,20-27) in the reply filed on 8/6/2026 is acknowledged.
Drawings
The drawings were received on 12/22/2023. These drawings are accepted.
Claim Objections
Claim 3 recites the limitation “the one or more query difficulty metrics” while claim 1 recites “one or more query difficulty metric values”. For clarity, please amend the claimed language for proper consistency.
Claim 23 recites the limitation “the one or more query difficulty metrics” while claim 21 recites “one or more query difficulty metric values”. For clarity, please amend the claimed language for proper consistency.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 4 recites the limitation "a first query difficulty level" in claim 1. There is insufficient antecedent basis for this limitation in the claim.
Claim 10 recites the limitation “the quality metrics” and “the query responses” in claim 1,8,9. There is insufficient antecedent basis for this limitation in the claim.
Claim 27 recites the limitation “the quality metric” in claim 21,25,26. Which quality metric is referenced: the quality metric for the first response or the quality metric for the second response? There is insufficient antecedent basis for this limitation in the claim.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1,20,21 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Atlan et al (US Publication No.: 20240185137).
Claim 1, Atlan et al discloses
At least one memory (Paragraph 99,100 discloses memory.);
At least one hardware processor coupled to the at least one memory (Paragraph 97 discloses processor. Paragraph 96 discloses one or more hardware modules of a computer system such as a processor may be configured by software.); and
One or more computer readable storage media storing computer-executable instructions, that when executed, cause the computing system (Paragraph 96,97) to perform routing operations for natural language generator models (paragraph 66, Fig. 2.) that reduce computing resource use for certain queries without a significant decrease in query response quality (Such is considered intended result from using or performing routing.), the operations comprising:
Receiving a first query (Fig. 2, label 230, paragraph 57 discloses 230 receives natural language queries.);
Generating for the first query a first set of one or more query difficulty metric values for a set of one or more query difficulty metrics (Paragraph 66 discloses 230 computes a complexity score of the query. This would occur for queries received generating a set of one or more query difficulty metrics.);
Determining, based on the first set of one or more query difficulty metric values, that the first query is of a first difficulty level (Paragraph 66 discloses computing a complexity score or difficulty metric.);
Based on the determining that the first query is a first query difficulty level, routing the first query to a first natural language generator model specified for the first query difficulty level (Paragraph 66 discloses selecting an LLM based on the complexity score and routing a prompt to the selected LLM. Paragraph 58 discloses the prompt may include text of query along with request.);
Receiving a second query (Fig. 2, label 230);
Generating for the second query a second set of one or more query difficulty metric values for the set of one or more query difficult metrics (Paragraph 66 discloses 230 computes a complexity score of the query. This would occur for queries received generating a set of one or more query difficulty metrics.);
Determining, based on the second set of one or more query difficulty metric values, that the second query is of a second query difficulty level, the second query difficulty level being different than the first query difficulty level (Paragraph 58 discloses queries are received. Paragraph 66 discloses for a query, a complexity score is calculated, indicating for a second query (different from a first query), a second complexity score is calculated for the second query.); and
Based on the determining that the second query is of the second query difficulty level, routing the second query to a second natural language generator model specified for the second query difficulty level (Paragraph 66 discloses selecting an LLM model based on a complexity or difficulty score for a respective query, such as a second difficulty score for a second query. Depending on the selection and the difficulty score, a second LLM is selected, different from the LLM selected for the first query.).
Claim 20, Atlan et al discloses
One or more computer-readable storage media comprising computer- executable instructions that cause a computing system to (Paragraph 96,97) perform routing operations for natural language generator models (paragraph 66, Fig. 2.) that reduce computing resource use for certain queries without a significant decrease in query response quality (Such is considered intended result from using or performing routing.), the one or more computer-readable storage media comprising:
computer-executable instructions that, when executed by a computing system (Paragraph 96,97) comprising
at least one hardware processor (paragraph 96,97) and at least one memory (paragraph 99) coupled to the at least one hardware processor (paragraph 96-99), cause the computing system to receive a first query (Fig. 2, label 230, paragraph 57 discloses 230 receives natural language queries.);
computer-executable instructions that, when executed by the computing system (paragraph 96-99), cause the computing system to generate for the first query a first set of one or more query difficulty metric values for a set of one or more query difficulty metrics (Paragraph 66 discloses 230 computes a complexity score of the query. This would occur for queries received generating a set of one or more query difficulty metrics.);
computer-executable instructions that, when executed by the computing system (paragraph 96-99), cause the computing system to determine, based on the first set of one or more query difficulty metric values, that the first query is of a first query difficulty level (Paragraph 66 discloses computing a complexity score or difficulty metric.);
computer-executable instructions that, when executed by the computing system, cause the computing system to, based on the determining that the first query is of the first query difficulty level, route the first query to a first natural language generator model specified for the first query difficulty level (Paragraph 66 discloses selecting an LLM based on the complexity score and routing a prompt to the selected LLM. Paragraph 58 discloses the prompt may include text of query along with request.);
computer-executable instructions that, when executed by the computing system (paragraph 96-99), cause the computing system to receive a second query (Fig. 2, label 230);
computer-executable instructions that, when executed by the computing system (paragraph 96-99), cause the computing system to generate for the second query a second set of one or more query difficult metric values for the set of one or more query difficulty metrics (Paragraph 66 discloses 230 computes a complexity score of the query. This would occur for queries received generating a set of one or more query difficulty metrics.);
computer-executable instructions that, when executed by the computing system (paragraph 96-99), cause the computing system to determine, based on the second set of one or more query difficulty metric values, that the second query is of a second query difficulty level, the second query difficulty level being different than the first query difficulty level (Paragraph 58 discloses queries are received. Paragraph 66 discloses for a query, a complexity score is calculated, indicating for a second query (different from a first query), a second complexity score is calculated for the second query.); and
computer-executable instructions that, when executed by the computing system (paragraph 96-99), cause the computing system to, based on the determining that the second query is of the second query difficulty level, route the second query to a second natural language generator model specified for the second query difficulty level (Paragraph 66 discloses selecting an LLM model based on a complexity or difficulty score for a respective query, such as a second difficulty score for a second query. Depending on the selection and the difficulty score, a second LLM is selected, different from the LLM selected for the first query.).
Claim 21, Atlan et al discloses
receiving a first query (Fig. 2, label 230, paragraph 57 discloses 230 receives natural language queries.);
generating for the first query a first set of one or more query difficulty metric values for a set of one or more query difficulty metrics (Paragraph 66 discloses 230 computes a complexity score of the query. This would occur for queries received generating a set of one or more query difficulty metrics.);
determining, based on the first set of one or more query difficulty metric values, that the first query is of a first query difficulty level (Paragraph 66 discloses computing a complexity score or difficulty metric.);
based on the determining that the first query is of the first query difficulty level, routing the first query to a first natural language generator model specified for the first query difficulty level (Paragraph 66 discloses selecting an LLM based on the complexity score and routing a prompt to the selected LLM. Paragraph 58 discloses the prompt may include text of query along with request.);
receiving a second query (Fig. 2, label 230);
generating for the second query a second set of one or more query difficulty metric values for the set of one or more query difficulty metrics (Paragraph 66 discloses 230 computes a complexity score of the query. This would occur for queries received generating a set of one or more query difficulty metrics.);
determining, based on the second set of one or more query difficulty metric values, that the second query is of a second query difficulty level, the second query difficulty level being different than the first query difficulty level (Paragraph 58 discloses queries are received. Paragraph 66 discloses for a query, a complexity score is calculated, indicating for a second query (different from a first query), a second complexity score is calculated for the second query.); and
based on the determining that the second query is of the second query difficulty level, routing the second query to a second natural language generator model specified for the second query difficulty level (Paragraph 66 discloses selecting an LLM model based on a complexity or difficulty score for a respective query, such as a second difficulty score for a second query. Depending on the selection and the difficulty score, a second LLM is selected, different from the LLM selected for the first query.).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 2,22 is/are rejected under 35 U.S.C. 103 as being unpatentable over Atlan et al (US Publication No.: 20240185137) in view of Daianu et al (US Patent No.: 11650996).
Claim 2, Atlan et al discloses determining query difficulty level (paragraph 66), but fails to disclose the generation of query difficulty level comprises generating a vector-space embedding of the first query; and submitting the vector-space embedding to a machine learning model to provide at least one query difficulty metric value of the one or more query difficulty metrics for the first query.
Daianu et al discloses generating a vector-space embedding of the first query (Fig. 1, label 122,124. Fig. 2a shows the internals of 122 where a query vector or vector-space embedding is generated of the first query.);
submitting the vector-space embedding to a machine learning model to provide at least one query difficulty metric value of the one or more query difficulty metrics for the first query (Fig. 4, label 124,142. Col. 5, lines 1-11 discloses the complexity model (also a machine learning model) 142 analyzes the query vector 124 to identify relative complexity level of the query vector 124.).
It would be obvious to one skilled in the art before the effective filing date of the application to modify the difficulty level of Atlan et al by modifying the generation as disclosed by Daianu et al so to improve understanding the query, hence improve response to the query.
Claim 22 recites similar limitations as claim 2 and is rejected on the same basis as claim 2.
Claim(s) 3-4,23-24 is/are rejected under 35 U.S.C. 103 as being unpatentable over Atlan et al (US Publication No.: 20240185137) in view of Lee et al (US Publication No.: 20240362468).
Claim 3, Atlan et al discloses query difficulty metric (paragraph 66), but fails to disclose wherein a query difficulty metric of the one or more query difficulty metrics corresponds to a probability of an evaluated query having the first query difficulty level.
Lee et al discloses wherein a query difficulty metric of the one or more query difficulty metrics corresponds to a probability of an evaluated query having the first query difficulty level (Paragraph 49 discloses complexity metrics are associated with received query and confidence scores or accuracy scores, wherein confidence or accuracy scores indicates a probability of the query having a difficulty level or score.).
It would be obvious to one skilled in the art before the effective filing date of the application to modify Atlan et al’s query difficulty metric by incorporating query difficulty metric as disclosed by Lee et al so to improve generating a response to a query, hence improving the user’s experience.
Claim 4, Atlan et al discloses wherein the determining, based on the first set of one or more query difficulty metric values, that the first query is of a first query difficulty level (Paragraph 66), comprises comparing the complexity score to a threshold (Paragraph 66 discloses the complexity score is compared to a set of complexity thresholds and based on the complexity score falls within a set of complexity thresholds.), and the first query is routed to the first natural language generator model based on determining that the probability satisfies the threshold probability (Paragraph 66 discloses the complexity score is compared to a set of complexity thresholds and based on the complexity score falls within a set of complexity thresholds, an LLM is selected and used for processing the prompt, wherein the prompt includes the first query (paragraph 58).).
Atlan et al fails to discloses the complexity score includes probability and the comparison comprises comparing the probability to a threshold probability.
Lee et al discloses the complexity score includes probability (Paragraph 49 discloses complexity metrics may be related to confidence or accuracy scores. Some confidence or accuracy scores are lower corresponding to more complex queries. This indicates probably or confidence/accuracy scores.) and the comparison comprises comparing the probability to a threshold probability (Paragraph 49 discloses lower or higher accuracy or confidence scores related to complexity metrics, wherein levels of accuracy or confidence scores are compared to other complex queries, indicating a threshold probability of other complex queries and comparison to probability threshold.).
It would be obvious to one skilled in the art before the effective filing date of the application to modify Atlan et al’s complexity metric with complexity metrics as disclosed by Lee et al so to improve response to the input query by using different systems for processing queries depending on the complexity.
Claim 23 recites similar limitations as claim 3 and is rejected on the same basis as claim 3.
Claim 24, Atlan et al discloses determining the first query is of the first query difficulty level comprising comparison between the level with a threshold (Paragraph 66 discloses the complexity score is compared to a set of complexity thresholds and based on the complexity score falls within a set of complexity thresholds, an LLM is selected and used for processing the prompt, wherein the prompt includes the first query (paragraph 58).), but fails to disclose the comparison comprises comparing the probability to a threshold probability.
Lee et al discloses the complexity score includes probability (Paragraph 49 discloses complexity metrics may be related to confidence or accuracy scores. Some confidence or accuracy scores are lower corresponding to more complex queries. This indicates probably or confidence/accuracy scores.) and the comparison comprises comparing the probability to a threshold probability (Paragraph 49 discloses lower or higher accuracy or confidence scores related to complexity metrics, wherein levels of accuracy or confidence scores are compared to other complex queries, indicating a threshold probability of other complex queries and comparison to probability threshold.).
It would be obvious to one skilled in the art before the effective filing date of the application to modify Atlan et al’s complexity metric with complexity metrics as disclosed by Lee et al so to improve response to the input query by using different systems for processing queries depending on the complexity.
Allowable Subject Matter
Claims 8-10,25-27 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LINDA WONG whose telephone number is (571)272-6044. The examiner can normally be reached 9-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew C Flanders can be reached at 571-272-7516. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/LINDA WONG/Primary Examiner, Art Unit 2655
Prosecution Insights