Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Claims 1-20 are pending. Claims 1, 8, and 15 are independent.
This Application was published as US 20250148220.
Apparent priority is 3 November 2023.
The instant Application is directed to a method of prompt engineering of splitting an LLM prompt into sub-prompts.
Applicant’s amendments and arguments are considered but are either unpersuasive or moot in view of the new grounds of rejection that, if presented, were necessitated by the amendments to the Claims.
This action is Final.
Response to Amendment
Applicant’s amendments to the Specification and have overcome each and every objection to the specification previously set forth in the Non-Final Office Action mailed 14 November 2025.
However, amendments to the claims have introduced new issues under 35 USC 112 which are discussed below.
Response to Arguments
35 USC 103
Applicant’s arguments with respect to 35 USC 103 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-20 rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Amended claim 1 includes: “calculate a distance between the reference state and the output state of the graph based on the first LLM output; based on the calculated distance being greater than zero, create a second LLM prompt…” Applicant’s remarks dated 17 Feb 2026 state that support for claim amendments is found in [0005], [0059], and [0095] of the Specification, and Claim 6, as filed. However, these paragraphs and claim do not disclose calculating a distance between graph nodes/states, or creating a second prompt based on the distance being greater than zero. [0054] and [0055] disclose calculating whether states have zero distance, but this is used to determine if the diagram commutes, or how far it is from the correct answer, rather than for determining whether to generate additional prompts.
Claims 8 and 15 rejected for the same issue.
Claims 2-7, 9-14, and 16-20 rejected as depending on claims 1, 8, and 15.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 6, 14, and 20 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 6 recites: “calculate a distance between the one or more other edges connecting the first state node and the second state node;” It is indefinite what it means to calculate a distance between one or more edges. Calculating a distance between a single edge does not have a clear meaning. If there are multiple edges, it is indefinite which edges are used as endpoints. For the purposes of prosecution, the claims are interpreted to mean calculating a distance between the nodes by summing the connecting edges.
Claims 14 and 20 are rejected for the same indefinite language.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-4, 6-9, 11-12, 14-18, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Khot et al. ("Decomposed Prompting: A MODULAR APPROACH FOR SOLVING COMPLEX TASKS") in view of Besta et al. ("Graph of Thoughts: Solving Elaborate Problems with Large Language Models") and Codecademy (“Greedy Best-First Search”).
Regarding claim 1, Khot discloses: A system comprising: a processor; and a computer-readable medium storing instructions that are operative upon execution by the processor to: ("In DECOMP, the core is a decomposer LLM that tries to solve a complex task by generating a prompting program P for it." pg. 3, para 5 - Both a program and an LLM inherently require a processor and memory to run it.)
receive an input large language model (LLM) prompt; (Fig. 3 shows an input LLM prompt of "QC: Concatenate the second letter of every word in "John Smith" using spaces")
generate a graph that includes an output state with a reference state, wherein the output state and the reference state have mathematical values; (not explicitly disclosed)
create a first LLM prompt based on the input LLM prompt, (Fig. 3 shows several created prompts)
the first LLM prompt representing a first step toward generating a solution to the input LLM prompt; (Fig. 3 shows a first sub-prompt of "Q2: (foreach) [str_pos] What is the second letter in #1?"" )
submit the first LLM prompt to an LLM as a first sub-query, thereby resulting in the generation of a first LLM output; (Fig. 3 shows a first LLM output "#2: ["o", "m"] – see also: "Each sub-task function f, in turn, is operationalized via a sub-task handler as an in-context prompting LLM (e.g., a separate CoT-style prompt or a additional prompting program dedicated to that sub-task), or any other symbolic or learned function (e.g., a calculator or specialized supervised trained model)." Pg. 3, para 6)
calculate a distance between the reference state and the output state of the graph based on the first LLM output; based on the calculated distance being greater than zero, (not explicitly disclosed)
create a second LLM prompt based on the input LLM prompt, the second LLM prompt representing a second step toward generating the solution to the input LLM prompt, the second LLM prompt including the first LLM output; (Fig. 3 shows a second prompt "Q3: [merge] Concatenate #2 with spaces". "#2" represents the first LLM output.)
submit the second LLM prompt to the LLM as a second sub-query, thereby resulting in the generation of a second LLM output; and (Fig. 3 shows output of the second prompt: #3: "o m" )
cause the second LLM output to be displayed as the solution to the input LLM prompt in response to the input LLM prompt. ("When the special end-of-questions [EOQ] marker is generated, the previous answer is returned as the final prediction." Fig. 3 description)
Khot does not explicitly disclose: generate a graph that includes an output state with a reference state, wherein the output state and the reference state have mathematical values; and
calculate a distance between the reference state and the output state of the graph based on the first LLM output; based on the calculated distance being greater than zero, create a second LLM prompt.
Besta discloses: generate a graph that includes an output state with a reference state, wherein the output state and the reference state have mathematical values; (Fig. 1 shows a graph on the right hand side with an output state. Any of the other states being considered reads on a reference state. Besta also discloses mathematical values for the states: “We have G′ = T (G, pθ) = (V ′,E′), where V ′ = (V ∪ V +) \ V − and E′ = (E ∪ E+) \ E−. V + and E+ are new vertices and edges inserted into G to model the new thoughts and their dependencies, respectively” Pg. 3, last para.)
Besta also discloses: based on the calculated distance being greater than zero, create a second LLM prompt (Fig. 1 shows that the process continues until the state reaches the Output. Besta discloses a controller that decides whether the process should be finalized or another round of interaction with the LLM should be initiated, Pg. 4, section 4.4.)
Khot and Besta are considered analogous art to the claimed invention because they disclose methods of prompt engineering. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Khot with a graph to determine the best solution as taught by Besta. Doing so would have been beneficial because it increases quality and reduces costs over state of the art. (Besta, Abstract.)
Besta does not explicitly disclose: calculate a distance between the reference state and the output state of the graph based on the first LLM output.
Codecademy discloses: calculate a distance between the reference state and the output state of the graph based on the first LLM output. (Figure on pg. 2 shows distance is calculated between each node. Codecademy further discloses that the graph traversal continues until the distance of the reference node to the output node is zero: “U has the lowest cost compared to M and R, so the search will continue by exploring U. Finally, S has a heuristic value of 0 since that is the target node” pg. 3, last para.)
Khot and Besta are considered analogous art to the claimed invention because they disclose methods of prompt engineering. Besta and Codecademy are considered analogous art to the claimed invention because they disclose graph traversal. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination with a greedy best-first search algorithm to determine the graph traversal path as taught by Codecademy. Doing so would have been beneficial to choose the quickest and shortest path to the target node. (Codecademy, pg. 1 first para.)
Regarding claim 2, Khot discloses: The system of claim 1, wherein the input LLM prompt includes a first sentence and a second sentence, wherein creating the first LLM prompt comprises creating the first LLM prompt to include at least the first sentence, wherein creating the second LLM prompt comprises creating the first LLM prompt to include at least the second sentence. (pg. 45, example 1, shows a first sentence ("There are 15 trees in the grove.") and a second sentence (" After they are done, there will be 21 trees."). A first LLM prompt ("So there must have been 21 − 15 = 6.") contains the first and second sentence information.)
Regarding claim 3, Khot discloses: The system of claim 1, wherein the instructions are further operative to identify a pre-configured template that matches a form of the input LLM prompt, wherein creating the first LLM prompt comprises creating the first LLM prompt based on matching one or more portions of the input LLM prompt to one or more portions of the pre-configured template. ("Most CoT systems Wei et al. (2022); Wang et al. (2023) rely on extracting the answer by finding the number following “answer is”." pg. 14, para 6 - matching the phrase "answer is" to determine the answer reads on a template.)
Regarding claim 4, Khot discloses: The system of claim 1, wherein the instructions are further operative to: create a deconstruction LLM prompt based on the input LLM prompt, ("In DECOMP, the core is a decomposer LLM that tries to solve a complex task by generating a prompting program P for it." pg. 3, para 5; See also: "Fig. 11 shows the decomposition prompt we use. The decomposer generates (singlehop) sub-questions and delegates them to retrieve odqa (described in Fig. 6)." pg. 8, last para.)
the deconstruction LLM prompt being formed to query the LLM to identify multiple sub-steps from within input LLM prompt; and ("Each step of P directs a simpler sub-query to a function in an auxiliary set of sub-task functions F available to the system." pg. 3, para 5)
submit the deconstruction LLM prompt to the LLM, thereby generating a deconstruction LLM output that includes at least a first step and a second step, wherein creating the first LLM prompt based on the input LLM prompt further includes creating the first LLM prompt based on the first step, wherein creating the second LLM prompt based on the input LLM prompt further includes creating the second LLM prompt based on the second step. (See example in Fig. 3, which shows each step is used in prompting the LLM based on the previous step. )
Regarding claim 6, Khot does not disclose the additional limitations.
Besta discloses: The system of claim 1, wherein the graph includes a first state node, a second state node, and one or more edges, the first state node representing an initial state of one or more variables identified by the input LLM prompt and the output state, the second state node representing a second state of the one or more variables and the reference state; (Fig. 1 shows a graph on the right hand side with multiple nodes and edges. Section 3.1 describes that vertices describe solutions (which can be initial, intermediate, or final). As shown in Fig. 2 (middle example), a solution to a sorting task would include modifying variables.)
wherein the instructions are further operative to: identify a first edge connecting the first state node and the second state node, the first edge representing a trusted application of an input LLM query to the one or more variables, thereby resulting in a trusted solution; identify one or more other edges connecting the first state node and the second state node, the one or more other edges representing application of the input LLM query via the LLM, thereby resulting in the solution;
calculate a distance between the one or more other edges connecting the first state node and the second state node; and
determine, from the calculated distance, whether or not the graph commutes based on whether the solution matches the trusted solution. (“3.3 Scoring & Ranking Thoughts Thoughts are scored to understand whether the current solution is good enough. A score is modeled as a general function E(v, G, pθ), where v is a thought to be evaluated. We use the state of the whole reasoning process (G) in E for maximum generality, because – for example – in some evaluation scenarios, scores may be relative to other thoughts." pg. 4, para 5 - See also Fig. 1, which shows certain solutions have a negative score and require backtracking.)
Khot and Besta are considered analogous art to the claimed invention because they disclose methods of prompt engineering. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Khot with a graph to determine the best solution as taught by Besta. Doing so would have been beneficial because it increases quality and reduces costs over state of the art. (Besta, Abstract.)
Besta does not explicitly disclose: calculate a distance between the one or more other edges connecting the first state node and the second state node; and determine, from the calculated distance, whether or not the graph commutes.
Codecademy discloses: calculate a distance between the one or more other edges connecting the first state node and the second state node; and determine, from the calculated distance, whether or not the graph commutes. (Figure on pg. 2 shows distance is calculated for each node. Codecademy further discloses that the algorithm determines from the distance if the graph commutes: “4. If a child node is the target, return a success…” pg. 1, last para. See also, “Finally, S has a heuristic value of 0 since that is the target node” pg. 3, last para)
Khot and Besta are considered analogous art to the claimed invention because they disclose methods of prompt engineering. Besta and Codecademy are considered analogous art to the claimed invention because they disclose graph traversal. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination with a greedy best-first search algorithm to determine the graph traversal path as taught by Codecademy. Doing so would have been beneficial to choose the quickest and shortest path to the target node. (Codecademy, pg. 1 first para.)
Regarding claim 7, Khot discloses: The system of claim 1, further comprising an LLM computing device that is configured to: receive the first LLM prompt from the processor; generate the first LLM output based on applying the first LLM prompt as input to the LLM; transmit the first LLM output to the processor; receive the second LLM prompt from the processor; generate the second LLM output based on applying the second LLM prompt as input to the LLM; and transmit the second LLM output to the processor. ("In DECOMP, the core is a decomposer LLM that tries to solve a complex task by generating a prompting program P for it." pg. 3, para 5 - Both a program and an LLM inherently require a processor and memory to run it. It is inherent that the prompts and outputs are received and transmitted by a processor. All other limitations are mapped in claim 1.)
Claim 8 is a method claim with limitations corresponding to the limitations of Claim 1 and is rejected under similar rationale.
Claim 9 is a method claim with limitations corresponding to the limitations of Claim 2 and is rejected under similar rationale.
Claim 11 is a method claim with limitations corresponding to the limitations of Claim 3 and is rejected under similar rationale.
Claim 12 is a method claim with limitations corresponding to the limitations of Claim 4 and is rejected under similar rationale.
Claim 14 is a method claim with limitations corresponding to the limitations of Claim 6 and is rejected under similar rationale.
Claim 15 is a computer storage device claim with limitations corresponding to the limitations of Claim 1 and is rejected under similar rationale.
Claim 16 is a computer storage device claim with limitations corresponding to the limitations of Claim 2 and is rejected under similar rationale.
Claim 17 is a computer storage device claim with limitations corresponding to the limitations of Claim 3 and is rejected under similar rationale.
Claim 18 is a computer storage device claim with limitations corresponding to the limitations of Claim 4 and is rejected under similar rationale.
Claim 20 is a computer storage device claim with limitations corresponding to the limitations of Claim 6 and is rejected under similar rationale.
Claim(s) 5, 13, and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Khot in view of Besta and Codecademy as applied in claim 1 above, further in view of Weng et al. ("Large Language Models are Better Reasoners with Self-Verification").
Regarding claim 5, Khot discloses: The system of claim 1, wherein the instructions are further operative to: create a verification LLM prompt based on the first LLM output; (not explicitly disclosed)
submit the verification LLM prompt to the LLM, thereby generating a verification LLM output; (not explicitly disclosed )
compare the verification LLM output to the solution based on a comparison metric; and cause a result of the comparison to be displayed. (Fig. 12 shows a display of a comparison to ground truth.)
Khot does not explicitly disclose creating and submitting a verification LLM prompt to verify the LLM output. Neither does Besta or Codecademy.
Weng discloses: The system of claim 1, wherein the instructions are further operative to: create a verification LLM prompt based on the first LLM output; ("By performing a backward verification of the answers that LLM deduced for itself, we can obtain interpretable answer validation scores to select the candidate answer with the highest score." Abstract)
submit the verification LLM prompt to the LLM, thereby generating a verification LLM output; ("In forward reasoning, the LLM reasoners generate candidate answers with the chain of thought prompting. We augment the input with several CoT prompts similar to the original query and then send it to the LLM. The LLM then performs sampling decoding to generate multiple candidates for verification." pg. 3, para 5)
compare the verification LLM output to the solution based on a comparison metric; and ("Figure 2: Example of self-verification. In the step one, LLM generates candidate answers and forms different conclusions. Then, in the step two, LLM verifies these conclusions in turn and computes the verification score." pg. 4)
cause a result of the comparison to be displayed. (Fig. 2 shows a display of the scores by either a check or an x. Additionally, outputting the correct answer would read on displaying a result of the comparison.)
Khot, Besta and Weng are considered analogous art to the claimed invention because they disclose methods of prompt engineering. Besta and Codecademy are considered analogous art to the claimed invention because they disclose graph traversal. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination with verification of the output as taught by Weng. Doing so would have been beneficial in order to select the candidate answer with the highest score. (Weng, Abstract.)
Claim 13 is a method claim with limitations corresponding to the limitations of Claim 5 and is rejected under similar rationale.
Claim 19 is a computer storage device claim with limitations corresponding to the limitations of Claim 5 and is rejected under similar rationale.
Claim(s) 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Khot in view of Besta and Codecademy as applied in claim 9 above, further in view of Guo (CN 116958320 A).
Regarding claim 10, Khot discloses: The method of claim 9, further comprising parsing the input LLM prompt based on one or more of sequence adverbs and temporal adverbs, (not explicitly disclosed)
wherein creating the first LLM prompt further comprises removing at least one adverb from the first sentence. (Pg. 45, example 1, contains the adverb "today" in the second sentence. It is removed in the prompt "Then there were 21 trees after some more were planted." )
Khot does not explicitly disclose parsing the input prompt for adverbs to be removed. Neither does Besta or Codecademy.
Guo discloses: The method of claim 9, further comprising parsing the input LLM prompt based on one or more of sequence adverbs and temporal adverbs, wherein creating the first LLM prompt further comprises removing at least one adverb from the first sentence. ("For a certain acquired prompt, the key element in the prompt can be extracted, specifically, the noun element (such as entity noun) of the prompt is reserved, and the adjectives and adverbs therein are removed..." pg. 9, para 9)
Khot, Besta, and Guo are considered analogous art to the claimed invention because they disclose methods of prompt engineering. Besta and Codecademy are considered analogous art to the claimed invention because they disclose graph traversal. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination with removal of adverbs as taught by Guo. Doing so would have been beneficial in order to focus on the key elements in the prompt. (Guo, pg. 9, para 9.)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Briliauskas (US 20240232355 A1). Briliauskas discloses traversing a graph until the distance to the target node is zero, in order to improve speed of a query. (“[0085] To further improve the speed for a set of target/query files, the search can stop a search when an exact match (e.g., distance=0) is found for a given node for the current target code and proceed to the next target code...”)
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JON C MEIS whose telephone number is (703)756-1566. The examiner can normally be reached Monday - Thursday, 8:30 am - 5:30 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hai Phan can be reached at 571-272-6338. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JON CHRISTOPHER MEIS/Examiner, Art Unit 2654
/HAI PHAN/Supervisory Patent Examiner, Art Unit 2654
Prosecution Insights