DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This office action is in response to applicant’s amendment filed on 09/11/2026.
Claims 1-3, 5-11, 13-19, 21-23 are pending and examined.
Claims 4, 12, 20 have been cancelled.
Claims 21-23 are new claims.
Amendment to specification filed on 09/11/2026 is accepted.
Response to Arguments
Applicant’s arguments filed on 09/11/2026 have been fully considered. However, they are moot in light of new grounds of rejection with a new reference (Park) applied.
The examiner is available for a phone interview with applicant.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3, 5, 9, 11, 13, 17, 19, 21 are rejected under 35 U.S.C. 103 as being unpatentable over Bucaioni et al. (Programming with ChatGPT: How far can we go? January 2024, provided in applicant's IDS) hereinafter Bucaioni, in view of Park et al. (US PGPUB 2011/0138236) hereinafter Park.
Per claim 1, Bucaioni further discloses a method of generating a code output in response to a natural language problem description, comprising: receiving, via a communication interface, the natural language problem description; generating, by a neural network based language model, a first candidate code snippet based on a first input prompt combining the natural language problem description and a first instruction to generate the code output; executing, at a code execution environment, the first candidate code snippet based on a unit test thereby producing a first feedback reflecting a correctness of the first candidate code snippet (page 3, right column, 2nd paragraph; “Once the prompt was generated using the GUI tool, it was stored in a GitHub repository. The prompt was then passed as input to ChatGPT to produce an output response, which was then documented in the same GitHub repository for further reference and analysis. Subsequently, the output response was fed as input to LeetCode’s built-in editor to submit it as a solution to the corresponding problem. If the solution generated by ChatGPT passes the test cases on LeetCode, the corresponding feedback, including metrics and statistics, was documented and collected using Google Sheets. In contrast, if an error occurs during the submission process, it is documented, and passed back to the GUI tool”; i.e. ChatGPT receives a prompt in natural language, then generates an output code, the output code is tested in LeetCode (code execution environment), the feedback (error info) is provided to ChatGPT); generating, by the neural network based language model, a second candidate code snippet based on a second input prompt combining the natural language problem description, the first candidate code snippet, and the first feedback; executing, at the code execution environment, the second candidate code snippet based on a runtime test thereby producing a second feedback reflecting the second candidate code snippet; generating, by the neural network based language model, a third candidate code snippet based on a third input prompt combining the natural language problem description, the second candidate code snippet, and the second feedback; and executing the third candidate code snippet at an application associated with the natural language problem description (page 3, right column, 2nd paragraph; “The error message was then formatted into our template for error occurrences, and the modified prompt was once again passed to ChatGPT for another iteration of generating a response. This process was repeated for up to a maximum of three iterations”; page 4, right column; “When a solution was not accepted as correct, the next iteration was based on the error message (a second feedback reflecting the second candidate code snippet) from previous solution. The format for prompts generated from error message was also straightforward and followed this structure: With input “x” output was “y” but expected result was “z”. If the error was a runtime error the complete error message was used as prompt to be fed to ChatGPT”; i.e. based on the natural language input and the test feedback, ChatGPT produces a second code, then tests the second code in LeetCode to produce second feedback, then repeats the same process again to produce and test a third code).
While Bucaioni discloses executing, at the code execution environment, the second candidate code snippet based on a runtime test thereby producing a second feedback reflecting the second candidate code snippet (page 3, right column, 2nd paragraph; “The error message was then formatted into our template for error occurrences, and the modified prompt was once again passed to ChatGPT for another iteration of generating a response. This process was repeated for up to a maximum of three iterations”; page 4, right column; “When a solution was not accepted as correct, the next iteration was based on the error message from previous solution”; i.e. providing the error message as a feedback of the second candidate code snippet to the neural network based language model), Bucaioni does not explicitly teach the error message (second feedback) reflecting an execution time consumed by the second candidate code snippet to pass the unit test. However, an error message containing an execution time consumed by a testing code during a unit test in a known practice in the field of the art, as evidenced in Park (paragraphs [0003][0008][0055]; generating an error log for a code conducting a test, the error log contains execution time of the code during the test). Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Bucaioni and Park to generate an error message containing execution time of the testing code as feedback to a large language model, as such error message provides useful information for debugging and optimizing the testing code.
Per claim 3, Bucaioni further discloses wherein the first feedback takes a form of one or more of a pass of the unit test, an execution failure, a syntax error, a program error, or a timeout error of the unit test, and in response to a failure feedback, the method further comprises: generating, by the neural network based language model, a corrected first candidate code snippet based on an updated input prompt combining the first feedback, the first candidate code snippet, and the natural language problem description; and executing, at the code execution environment, the corrected first candidate code snippet based on the unit test thereby producing an updated first feedback reflecting a correctness of the corrected first candidate code snippet (page 3, right column, 2nd paragraph; “The error message was then formatted into our template for error occurrences, and the modified prompt was once again passed to ChatGPT for another iteration of generating a response. This process was repeated for up to a maximum of three iterations”; i.e. based on the test feedback (error message), generate a new modified prompt, then ChatGPT produces a second code, then tests the second code in LeetCode to produce a second feedback).
Per claim 5, Bucaioni further discloses revalidating a correctness of a neural network generated code output, wherein the revalidation comprises: executing, at the code execution environment, the second candidate code snippet based on the unit test thereby producing a third feedback reflecting a correctness of the second candidate code snippet (page 3, right column, 2nd paragraph; “The error message was then formatted into our template for error occurrences, and the modified prompt was once again passed to ChatGPT for another iteration of generating a response. This process was repeated for up to a maximum of three iterations”; page 3, right column, 2nd paragraph; “LeetCode includes a built-in editor and compiler, which allows users to test their code against a set of predefined test cases for evaluating its correctness and efficiency”; i.e. different users can test/validate a code (second candidate code) using a set of test cases in LeetCode to produce different (second and third) feedbacks).
Claims 9, 11, 13 recite similar limitations as claims 1, 3, 5. Therefore, claims 9, 11, 13 are rejected under similar rationales as claims 1, 3, 5.
Claims 17, 19, 21 recite similar limitations as claims 1, 3, 5. Therefore, claims 17, 19, 21 are rejected under similar rationales as claims 1, 3, 5.
Claims 2, 10 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Bucaioni, in view of Park, in view of Kadel et al. (US patent 11226810) hereinafter Kadel.
Per claim 2, Bucaioni does not explicitly teach wherein the generating of the first feedback comprises: executing the first candidate code snippet based on a testing input to generate a testing output; comparing the testing output with an expected value corresponding to the unit test; determining the first candidate code snippet pass the unit test in response to a difference between the testing output and the expected value being in a predetermined range; and determining the first candidate code snippet fails the unit test in response to the difference between the testing output and the expected value being outside the predetermined range. However, Kadel suggests the above (column 13, line 25-35; testing a code, determine if the test result is within an error range of the expected result, if it does, the code passes the test and can be deployed). Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Bucaioni, Park and Kadel to determine if the test result of a code is within an error range of the expected result, to ensure the fault tolerance and quality of the code.
Claims 10 and 18 recite similar limitations as claim 2. Therefore, claims 10 and 18 are rejected under similar rationales as claim 2.
Claims 6-8, 14-16, 22-23 are rejected under 35 U.S.C. 103 as being unpatentable over Bucaioni, in view of Park, in view of Vembu et al. (US PGPUB 2024/0281225) hereinafter Vembu.
Per claim 6, Bucaioni suggests in response to repeatedly receiving a negative feedback on correctness after a pre- defined quantity of regeneration: measuring execution times of generated candidate code snippets based on the unit test (page 3, right column, 2nd paragraph; “If the solution generated by ChatGPT passes the test cases on LeetCode, the corresponding feedback, including metrics and statistics, was documented and collected using Google Sheets. The error message was then formatted into our template for error occurrences, and the modified prompt was once again passed to ChatGPT for another iteration of generating a response. This process was repeated for up to a maximum of three iterations”; page 2, right column, last paragraph; “In this study, runtime performance refers to the total duration required for a solution to execute”; i.e. iterative testing and receiving error messages and execution time information). Bucaioni does not explicitly teach selecting a candidate code snippet having a shortest execution time. However, Vembu suggests the above (Vembu, paragraph [0130]; testing a plurality of candidate codes, profiling their runtime performance, selecting the candidate code with the best execution time). Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Vembu, Park and Bucaioni to select the candidate code with the best execution time and providing it as feedback to a language model, as this feedback info would help the language model to generate better performing code (for optimization purpose).
Per claim 7, Bucaioni does not explicitly teach wherein multiple candidate code snippets are executed based on the runtime test, each producing a respective runtime efficiency metric, and the method further comprises: selecting one of the multiple candidate code snippets with a highest runtime efficiency metric and a corresponding feedback as part of the third input prompt. However, Vembu in view Bucaioni suggests the above (Vembu, paragraph [0130]; testing a plurality of candidate codes, profiling their runtime performance, selecting the candidate code with the best execution time). Bucaioni further suggests using the runtime profiling info as a part of the third input prompt (page 3, right column, 2nd paragraph). Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Vembu, Park and Bucaioni to select the candidate code with the best execution time and providing it as feedback to a language model, as this feedback info would help the language model to generate better performing code (for optimization purpose).
Per claim 8, Bucaioni does not explicitly teach wherein the code execution environment comprises a hardware environment based on one or more of a central processing unit (CPU), a graphics processing unit (GPU), or an application specific integrated circuit (ASIC). However, Vembu in suggests the above (paragraph [0046]; profiling code performance in an execution environment including CPU utilization rate). Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Vembu and Bucaioni that the code execution environment comprises a CPU, which is a commonly used hardware component to execute a code.
Claims 14-16 recite similar limitations as claims 6-8. Therefore, claims 14-16 are rejected under similar rationales as claims 6-8.
Claims 22-23 recite similar limitations as claims 6-7. Therefore, claims 22-23 are rejected under similar rationales as claims 6-7.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HANG PAN whose telephone number is (571)270-7667. The examiner can normally be reached 9 AM to 5 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chat Do can be reached at 571-272-3721. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HANG PAN/Primary Examiner, Art Unit 2193
Prosecution Insights