CTNF 18/743,866 CTNF 83062 DETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia 1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. 2. This action is responsive to the application file on June 14, 2024. Claims 1-20 are pending and are presenting for examination. Examiner Notes 3. Examiner cites particular columns and line numbers in the references as applied to the claims below for the convenience of the applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested that, in preparing responses, the applicant fully consider the references in entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the examiner. 07-06 AIA 15-10-15 4. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. Claim Objections 07-29-01 AIA 5. Claim s 1-20 are objected to because of the following informalities: As to claims 1, 10, and 15, recite to include the following limitation “LLM” in the claims. As acronym is likely to change its meaning over time, thus, it (API) needs to be spelled out once in the claims. As to claim 9, line 3, recites to include the following limitation “an LLM” should be changed to, for example – [[an]] the LLM – instead . Appropriate correction is required. Claims 2-8, 11-14 and 16-20 are also objected to for being depended upon the objection of base claims 1, 10, and 15 respectively. Claim Rejections - 35 USC § 101 07-04-01 AIA 07-04 6. 35 U.S.C. 101 reads as follows: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. 7. Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Independent Claim 1 recites: One or more non-transitory computer-readable media that include stored thereon computer-executable instructions that when executed by at least a processor cause a computing system to: [a] access a collection of software code samples, wherein the software code samples includes intermixed sample code and human language description of the sample code; [b] generate prompts to an LLM to write code that performs as described by the human language description of the sample code; [c] fine-tune a large language model to generate software code based on a code generation loss function that evaluates code generated by the LLM in response to the prompts; [d] generate an evaluation score for performance of the tuned large language model as a code generator based on a value of the code generation loss function for second generated code; and [e] automatically determine to deploy the tuned large language model to a production environment for code generation in response to the evaluation score satisfying a threshold. Step 2A – prong 1: The claim 1 recites the limitation of: [a] access a collection of software code samples, wherein the software code samples includes intermixed sample code and human language description of the sample code; [b] generate prompts to write code that performs as described by the human language description of the sample code; [c] fine-tune a large language model to generate software code based on a code generation loss function that evaluates code generated by the LLM in response to the prompts; [d] generate an evaluation score for performance of the tuned large language model as a code generator based on a value of the code generation loss function for second generated code; and [e] automatically determine to deploy the tuned large language model to a production environment for code generation in response to the evaluation score satisfying a threshold. These limitations of steps [a]-[e] as draft, are functions that, under its broadest reasonable interpretation, recite the abstract idea of a mental process. The limitations encompass a human mind carrying out the function through observation, evaluation judgment and /or opinion, or even with the aid of pen and paper. Thus, this limitation recites and falls within the “Mental Processes” grouping of abstract ideas under Prong 1. Step 2A – Prong 2: Under Prong 2, this judicial exception is not integrated into a practical application. The claims recite the following additional elements: “ One or more non-transitory computer-readable media that include stored thereon computer-executable instructions that when executed by at least a processor cause a computing system ” and “to and LLM” merely recite instructions to implement an abstract idea on a generic computer, or merely use a generic computer or computer components as a tool to perform the abstract idea, thus is not a practical application under Prong 2. See MPEP 2106.05(f). Accordingly, the additional elements do not integrate the recited judicial exception into a practical application and the claim is therefore directed to the judicial exception. Step 2B: Under Step 2B, the claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of “ One or more non-transitory computer-readable media that include stored thereon computer-executable instructions that when executed by at least a processor cause a computing system ” and “to and LLM” are amount to no more than mere instructions, or generic computer/computer components as a tool to carry out the exception. Accordingly, the claim is not patent eligible under 35 USC 101. Independent Claims 10 and 15 recites: A computer-implemented method, comprising: [1] accessing a collection of software code samples, wherein the software code samples includes intermixed sample code and human language description of the sample code; [2] generating prompts to an LLM to write code that performs as described by the human language description of the sample code; [3] fine-tuning a large language model to generate software code based on a code generation loss function that evaluates code generated by the LLM in response to the prompts; [4] generating an evaluation score for performance of the tuned large language model as a code generator based on a value of the code generation loss function for second generated code; and [5] automatically signal that the fine tuning of the tuned large language model is complete in response to the evaluation score satisfying a threshold. Step 2A – prong 1: The claims 10 and 15 recite the limitation of: [1] accessing a collection of software code samples, wherein the software code samples includes intermixed sample code and human language description of the sample code; [2] generating prompts to write code that performs as described by the human language description of the sample code; [3] fine-tuning a large language model to generate software code based on a code generation loss function that evaluates code generated by the LLM in response to the prompts; [4] generating an evaluation score for performance of the tuned large language model as a code generator based on a value of the code generation loss function for second generated code; and [5] automatically signal that the fine tuning of the tuned large language model is complete in response to the evaluation score satisfying a threshold. These limitations of steps [1]-[5] as draft, are functions that, under its broadest reasonable interpretation, recite the abstract idea of a mental process. The limitations encompass a human mind carrying out the function through observation, evaluation judgment and /or opinion, or even with the aid of pen and paper. Thus, this limitation recites and falls within the “Mental Processes” grouping of abstract ideas under Prong 1. Step 2A – Prong 2: Under Prong 2, this judicial exception is not integrated into a practical application. The claims recite the following additional elements: “A computing system, comprising: at least one processor connected to at least one memory; one or more non-transitory computer-readable media including computer-executable instructions stored thereon that when executed by at least the processor cause the computing system” and “to and LLM” merely recite instructions to implement an abstract idea on a generic computer, or merely use a generic computer or computer components as a tool to perform the abstract idea, thus is not a practical application under Prong 2. See MPEP 2106.05(f). Accordingly, the additional elements do not integrate the recited judicial exception into a practical application and the claims are therefore directed to the judicial exception. Step 2B: Under Step 2B, the claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of “A computing system, comprising: at least one processor connected to at least one memory; one or more non-transitory computer-readable media including computer-executable instructions stored thereon that when executed by at least the processor cause the computing system” and “to and LLM” are amount to no more than mere instructions, or generic computer/computer components as a tool to carry out the exception. Accordingly, the claims are not patent eligible under 35 USC 101. Regarding to claim 2 , the limitation of “generate, as a component of the code generation loss function, a value of code matching loss that indicates an extent of dissimilarity between the sample code and the generated code” recites further mental process. Furthermore, the following additional element “further comprising computer-executable instructions that when executed by at least the processor cause the computing system” merely recites instructions to implement an abstract idea on a generic computer, or merely uses a generic computer or computer components as a tool to perform the abstract idea, thus is not a practical application under Prong 2. See MPEP 2106.05(f). Accordingly, the claim is not patent eligible under 35 USC 101. Regarding to claim 3 , the limitation of “generate, as a component of the code generation loss function, a value of code matching loss that indicates an extent of dissimilarity between the sample code and the generated code” recites further mental process. Furthermore, the following additional element “further comprising computer-executable instructions that when executed by at least the processor cause the computing system” merely recites instructions to implement an abstract idea on a generic computer, or merely uses a generic computer or computer components as a tool to perform the abstract idea, thus is not a practical application under Prong 2. See MPEP 2106.05(f). Accordingly, the claim is not patent eligible under 35 USC 101. Regarding to claim 4 , the limitation of “generate a value of code matching loss further cause to: generate a value of recall between the sample code and the generated code; generate a value of precision between the sample code and the generated code; and combine the values of recall and precision to produce the value of code matching loss” recites further mental process. Furthermore, the following additional element “further comprising computer-executable instructions that when executed by at least the processor cause the computing system” merely recites instructions to implement an abstract idea on a generic computer, or merely uses a generic computer or computer components as a tool to perform the abstract idea, thus is not a practical application under Prong 2. See MPEP 2106.05(f). Accordingly, the claim is not patent eligible under 35 USC 101. Regarding to claim 5 , the limitation of “generate, as a component of the code generation loss function, a value of non-linear code completeness loss that indicates an extent to which the generated code fails to produce expected output on a line-by-line basis.” recites further mental process. Furthermore, the following additional element “further comprising computer-executable instructions that when executed by at least the processor cause the computing system” merely recites instructions to implement an abstract idea on a generic computer, or merely uses a generic computer or computer components as a tool to perform the abstract idea, thus is not a practical application under Prong 2. See MPEP 2106.05(f). Accordingly, the claim is not patent eligible under 35 USC 101. Regarding to claim 6 , the limitation of “access a test case associated with the sample code; record line-by-line outputs of executing the sample code and the generated code on the test case; compare the outputs of corresponding individual lines of the sample code and the generated code; and where the outputs differ for a corresponding individual line, add to the value of non-linear code completeness loss an amount based on a line number of the corresponding individual line” recites further mental process. Furthermore, the following additional element “further comprising computer-executable instructions that when executed by at least the processor cause the computing system” merely recites instructions to implement an abstract idea on a generic computer, or merely uses a generic computer or computer components as a tool to perform the abstract idea, thus is not a practical application under Prong 2. See MPEP 2106.05(f). Accordingly, the claim is not patent eligible under 35 USC 101. Regarding to claim 7 , the limitation of “generate, as a component of the code generation loss function, a value of unit test passing loss that indicates an extent to which the generated code fails to produce output consistent with the sample code” recites further mental process. Furthermore, the following additional element “further comprising computer-executable instructions that when executed by at least the processor cause the computing system” merely recites instructions to implement an abstract idea on a generic computer, or merely uses a generic computer or computer components as a tool to perform the abstract idea, thus is not a practical application under Prong 2. See MPEP 2106.05(f). Accordingly, the claim is not patent eligible under 35 USC 101. Regarding to claim 8 , the limitation of “access a test case associated with the sample code; compare results of the test case executed by the sample code and executed by the generated code; and where the results differ, set the value of the unit test passing loss to indicate that unit test has failed” recites further mental process. Furthermore, the following additional elements “ execute the sample code and the generated code on the test case ” is merely applying the judicial exception or abstract idea, and “further comprising computer-executable instructions that when executed by at least the processor cause the computing system” merely recites instructions to implement an abstract idea on a generic computer, or merely uses a generic computer or computer components as a tool to perform the abstract idea, thus is not a practical application under Prong 2. See MPEP 2106.05(f). Accordingly, the claim is not patent eligible under 35 USC 101. Regarding to claim 9 , the limitation of “generate prompts to an LLM to write code that performs as described by the human language description of the sample code further to: automatically detect a coding language from the sample code; and include the detected coding language in the prompts” recites further mental process. Furthermore, the following additional element “further comprising computer-executable instructions that when executed by at least the processor cause the computing system” merely recites instructions to implement an abstract idea on a generic computer, or merely uses a generic computer or computer components as a tool to perform the abstract idea, thus is not a practical application under Prong 2. See MPEP 2106.05(f). Accordingly, the claim is not patent eligible under 35 USC 101. Regarding to claim 11 , the limitation of “generate, as a component of the code generation loss function, a value of code matching loss that indicates an extent of dissimilarity between the sample code and the generated code” recites further mental process. Furthermore, the following additional element “wherein the computer-executable instructions further include instructions that when executed by at least the processor cause the computing system” merely recites instructions to implement an abstract idea on a generic computer, or merely uses a generic computer or computer components as a tool to perform the abstract idea, thus is not a practical application under Prong 2. See MPEP 2106.05(f). Accordingly, the claim is not patent eligible under 35 USC 101. Regarding to claim 12 , the limitation of “generate, as a component of the code generation loss function, a value of non-linear code completeness loss that indicates an extent to which the generated code fails to produce expected output on a line-by-line basis” recites further mental process. Furthermore, the following additional element “wherein the computer-executable instructions further include instructions that when executed by at least the processor cause the computing system” merely recites instructions to implement an abstract idea on a generic computer, or merely uses a generic computer or computer components as a tool to perform the abstract idea, thus is not a practical application under Prong 2. See MPEP 2106.05(f). Accordingly, the claim is not patent eligible under 35 USC 101. Regarding to claim 13 , the limitation of “generate, as a component of the code generation loss function, a value of unit test passing loss that indicates an extent to which the generated code fails to produce output consistent with the sample code” recites further mental process. Furthermore, the following additional element “wherein the computer-executable instructions further include instructions that when executed by at least the processor cause the computing system” merely recites instructions to implement an abstract idea on a generic computer, or merely uses a generic computer or computer components as a tool to perform the abstract idea, thus is not a practical application under Prong 2. See MPEP 2106.05(f). Accordingly, the claim is not patent eligible under 35 USC 101. Regarding to claim 14 , the limitation of “automatically generate one or more test cases based on Monte Carlo simulation of inputs to the sample code” recites further mental process. Furthermore, the following additional element “wherein the computer-executable instructions further include instructions that when executed by at least the processor cause the computing system” merely recites instructions to implement an abstract idea on a generic computer, or merely uses a generic computer or computer components as a tool to perform the abstract idea, thus is not a practical application under Prong 2. See MPEP 2106.05(f). Accordingly, the claim is not patent eligible under 35 USC 101. Regarding to claim 16 , the limitation “generating a value of recall between the sample code and the generated code; generating a value of precision between the sample code and the generated code; and combining the values of recall and precision to produce a value of code matching loss, wherein the value of code matching loss is a component of the code generation loss function” recites further mental process. The claim does not include any additional element, thus, no limitation that needs to be analyzed under prong 2 for practical application, or under step 2B for significantly more. Regarding to claim 17 , the limitation “generating a value of recall between the sample code and the generated code; generating a value of precision between the sample code and the generated code; and combining the values of recall and precision to produce a value of code matching loss, wherein the value of code matching loss is a component of the code generation loss function” recites further mental process. The claim does not include any additional element, thus, no limitation that needs to be analyzed under prong 2 for practical application, or under step 2B for significantly more. Regarding to claim18 , the limitation “accessing a test case associated with the sample code; recording line-by-line outputs of executing the sample code and the generated code on the test case; comparing the outputs of corresponding individual lines of the sample code and the generated code; where the outputs differ for a corresponding individual line, assign a loss amount based on a line number of the corresponding individual line; and sum the assigned loss amounts for the individual lines to produce a value of non-linear code completeness loss, wherein the value of non-linear code completeness loss is a component of the code generation loss function” recites further mental process. The claim does not include any additional element, thus, no limitation that needs to be analyzed under prong 2 for practical application, or under step 2B for significantly more. Regarding to claim 19 , the limitation “ accessing a set of test cases automatically generated for inputs to the sample code; for test cases in the set of test cases, comparing results of the executing the sample code on the test case and the executing the generated code on the test case, where the results differ, incrementing a tally of differing results; and determining a ratio of the tally of differing results to a count of the test cases to produce a value of unit test passing loss, wherein the value of unit test passing loss is a component of the code generation loss function” recites further mental process. Furthermore, the following additional elements “ execute the sample code and the generated code on the test case ” is merely applying the judicial exception or abstract idea. Accordingly, the claim is not patent eligible under 35 USC 101. Regarding to claim 20 , the limitation “accessing one or more test cases” recites further mental process. The claim does not include any additional element, thus, no limitation that needs to be analyzed under prong 2 for practical application, or under step 2B for significantly more. Allowable Subject Matter 8. Claims 1-20 would be allowable if rewritten or amended to overcome the objections and rejection(s) under 35 U.S.C. 101, set forth in this Office action. 9. The following is an Examiner’s statement of reasons for allowance: The prior arts of record or made of record, taken alone or in combination do not disclose and/or suggest, and/or motivation to combine, at least “generate prompts to an LLM to write code that performs as described by the human language description of the sample code; fine-tune a large language model to generate software code based on a code generation loss function that evaluates code generated by the LLM in response to the prompts; generate an evaluation score for performance of the tuned large language model as a code generator based on a value of the code generation loss function for second generated code; and automatically determine to deploy the tuned large language model to a production environment for code generation in response to the evaluation score satisfying a threshold” as limitations recited in as such manners as in independent claim 1. The prior arts of record or made of record, taken alone or in combination do not disclose and/or suggest, and/or motivation to combine, at least “ fine-tuning a large language model to generate software code based on a code generation loss function that evaluates code generated by the LLM in response to the prompts; generating an evaluation score for performance of the tuned large language model as a code generator based on a value of the code generation loss function for second generated code; and automatically signal that the fine tuning of the tuned large language model is complete in response to the evaluation score satisfying a threshold” as limitations recited in as such manners as in independent claim 10 or variants thereof in other independent claim 15. Conclusion 10. The prior art made of record and not relied upon (cited on 892 form) is considered pertinent to application disclosure. Deng et al. (US-11941373-B2) disclosed code generation via reinforcement learning using a reward score that considers the quality of the source code predicted during the tuning process. Gardner et al. (US-12008332-B1) disclosed summarization of content using large language models like GPT-3, Codex, PaLM etc., in a controlled and configurable manner. SHARPE et al. (US-20250068858-A1) disclosed generate summarization associated with set of inputs by providing Large Language Model with natural language phrases generated from conditionals and priority values. Benton et al. (US-20190129694-A1) disclosed Monte Carlo simulation and testing inputs. 11. Any inquiry concerning this communication or earlier communications from the examiner should be directed to MARINA LEE whose telephone number is (571)270-1648. The examiner can normally be reached Monday to Friday (8 am to 4: 30 pm ET). Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hyung S. Sough can be reached on (571)-272-6799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /MARINA LEE/Primary Examiner, Art Unit 2192 Application/Control Number: 18/743,866 Page 2 Art Unit: 2192 Application/Control Number: 18/743,866 Page 3 Art Unit: 2192 Application/Control Number: 18/743,866 Page 4 Art Unit: 2192 Application/Control Number: 18/743,866 Page 5 Art Unit: 2192 Application/Control Number: 18/743,866 Page 6 Art Unit: 2192 Application/Control Number: 18/743,866 Page 7 Art Unit: 2192 Application/Control Number: 18/743,866 Page 8 Art Unit: 2192 Application/Control Number: 18/743,866 Page 9 Art Unit: 2192 Application/Control Number: 18/743,866 Page 10 Art Unit: 2192 Application/Control Number: 18/743,866 Page 11 Art Unit: 2192 Application/Control Number: 18/743,866 Page 12 Art Unit: 2192 Application/Control Number: 18/743,866 Page 13 Art Unit: 2192 Application/Control Number: 18/743,866 Page 14 Art Unit: 2192 Application/Control Number: 18/743,866 Page 15 Art Unit: 2192 Application/Control Number: 18/743,866 Page 16 Art Unit: 2192 Application/Control Number: 18/743,866 Page 17 Art Unit: 2192 Application/Control Number: 18/743,866 Page 18 Art Unit: 2192
Prosecution Insights