DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Drawings
The drawings were received on 01/31/2024. These drawings are acceptable.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on the following date(s): 7/31/2026 has been considered by the examiner.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 3 and 14 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claims 3 and 14, the limitation "the second prompt" in “wherein the second prompt includes an indication of the error”. There is insufficient antecedent basis for this limitation in the claim. There is no prior recitation of a second prompt or what prompt the recited ‘the second prompt” refers too.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e. an abstract idea) without significantly more.
Claim 1: Dose claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
(Considered directed to a Mental Process: Making evaluations and judgements of observations for formulating observations, evaluations and judgements as claimed; see MPEP § 2106.04(a)(2), subsection III)
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
storing a set of positive training samples for training a first language model; (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to insignificant solution activity, e.g. Storing and retrieving information in memory)
wherein each correction training sample in the set includes an error from processing a faulty test of particular code (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation generally link the judicial exception to a particular technological environment or field of use. See 2106.05(h).)
training a second language model based on the set of correction training samples (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation merely include instructions to implement an abstract idea on a computer, or merely use a computer as a tool to perform an abstract idea; Thus claim limitations amount to mere instructions to apply the judicial exception using a computer/computing environment as a tool, as discussed in MPEP § 2106.05(f).)
receiving a first test, of code, that was generated by the first language model (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to insignificant solution activity, e.g. receiving or transmitting data over a network)
inputting the first correction prompt into the second language model that outputs a second test that is a corrected version of the first test (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to insignificant solution activity, e.g. receiving or transmitting data over a network)
wherein the method is performed by one or more computing devices. (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation merely recites a particularity of the application of the judicial exception, as discussed in MPEP § 2106.05(f).)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application.
First, the additional limitations are directed to elements that generally link the use of a judicial exception to a particular technological environment or field of use and merely invoke the use of computer technology as a tool for applying the judicial exception; and that generally link the use of a judicial exception to a particular technological environment and/or directed to invoking computers or other machinery merely as a tool to perform the claimed process/judicial exception.
Secondly, the limitations directed to insufficient to transform the judicial exception to a patentable invention because the recitation is directed to insignificant solution activity for as noted above. The courts have deemed these types of activity as well-known routine and convectional, see evidences noted below:
Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network); but see DDR Holdings, LLC v. Hotels.com, L.P., 773 F.3d 1245, 1258, 113 USPQ2d 1097, 1106 (Fed. Cir. 2014) ("Unlike the claims in Ultramercial, the claims at issue here specify how interactions with the Internet are manipulated to yield a desired result‐‐a result that overrides the routine and conventional sequence of events ordinarily triggered by the click of a hyperlink." (emphasis added));
Storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93;
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 2: Dose claim fall within a statutory category? Yes
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
Recites the abstract idea of claim 1.
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
wherein the first language model is the same as the second language model. (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to generally linking the judicial exception to a particular technological environment or field of use. See 2106.05(h).)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application.
First, the additional limitations are directed to elements that generally link the judicial exception to a particular technological environment.
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 3: Dose claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
Recites the abstract idea of claim 1.
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
wherein the second prompt includes an indication of the error. (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation generally links the judicial exception to a particular technological environment or field of use. See 2106.05(h).)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application.
First, the additional limitations are directed to elements that generally link the use of a judicial exception to a particular technological environment or field of use and merely invoke the use of computer technology as a tool for applying the judicial exception.
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 4: Dose claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
further comprising: determining whether the second result indicates an error in processing the second result; in response to determining that the second result indicates an error in processing the second test against the code, generating, based on the second result, a second correction prompt that is different than the first correction prompt; . (Considered directed to a Mental Process: Making evaluations and judgements of observations for formulating observations, evaluations and judgements as claimed)
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
inputting the third correction prompt into the second language model that outputs a third test that is a corrected version of the second test (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to insignificant solution activity, e.g. receiving or transmitting data over a network)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application. Specifically, the additional limitations are directed to elements that generally link the judicial exception to a particular technological environment or field of use.
Secondly, the limitations directed to insufficient to transform the judicial exception to a patentable invention because the recitation is directed to insignificant solution activity for as noted above. The courts have deemed these types of activity as well-known routine and convectional, see evidences noted below:
Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network); but see DDR Holdings, LLC v. Hotels.com, L.P., 773 F.3d 1245, 1258, 113 USPQ2d 1097, 1106 (Fed. Cir. 2014) ("Unlike the claims in Ultramercial, the claims at issue here specify how interactions with the Internet are manipulated to yield a desired result‐‐a result that overrides the routine and conventional sequence of events ordinarily triggered by the click of a hyperlink." (emphasis added));
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 5: Dose claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
further comprising: determining a number of tests that were processed against the code and that resulted in an error; in response to determining that the number of tests is equal to a threshold number, determining to not generate any more tests for the code. (Considered directed to a Mental Process: Making evaluations and judgements of observations for formulating observations, evaluations and judgements as claimed)
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application.
The additional limitations are directed to elements that generally link the judicial exception to a particular technological environment.
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 6: Dose claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
further comprising: determining whether the second result indicates an error in processing the second result; in response to determining that the second result does not indicate an error in processing the second test; (Considered directed to a Mental Process: Making evaluations and judgements of observations for formulating observations, evaluations and judgements as claimed; see MPEP § 2106.04(a)(2), subsection III)
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
storing data that indicates that the second test is a valid test. (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to insignificant solution activity, e.g. Storing and retrieving information in memory)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application.
First, the additional limitations are directed to elements that generally link the judicial exception to a particular technological environment or field of use and merely invoke the use of computer technology as a tool for applying the judicial exception.
Secondly, the limitations directed to insufficient to transform the judicial exception to a patentable invention because the recitation is directed to insignificant solution activity for as noted above. The courts have deemed these types of activity as well-known routine and convectional, see evidences noted below:
Storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93;
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 7: Dose claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
further comprising, (Considered directed to a Mental Process: Making evaluations and judgements of observations for formulating observations, evaluations and judgements as claimed; see MPEP § 2106.04(a)(2), subsection III)
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
further comprising, prior to receiving the first test: (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation merely recites a particularity of the application of the judicial exception, as discussed in MPEP § 2106.05(f).)
calling the function with the input data, wherein calling the function results in output from the function (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation merely include instructions to implement an abstract idea on a computer, or merely use a computer as a tool to perform an abstract idea; Thus claim limitations amount to mere instructions to apply the judicial exception using a computer/computing environment as a tool, as discussed in MPEP § 2106.05(f).)
Alternatively: calling the function with the input data, wherein calling the function results in output from the function (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to insignificant solution activity, e.g. receiving or transmitting data over a network)
positive training sample that comprises the function, the input data, and the output; (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation generally links the judicial exception to a particular technological environment or field of use. See 2106.05(h).)
adding the positive training sample to the set of positive training samples. (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation merely recites a particularity of the application of the judicial exception, as discussed in MPEP § 2106.05(f).)
Alternatively adding the positive training sample to the set of positive training samples. (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation is directed to insignificant solution activity, e.g. Storing and retrieving information in memory)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application.
First, the additional limitations are directed to elements that generally link the judicial exception to a particular technological environment or field of use and merely invoke the use of computer technology as a tool for applying the judicial exception; and that generally link the use of a judicial exception to a particular technological environment and/or directed to invoking computers or other machinery merely as a tool to perform the claimed process/judicial exception.
Secondly, the limitations directed to insufficient to transform the judicial exception to a patentable invention because the recitation is directed to insignificant solution activity for as noted above. The courts have deemed these types of activity as well-known routine and convectional, see evidences noted below:
Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network); but see DDR Holdings, LLC v. Hotels.com, L.P., 773 F.3d 1245, 1258, 113 USPQ2d 1097, 1106 (Fed. Cir. 2014) ("Unlike the claims in Ultramercial, the claims at issue here specify how interactions with the Internet are manipulated to yield a desired result‐‐a result that overrides the routine and conventional sequence of events ordinarily triggered by the click of a hyperlink." (emphasis added));
Storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93;
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 8 Dose claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
Abstract idea noted in claim 1.
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
wherein each correction training sample in the set of correction training samples comprises particular code to be tested, a mutated test that resulted in a particular error, a description of the particular error, and a corrected test that is a modified version of the mutated test.. (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation simply link the judicial exception to a field of use and/or technology environment, see MPEP 2106.05(h).)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application.
Specifically, the additional limitations directed to elements that generally link the judicial exception to a particular technological environment or field of use.
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 9: Dose claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
wherein generating the set of correction training samples comprises, given a positive training sample, in the set of positive training samples, that includes particular code: mutating one or more test inputs of a valid test in the positive training sample to generate mutated data; (Considered directed to a Mental Process: Making evaluations and judgements of observations for formulating observations, evaluations and judgements as claimed; see MPEP § 2106.04(a)(2), subsection III)
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
executing the particular code with the mutated data, which executing results in generation of a particular error; (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation merely recites a particularity of the application of the judicial exception, as discussed in MPEP § 2106.05(f).)
including, in a correction training sample, the mutated data, the particular error, and the valid test; (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation generally links the judicial exception to a particular technological environment or field of use. See 2106.05(h).)
finetuning the second language model based on the correction training sample. (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation merely include instructions to implement an abstract idea on a computer, or merely use a computer as a tool to perform an abstract idea; Thus claim limitations amount to mere instructions to apply the judicial exception using a computer/computing environment as a tool, as discussed in MPEP § 2106.05(f).)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application.
Specifically, the additional limitations directed to elements that generally link the judicial exception to a particular technological environment or field of use.
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Claim 10: Dose claim fall within a statutory category? Yes.
Step 2A Prong 1: Evaluate whether the claim recites a judicial exception.
wherein generating the set of correction training samples comprises, given a positive training sample, in the set of positive training samples, that includes particular code: mutating one or more test outputs of a valid test in the positive training sample to generate a mutated test; (Considered directed to a Mental Process: Making evaluations and judgements of observations for formulating observations, evaluations and judgements as claimed; see MPEP § 2106.04(a)(2), subsection III)
Step 2A Prong 2: Evaluate whether the claim as a whole integrates the recited judicial exception into a practical application of the exception
The preamble is deemed insufficient to transform the judicial exception to a patentable invention because the preamble generally links the use of a judicial exception to a particular technological environment or field of use, see MPEP 2106.05(h).
executing the particular code based on the mutated test, which executing results in a failure of the mutated test; (Deemed insufficient to transform the judicial exception to a patentable invention because the claim generically recites an effect of the judicial exception or claims every mode of accomplishing that effect; thus, the claim amounts to a claim that is merely adding the words "apply it" to the judicial exception. See Internet Patents Corporation v. Active Network, Inc., 790 F.3d 1343, 1348, 115 USPQ2d 1414, 1418 (Fed. Cir. 2015), as discussed in MPEP § 2106.05(f).
including, in a correction training sample, the mutated test, a description of the failure, and the valid test; (Deemed insufficient to transform the judicial exception to a patentable invention because the recitation generally links the judicial exception to a particular technological environment or field of use. See 2106.05(h).)
The additional elements do not appear to be sufficient to transform the judicial exception into a practical application at Step 2A as analyzed above.
Step 2B: Evaluates whether the claim as a whole/in combination integrates the recited judicial exception into a practical application of the exception
The claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception and fail to integrate the abstract into practical application.
The additional limitations are directed to elements that generally link the use of a judicial exception to a particular technological environment or field of use and merely invoke the use of computer technology as a tool for applying the judicial exception; and that generally link the use of a judicial exception to a particular technological environment and/or directed to invoking computers or other machinery merely as a tool to perform the claimed process/judicial exception.
These types of claimed elements cannot transform the judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. This claim is not patent eligible.
Regarding claims 11-12, the claim limitations are similar to claim 1 limitations and thus rejected under the same rationale.
Regarding claims 13-20, the claim limitations are similar to claim 2-9 respectively and thus rejected under the same rationale.
As shown above, claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed a judicial exception and does not recite, when claim elements are examined individually and as a whole, elements that the courts have identified as "significantly more” than the recited judicial exception. The claims are therefore directed to an abstract idea.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Singh et al. (US 11899566, hereinafter ‘Sin’) in view of Zhang et al. (US 20240256423, ‘Zhang’)
Regarding independent claim 1, Sin teaches a method comprising: storing a set of positive training samples for training a first language model; (in 2:21-37: In some implementations, a method implemented by one or more processors is provided that includes identifying a plurality of ground truth source code, unit test pairs each including a corresponding ground truth source code unit, and a corresponding ground truth unit test for the ground truth source code unit. The method further includes, for each of the ground truth source code, unit test pairs, processing the corresponding ground truth source code unit, using a code-to-embedding machine learning model, to generate one or more corresponding code unit embeddings, processing the corresponding ground truth unit test, using the code-to-embedding machine learning model, to generate one or more corresponding unit test embeddings, and generating a corresponding positive training instance [storing a set of positive training samples for training a first language model] that includes the one or more corresponding code unit embeddings as input, and the one or more corresponding unit test embeddings as output…)
based on the set of positive training samples, generating a set of correction training samples, wherein each correction training sample in the set includes an error from processing a faulty test of particular code; training a second language model based on the set of correction training samples; (in 2:38-47: The method further includes, for each of the corresponding positive training instances, processing the corresponding one or more code unit embeddings of the input, using a code embedding-to-test embedding machine learning model, to generate one or more predicted unit test embeddings [based on the set of positive training samples, generating a set of correction training samples], and generating a corresponding error based on comparing the one or more predicted unit test embeddings to the one or more corresponding unit test embeddings of the output. The method further includes training the code embedding-to-test embedding machine learning model [training a second language model based on the set of correction training samples] based on the corresponding errors [wherein each correction training sample in the set includes an error from processing a faulty test of particular code].)
receiving a first test, of code, that was generated by the first language model; generating a first result of processing the first test against the code; (in 2:52-64: In some implementations, the method further includes, subsequent to training the code embedding-to-test embedding machine learning model, identifying a given source code unit [receiving a first test, of code, that was generated by the first language model], processing the given source code unit, using the code-to-embedding machine learning model, to generate one or more given source code unit embeddings, processing the given source code unit embedding, using the code embedding-to-test embedding machine learning model, to generate one or more given predicted unit test embeddings [generating a first result of processing the first test against the code], using the one or more given predicted unit test embeddings to identify a given unit test for the given source code unit, and evaluating the given source code unit using at least the given unit test.)
in response to determining that the first result indicates an error in processing the first test against the code, generating, based on the first result, a first correction prompt; (in 10:39-53: The generated predicted unit tests 114 can be used by the evaluation engine 160 in testing the source code unit 111 [in response to determining that the first result indicates an error in processing the first test against the code, generating, based on the first result, a first correction prompt]. Further, based on that testing, the evaluation engine can generate evaluation result(s) 115. The evaluation result(s) 115 can graphically and/or audibly convey various metric(s) such as whether the unit test(s) were successful, a number of failure(s) from the unit test(s), details on the failure(s), a number of success(es) from the unit test(s), and/or other metrics [in response to determining that the first result indicates an error in processing the first test against the code, …]. The evaluation result(s) can be caused to be rendered, for example, on a client device (not illustrated) of a developer. For instance, it can be rendered in an interface of a development application to enable the developer to efficiently ascertain whether the source code unit 111 is robust and/or accurate or, instead, requires revision(s) to ensure its accuracy [generating, based on the first result, a first correction prompt]. )
inputting the first correction prompt into the second language model that outputs a second test that is a corrected version of the first test; (2:65-3:3: In some versions of those implementations, using the one or more given predicted unit test embeddings to identify the given unit test for the given source code unit includes processing the one or more given predicted unit test embeddings, using an embedding-to-code machine learning model, to generate the given unit test [a corrected version of the first test] for the given source code unit [inputting the first correction prompt into the second language model that outputs a second test that is a corrected version of the first test]… And in 3:19-22: … In yet further versions of those implementations, the method further includes using the one or more given predicted unit test embeddings to identify an additional given unit test [inputting the first correction prompt into the second language model that outputs a second test that is a corrected version of the first test] for the given source code unit. Evaluating the given source code unit is further using at least additional given unit test, and identifying the additional given unit test includes generating, based on the corresponding probability distributions of the sequence of outputs, the additional given unit test. The additional given unit test includes one or more portions that differ from the given unit test based [ … a second test that is a corrected version of the first test] on the one or more portions being generated based on non-highest probabilities in the corresponding probability distributions of the sequence of outputs)
generating a second result of processing the second test; (in 3:22-31: Evaluating the given source code unit is further using at least additional given unit test, and identifying the additional given unit test includes generating, based on the corresponding probability distributions of the sequence of outputs, the additional given unit test [generating a second result of processing the second test]. The additional given unit test includes one or more portions that differ from the given unit test based on the one or more portions being generated based on non-highest probabilities in the corresponding probability distributions of the sequence of outputs.)
wherein the method is performed by one or more computing devices. (in 7:18-34: In addition, some implementations include one or more processors (e.g., CPU(s), GPU(s), and/or TPU(s)) of one or more computing devices [wherein the method is performed by one or more computing devices], where the one or more processors are operable to execute instructions stored in associated memory, and where the instructions are configured to cause performance of any of the methods disclosed herein. Some implementations also include one or more non-transitory computer readable storage media storing computer instructions executable by one or more processors to perform any of the methods disclosed herein. It should be appreciated that all combinations of the foregoing concepts and additional concepts described in greater detail herein are contemplated as being part of the subject matter disclosed herein. For example, all combinations of claimed subject matter appearing at the end of this disclosure are contemplated as being part of the subject matter disclosed herein)
While Sin teaches the machine learning modeling system for automated generation of test code using programing code data as claimed respective language model, Sin does not expressly recite the term language model.
Zhang expressly recites the term language models, in [0040] Some embodiments herein utilize or include language models of various sizes, e.g., language models [first language model … second language model … ] having at least one billion parameters, at least ten billion parameters, or at least one hundred billion parameters, or language models having less than one billion parameters, less than ten billion parameters, or less than one hundred billion parameters language... Models may be large models or edge models, and may be fine-tuned models or not fine-tuned, depending on the embodiment… [0041] Upon execution of the syntactic phase code transformer 306 by the processor set, the syntax checker identifies 1112 a syntax error 904 in a first version 210 of a source code 210 [storing a set of positive training samples for training a first language model], 130, the program chunker extracts 1104 a code chunk 920 from the first version of the source code, the code chunk including the syntax error, the syntactic prompt generator receives 1114 the code chunk and produces 1116 a syntactic prompt 906 which contains at least the syntax error [based on the set of positive training samples, generating a set of correction training samples, wherein each correction training sample in the set includes an error from processing a faulty test of particular code], and the model interface receives 1008 the syntactic prompt and produces 1010 at least a portion of the second version of the source code in which the syntax error has been repaired. …
Zhang and Sin are analogous art because both involve developing information retrieval and data processing techniques using machine learning systems and algorithms.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of the prior art for implementing data processing techniques for software development using language models as disclosed by Zhang with the method of for training machine learning models in automatically generating test cases for source code as disclosed by Sin.
One of ordinary skill in the arts would have been motivated to combine the methods disclosed by Zhang and Sin, as noted above. Doing so allows using an language models to remove the need for custom symbolic repair logic or retraining of a new neural model, and enables techniques that can handle both syntactic and semantic mistakes, (Zhang, 0083).
Regarding claim 2, the rejection of claim 1 is incorporated and Sin in combination with Zhang teaches the method of claim 1, wherein the first language model is the same as the second language model. (in 1:64-2:3: Implementations disclosed herein relate to training and/or utilization of machine learning model(s) (e.g., neural network model(s) [wherein the first language model is the same as the second language model as same model type]) in automatically generating test case(s) for source code. Techniques disclosed herein can be utilized in generating test case(s) for unit test testing (or other white-box testing) and/or for functional testing (or other black-box testing).)
Regarding claim 3, the rejection of claim 1 is incorporated and Sin in combination with Zhang teaches the method of claim 1, wherein the second prompt includes an indication of the error. (in 2:12-47: … using the code-to-embedding machine learning model, to generate one or more corresponding unit test embeddings, and generating a corresponding positive training instance that includes the one or more corresponding code unit embeddings as input, and the one or more corresponding unit test embeddings as output. The method further includes, for each of the corresponding positive training instances, processing the corresponding one or more code unit embeddings of the input, using a code embedding-to-test embedding machine learning model, to generate one or more predicted unit test embeddings, and generating a corresponding error [wherein the second prompt includes an indication of the error] based on comparing the one or more predicted unit test embeddings to the one or more corresponding unit test embeddings of the output. The method further includes training the code embedding-to-test embedding machine learning model based on the corresponding errors.)
Regarding claim 4, the rejection of claim 1 is incorporated and Sin in combination with Zhang teaches the method of claim 1, further comprising: determining whether the second result indicates an error in processing the second result; in response to determining that the second result indicates an error in processing the second test against the code, generating, based on the second result, a second correction prompt that is different than the first correction prompt; (in 3:22-31: … Evaluating the given source code unit is further using at least additional given unit test [determining whether the second result indicates an error in processing the second result], and identifying the additional given unit test includes generating, based on the corresponding probability distributions of the sequence of outputs, the additional given unit test. The additional given unit test includes one or more portions that differ from the given unit test [in response to determining that the second result indicates an error in processing the second test against the code, generating, based on the second result, a second correction prompt that is different than the first correction prompt] based on the one or more portions being generated based on non-highest probabilities in the corresponding probability distributions of the sequence of outputs. And in 1:38-47: The method further includes, for each of the corresponding positive training instances, processing the corresponding one or more code unit embeddings of the input, using a code embedding-to-test embedding machine learning model, to generate one or more predicted unit test embeddings, and generating a corresponding error based on comparing the one or more predicted unit test embeddings to the one or more corresponding unit test embeddings of the output. The method further includes training the code embedding-to-test embedding machine learning model based on the corresponding errors [in response to determining that the second result indicates an error in processing the second test against the code, generating, based on the second result, a second correction prompt that is different than the first correction prompt].)
inputting the third correction prompt into the second language model that outputs a third test that is a corrected version of the second test. (in 4:18-24: … Determining, based on the evaluating, whether to render the given source code unit in the development application as the suggested translation includes determining, based on the evaluating, whether to render the given source code unit, or an alternate one of the automated translations [inputting the third correction prompt into the second language model that outputs a third test that is a corrected version of the second test], as a suggested translation of the corresponding source code unit.)
Additionally, Zhang teaches in [0008] FIG. 2 is a diagram illustrating aspects of a computing environment and an enhanced system configured with software code improvement functionality which leverages a trained machine learning model; … [0054] Some embodiments provide or utilize a process to improve a first version of a source code [inputting the third correction prompt into the second language model that outputs a third test that is a corrected version of the second test], the process performed (executed) by a computing system 202, the process including: repairing 1002 any syntax errors in the first version of the source code or confirming 1004 that the first version of the source code is free of syntax errors, or both, thereby yielding a syntactically correct version of the source code; generating 1006 a multimodal prompt which includes at least two of the following: a chunk 920 or other portion of the syntactically correct version of the source code, a natural language description 706 of a task to be accomplished by any improved version of the source code, or a test case 704 to be satisfied by the improved version of the source code; submitting 1008 the multimodal prompt to a large language machine learning model trained on source codes (LLMC) 208; obtaining 1010 candidate versions [inputting the third correction prompt into the second language model that outputs a third test that is a corrected version of the second test] of the improved version of the source code from the LLMC;)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Zhang and Sin for the same reasons disclosed above.
Regarding claim 5, the rejection of claim 4 is incorporated and Zhang teaches the method of claim 4, further comprising: determining a number of tests that were processed against the code and that resulted in an error; in response to determining that the number of tests is equal to a threshold number, determining to not generate any more tests for the code. (in [0114] If a program candidate has no syntax errors, it can move on to the semantic phase. If any syntax errors remain, the syntax phase is repeated on this candidate program. This iteration allows the repair of multiple, spatially-independent, syntax errors. In the evaluation, this procedure iterated at most two times, to limit repair times [determining a number of tests that were processed against the code and that resulted in an error; in response to determining that the number of tests is equal to a threshold number, determining to not generate any more tests for the code].)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Zhang and Sin for the same reasons disclosed above.
Regarding claim 6, the rejection of claim 1 is incorporated and in combination with Zhang teaches the method of claim 1, further comprising: determining whether the second result indicates an error in processing the second result; in response to determining that the second result does not indicate an error in processing the second test; storing data that indicates that the second test is a valid test. (in 10:39-49: The generated predicted unit tests 114 can be used by the evaluation engine 160 in testing the source code unit 111 [determining whether the second result indicates an error in processing the second result]. Further, based on that testing, the evaluation engine can generate evaluation result(s) 115. The evaluation result(s) 115 can graphically and/or audibly convey various metric(s) such as whether the unit test(s) were successful [in response to determining that the second result does not indicate an error in processing the second test; storing data that indicates that the second test is a valid test], a number of failure(s) from the unit test(s), details on the failure(s) [determining whether the second result indicates an error in processing the second result], a number of success(es) from the unit test(s) [in response to determining that the second result does not indicate an error in processing the second test; storing data that indicates that the second test is a valid test], and/or other metrics. The evaluation result(s) can be caused to be rendered, for example, on a client device (not illustrated) of a developer...)
Regarding claim 7, the rejection of claim 1 is incorporated and Sin in combination with Zhang teaches the method of claim 1, further comprising, prior to receiving the first test: sampling input data from an input space associated with a function in the code; calling the function with the input data, wherein calling the function results in output from the function; (in 8:13-21: … The ground truth source code unit 101 can be, for example, a method (function) in a class or other code unit [further comprising, prior to receiving the first test: sampling input data from an input space associated with a function in the code; calling the function with the input data, wherein calling the function results in output from the function]. The ground truth unit test 104 is one that is generated (e.g., by a human developer) for unit testing of the ground truth source code unit 101. The ground truth unit test 104 can itself include source code. The ground truth source code unit 101 and ground truth unit test 104 can be considered a pair since the unit test 104 is specifically generated for the ground truth source code unit…)
generating a positive training sample that comprises the function, the input data, and the output; adding the positive training sample to the set of positive training samples. (in 2:27-45: …The method further includes, for each of the ground truth source code, unit test pairs, processing the corresponding ground truth source code unit [that comprises the function], using a code-to-embedding machine learning model, to generate one or more corresponding code unit embeddings, processing the corresponding ground truth unit test, using the code-to-embedding machine learning model, to generate one or more corresponding unit test embeddings [wherein calling the function results in output from the function], and generating a corresponding positive training instance [generating a positive training sample that comprises the function, the input data, and the output; adding the positive training sample to the set of positive training samples] that includes the one or more corresponding code unit embeddings as input, and the one or more corresponding unit test embeddings as output. The method further includes, for each of the corresponding positive training instances, processing the corresponding one or more code unit embeddings of the input, using a code embedding-to-test embedding machine learning model, to generate one or more predicted unit test embeddings, and generating a corresponding error based on comparing the one or more predicted unit test embeddings to the one or more corresponding unit test embeddings of the output…)
Regarding claim 8, the rejection of claim 1 is incorporated and Sin in combination with Zhang teaches the method of claim 1, wherein each correction training sample in the set of correction training samples comprises particular code to be tested, a mutated test that resulted in a particular error, a description of the particular error, and a corrected test that is a modified version of the mutated test. (in 4:15-24: … In some further versions of those implementations, the given source code unit is one of multiple candidate automated translations of the corresponding source code unit [wherein each correction training sample in the set of correction training samples comprises particular code to be tested, a mutated test that resulted in a particular error, a description of the particular error]. Determining, based on the evaluating, whether to render the given source code unit in the development application as the suggested translation includes determining, based on the evaluating, whether to render the given source code unit, or an alternate one of the automated translations [wherein each correction training sample in the set of correction training samples comprises particular code to be tested, a mutated test that resulted in a particular error, a description of the particular error, and a corrected test that is a modified version of the mutated test], as a suggested translation of the corresponding source code unit. And in 2:65-3:3: In some versions of those implementations, using the one or more given predicted unit test embeddings to identify the given unit test for the given source code unit includes processing the one or more given predicted unit test embeddings, using an embedding-to-code machine learning model, to generate the given unit test [and a corrected test that is a modified version of the mutated test] for the given source code unit … And in 3:19-22: … In yet further versions of those implementations, the method further includes using the one or more given predicted unit test embeddings to identify an additional given unit test [and a corrected test that is a modified version of the mutated test] for the given source code unit. Evaluating the given source code unit is further using at least additional given unit test, and identifying the additional given unit test includes generating, based on the corresponding probability distributions of the sequence of outputs [a mutated test that resulted in a particular error, a description of the particular error], the additional given unit test. The additional given unit test includes one or more portions that differ from the given unit test based [ … and a corrected test that is a modified version of the mutated test] on the one or more portions being generated based on non-highest probabilities in the corresponding probability distributions of the sequence of outputs) … 10:39-45: The generated predicted unit tests 114 can be used by the evaluation engine 160 in testing the source code unit 111. Further, based on that testing, the evaluation engine can generate evaluation result(s) 115. The evaluation result(s) 115 can graphically and/or audibly convey various metric(s) such as whether the unit test(s) were successful, a number of failure(s) from the unit test(s), details on the failure(s) [a mutated test that resulted in a particular error, a description of the particular error], …)
Zhang teaches wherein each correction training sample in the set of correction training samples comprises particular code to be tested, a mutated test that resulted in a particular error, a description of the particular error, and a corrected test that is a modified version of the mutated test. (in [0020] Some embodiments taught herein include both a syntactic phase code transformer and a semantic phase code transformer. The syntactic phase code transformer produces a version of a source code in which all syntax errors (if any) have been repaired. The semantic phase code transformer then produces a version of the source code in which all semantic errors (if any) have been mitigated [wherein each correction training sample in the set of correction training samples comprises particular code to be tested, a mutated test that resulted in a particular error, a description of the particular error, and a corrected test that is a modified version of the mutated test]. )
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Zhang and Sin for the same reasons disclosed above.
Regarding claim 9, the rejection of claim 1 is incorporated and Sin in combination with Zhang teaches the method of claim 1, wherein generating the set of correction training samples comprises, given a positive training sample, in the set of positive training samples, that includes particular code: mutating one or more test inputs of a valid test in the positive training sample to generate mutated data; (in 2:65-3:31: n some versions of those implementations, using the one or more given predicted unit test embeddings to identify the given unit test for the given source code unit includes processing the one or more given predicted unit test embeddings, using an embedding-to-code machine learning model, to generate the given unit test for the given source code unit… In yet further versions of those implementations, the method further includes using the one or more given predicted unit test embeddings to identify an additional given unit test for the given source code unit. Evaluating the given source code unit is further using at least additional given unit test, and identifying the additional given unit test includes generating, based on the corresponding probability distributions of the sequence of outputs, the additional given unit test. The additional given unit test includes one or more portions that differ from the given unit test [mutating one or more test inputs of a valid test in the positive training sample to generate mutated data] based on the one or more portions being generated based on non-highest probabilities in the corresponding probability distributions of the sequence of outputs.)
executing the particular code with the mutated data, which executing results in generation of a particular error; including, in a correction training sample, the mutated data, the particular error, and the valid test; finetuning the second language model based on the correction training sample. (in 2:37-58: The method further includes, for each of the corresponding positive training instances, processing the corresponding one or more code unit embeddings of the input, using a code embedding-to-test embedding machine learning model, to generate one or more predicted unit test embeddings, and generating a corresponding error based on comparing the one or more predicted unit test embeddings to the one or more corresponding unit test embeddings of the output. The method further includes training the code embedding-to-test embedding machine learning model based on the corresponding errors [executing the particular code with the mutated data, which executing results in generation of a particular error; including, in a correction training sample, the mutated data, the particular error, and the valid test; finetuning the second language model based on the correction training sample]... 10:39-45: The generated predicted unit tests 114 can be used by the evaluation engine 160 in testing the source code unit 111. Further, based on that testing, the evaluation engine can generate evaluation result(s) 115. The evaluation result(s) 115 can graphically and/or audibly convey various metric(s) such as whether the unit test(s) were successful, a number of failure(s) from the unit test(s), details on the failure(s) [which executing results in generation of a particular error; including, in a correction training sample, the mutated data, the particular error, and the valid test],…)
Additionally, Zhang teaches finetuning the second language model based on the correction training sample, in [0040] ... Some embodiments herein utilize or include language models trained on at least ten gigabytes of data, or at least one hundred gigabytes of data, or at least one terabyte of data, or language models trained on at most ten gigabytes of data, or at most one hundred gigabytes of data, or at most one terabyte of data. Models may be large models or edge models, and may be fine-tuned models [finetuning the second language model based on the correction training sample,] … And as depicted in Fig 7.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Zhang and Sin for the same reasons disclosed above.
Regarding claim 10, the limitations are similar to those in claim 9 and rejected under the same rationale.
Regarding claims 11-12, the claim limitations are similar to claim 1 limitations and thus rejected under the same rationale.
Regarding claims 13-20, the claim limitations are similar to claim 2-9 respectively and thus rejected under the same rationale.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Kumar et al. (US 20220091968): teaches in 0002: … automated validation testing of a software application during the development process or at the end of the development process, and more particularly the present invention relates to the use of machine learning to generate test cases and test data from product specification for the validation testing of the software application.
Clement et al. (US 12321735): teaches in 1:36-47: a neural transcompilation model that translates source code of a source programming language into source code of a different, target programming language is tested with a set of syntax unit tests to determine the syntax elements of the source programming language that fail to translate properly in a target programming language. The neural transcompilation model is then fine-tuned with training samples of the syntax elements having the highest failure rate and their paired correct translation in order to teach the model to learn the association between the poorly understood syntax element and its correct translation in the target programming language.
Peng et al. (US 20250362885): teaches in 0018: the code generation framework includes a large language model (LLM) and a code execution environment, such as a hardware or software based simulator, and/or the like. First, the LLM may receive an input prompt comprising a text description (e.g., of an issue, or a function a code is intended to achieve) and an instruction for the LLM to generate a candidate code snippet output. The generated candidate code snippet snippet is passed to an execution environment, which generates a first execution feedback regarding the correctness of the candidate code snippet snippet, compared with the original problem description. The first execution feedback is used by the LLM to evaluate whether the candidate code snippet snippet needs to be amended to pass a correctness test. Based on the first execution feedback, a second input prompt combining the original problem description, the candidate code snippet output, and the first execution feedback to the LLM to generate an refined code snippet.
Yoshida et al. (US 20170161182): teaches in 0004: a method for identifying a fault location in a software program using a test suite. The method may further include determining, using machine learning, a repair effectiveness indication that indicates a potential effectiveness of performing a potential repair operation at the fault location. In addition, the method may include prioritizing implementing a repair at the fault location based on the repair effectiveness indication. The method may further include performing repair operations with respect to the software program based on the prioritization of the fault location.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to OLUWATOSIN ALABI whose telephone number is (571)272-0516. The examiner can normally be reached Monday-Friday, 8:00am-5:00pm EST..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael Huntley can be reached at (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/OLUWATOSIN ALABI/ Primary Examiner, Art Unit 2129
Prosecution Insights