Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendment filed May 8th, 2026 has been entered. Claims 1-20 remain pending in the application. The applicant’s amendments to the claims have overcome all claim objections previously set forth in the Non-Final Office Action of March 9th, 2026.
Claim Objections
Claims 1-8 are objected to because of the following informalities:
Claim 1, line 8, “the program” lacks proper antecedent basis.
Claims 2-8 are objected to for being dependent upon claim 1.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 6 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 6, line 2, “the program” is unclear whether it is referring to “the program” in line 8 or “a program” in line 15 of claim 1. For examination purposes, “the program” in line 2 of claim 6 will be treated as referencing “the program” cited in claim 1 line 8.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claims 1, 9, and 17:
Step 1:
Claims 1-8 are directed to a process, claims 9-16 are directed to a system, and claims 17-20 are directed to a non-transitory computer readable medium. Thus, they fall into statutory categories.
Step 2A, Prong 1:
Claims 1 recites:
generating, using one or more pre-trained large language models (LLMs), a plurality of sub-modules from a problem description in a natural language, wherein the problem description describes a programming task and includes test cases comprising input and output pairs, wherein each sub-module in the plurality of sub-modules comprises source code representing a sub-task of the problem description
generating new sub-modules from the plurality of sub-modules over a plurality of iterations until the program generates expected results specified by the output pairs for the problem description
grouping the plurality of sub-modules into a plurality of clusters
selecting representative sub-modules from the plurality of clusters
augmenting the problem description with the representative sub-modules
generating, using the one or more pre-trained LLMs, the new sub-modules from the augmented problem description
generating a program from the new sub-modules
Steps (a), (b), (c), (d), (e), (f), and (g) are mental processes that can be performed in the human mind or by a human using pen and paper.
Claim 9 recites:
receiving a problem description in a natural language, wherein the problem description describes a programming task and includes test cases comprising input and output pairs
generating, using the one or more pre-trained LLMs, a plurality of sub-modules from the problem description, wherein each sub-module in the plurality of sub-modules comprises source code representing a sub-task of the problem description
encoding, using the one or more pre-trained LLMs, the plurality of sub-modules into sub-module encodings in an embedding space
clustering the sub-module encodings into a plurality of clusters
selecting a subset of sub-module encodings from the plurality of clusters
generating representative sub-modules from the subset of sub-module encodings
generating, using the one or more pre-trained LLMs, the problem description and the representative sub-modules, new sub-modules
Steps (a), (b), (c), (d), (e), (f), and (g) are mental processes that can be performed in the human mind or by a human using pen and paper.
Claim 17 recites:
receiving a problem description in a natural language
generating, using one or more pre-trained large language models (LLMs), a plurality of sub-modules from the problem description, wherein each sub-module in the plurality of sub-modules comprises source code representing a sub-task of the problem description
generating new sub-modules from the plurality of sub-modules over a plurality of iterations until a program generates expected results specified in the problem description
grouping the plurality of sub-modules into a plurality of clusters
selecting representative sub-modules from the plurality of clusters
augmenting the problem description with the representative sub-modules
generating, using the one or more pre-trained LLMs, the new sub-modules from the augmented problem description.
generating the program from the new sub-modules
Steps (a), (b), (c), (d), (e), (f), (g), and (h) are mental processes that can be performed in the human mind or by a human using pen and paper.
Step 2A Prong 2:
Claims recite additional elements, “LLMs” in claim 1; “a memory”, “processor”, and “LLMs” in claim 9; and “medium”, “a processor” and “LLMs” in claim 17. However, they are recited at a high level of generality. Thus, the claims as a whole do not integrate the exception into a practical application.
Step 2B:
The additional elements, considering them both individually and in combination, do not amount to significantly more than the judicial exception itself.
Regarding claim 2, the claim recites: wherein the grouping further comprises: encoding, using the one or more pre-trained LLMs, the plurality of sub-modules into sub-module encodings in an embedding space; and clustering the sub-module encodings into the plurality of clusters. The limitations “encoding” and “clustering are functions that can be performed in the human mind or by a human using pen and paper. Furthermore, the additional elements “pre-trained LLMs” and “embedding space” are recited at a high level of generality. Therefore, the claim does not integrate the judicial exception into a practical application at Step 2A Prong 2, or recite additional elements that amount to significantly more than the judicial exception at Step 2B.
Regarding claim 3, the claim recites: wherein the selecting further comprises: determining centroids in the plurality of clusters; and selecting a subset of sub-module encodings that are closest to the centroid. The limitation “determining” is a function that can be performed in the human mind or by a human using pen and paper. Furthermore, the additional element “centroid” is recited at a high level of generality. Therefore, the claim does not integrate the judicial exception into a practical application at Step 2A Prong 2, or recite additional elements that amount to significantly more than the judicial exception at Step 2B.
Regarding claim 4, the claim recites: converting the subset of sub-module encodings into the representative sub-modules. The limitation “converting” is nothing more than an insignificant extra solution activity, and is therefore not integrated into a practical application under Step 2A Prong 2. Furthermore, the additional element does not amount to significantly more than the judicial exception at Step 2B.
Regarding claim 5, the claim recites: generating the source code from the new sub-modules. The limitation “generating the source code from the new sub-modules”, is a function that can be performed in the human mind or by a human using pen and paper. Furthermore, the claim does not recite any other additional elements. Therefore, the claim does not integrate the judicial exception into a practical application at Step 2A Prong 2, or recite additional elements that amount to significantly more than the judicial exception at Step 2B.
Regarding claim 6, the claim recites: executing the program to generate a solution for the problem description. The limitation “executing the program” is nothing more than an insignificant extra solution activity, and is therefore not integrated into a practical application under Step 2A Prong 2. Furthermore, the additional element does not amount to significantly more than the judicial exception at Step 2B.
Regarding claim 7, the claim recites: wherein generating the plurality of sub-modules further comprises: generating outlines describing the plurality of sub-modules; and generating source code from the outlines to be included in the plurality of sub-modules. The limitations “generating outlines” and “generating source code” are functions that can be performed in the human mind or by a human using pen and paper. Furthermore, the claim does not recite any other additional elements. Therefore, the claim does not integrate the judicial exception into a practical application at Step 2A Prong 2, or recite additional elements that amount to significantly more than the judicial exception at Step 2B.
Regarding claim 8, the claim recites: wherein an outline in the outlines includes a function header and a description statement. The additional elements “function header” and “description statement” are recited at a high level of generality. Therefore, the claim does not integrate the judicial exception into a practical application at Step 2A Prong 2, or recite additional elements that amount to significantly more than the judicial exception at Step 2B.
Regarding claim 10, the claim recites: wherein the operations for selecting the subset of the sub-module encodings further comprise operations: selecting sub-module encodings into the subset of sub-module encodings that are with a predefined distance to centroids of the plurality of clusters in the embedding space. The limitation “selecting” is a function that can be performed in the human mind or by a human using pen and paper. Furthermore, the additional element “centroid” is recited at a high level of generality. Therefore, the claim does not integrate the judicial exception into a practical application at Step 2A Prong 2, or recite additional elements that amount to significantly more than the judicial exception at Step 2B.
Regarding claim 10, the claim recites: wherein the operations for selecting the subset of the sub-module encodings further comprise operations: selecting sub-module encodings into the subset of sub-module encodings that are with a predefined distance to centroids of the plurality of clusters in the embedding space. The limitation “selecting” is a function that can be performed in the human mind or by a human using pen and paper. Furthermore, the additional element “centroid” is recited at a high level of generality. Therefore, the claim does not integrate the judicial exception into a practical application at Step 2A Prong 2, or recite additional elements that amount to significantly more than the judicial exception at Step 2B.
Regarding claim 11, the claim recites: wherein the operations for selecting the subset of the sub-module encodings further comprise operations: selecting one sub-module encoding from one cluster in the plurality of clusters into the subset of sub-module encodings, wherein the one sub-module encoding has a closest distance to a centroid of the one cluster. The limitation “selecting” is a function that can be performed in the human mind or by a human using pen and paper. Furthermore, the additional element “centroid” is recited at a high level of generality. Therefore, the claim does not integrate the judicial exception into a practical application at Step 2A Prong 2, or recite additional elements that amount to significantly more than the judicial exception at Step 2B.
Regarding claim 12, the claim recites: selecting a pre-defined number of sub-modules from the plurality of sub-modules; and wherein the encoding further comprises, encoding the pre-defined number of sub-modules into the sub-module encodings. The limitations “selecting” and “encoding” are functions that can be performed in the human mind or by a human using pen and paper. Furthermore, the claim does not recite any other additional elements. Therefore, the claim does not integrate the judicial exception into a practical application at Step 2A Prong 2, or recite additional elements that amount to significantly more than the judicial exception at Step 2B.
Regarding claim 13, the claim recites: wherein the operations further comprise: combining the representative sub-modules into source code. The limitation “combining” is a function that can be performed in the human mind or by a human using pen and paper. Furthermore, the additional element “source code” is recited at a high level of generality. Therefore, the claim does not integrate the judicial exception into a practical application at Step 2A Prong 2, or recite additional elements that amount to significantly more than the judicial exception at Step 2B.
Regarding claim 14, the claim recites: wherein the operations further comprise: executing the source code to generate a solution to the problem description. The limitation “executing the source code” is nothing more than an insignificant extra solution activity, and is therefore not integrated into a practical application under Step 2A Prong 2. Furthermore, the additional element does not amount to significantly more than the judicial exception at Step 2B.
Regarding claim 15, the claim recites: wherein generating the plurality of sub-modules, the operations further comprise: generating outlines describing the plurality of sub-modules; and generating source code from the outlines to be included in the plurality of sub-modules. The limitations “generating outlines” and “generating source code” are functions that can be performed in the human mind or by a human using pen and paper. Furthermore, the claim does not recite any other additional elements. Therefore, the claim does not integrate the judicial exception into a practical application at Step 2A Prong 2, or recite additional elements that amount to significantly more than the judicial exception at Step 2B.
Regarding claim 16, the claim recites: wherein an outline in the outlines includes a function header and a description statement. The additional elements “function header” and “description statement” are recited at a high level of generality. Therefore, the claim does not integrate the judicial exception into a practical application at Step 2A Prong 2, or recite additional elements that amount to significantly more than the judicial exception at Step 2B.
Regarding claim 18, the claim recites: encoding using the one or more pre-trained LLMs, the plurality of sub-modules into sub-modules encoding in an embedded space and clustering the sub-module encodings into the plurality of clusters. However, they are mental processes that can be performed in the human mind or by a human using pen and paper.
Regarding claim 19, the claim recites: generating source code from the new sub-modules. However, it is mental process that can be performed in the human mind or by a human using pen and paper.
Regarding claim 20, the claim recites: selecting sub-module encodings from the plurality of clusters, one sub-module encoding from one cluster in the plurality of clusters. However, it is mental process that can be performed in the human mind or by a human using pen and paper.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, 3, 5, 6, 9, 10, 11, 12, 17, 18, 19, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al. (US20240020096, Chen hereinafter), in view of Buliani et al. (US12530173, Buliani hereinafter), and Ye et al. (US20210182031, Ye hereinafter).
Regarding claim 1, Chen discloses: A method comprising:
generating, (see Chen, paragraph [0005], “…in an embodiment, a method for… generating, using a trained machine-learning model and based on the docstring, one or more computer code samples…”), (see Chen, paragraph [0005], “…in an embodiment, a method for generating computer code based on natural language input…”),
wherein the problem description describes a programming task (see Chen et al., paragraph [0005]) and
includes test cases comprising input and output pairs (see Chen, paragraph [0063], “…Unit tests may be based on known input-output data combinations (e.g., test cases), known behavioral data (e.g., properties, invariants, edge data), or a combination of both…”),
generating new sub-modules from the plurality of sub-modules over a plurality of iterations until the program generates expected results specified by the output pairs for the problem description (see Chen, paragraph [0084], “…step 530 may include initializing the machine learning model with random weights and biases, iterating over the training data (e.g., performing a forward pass, calculating the loss/error between predicted output and true output, and performing a backward pass), and/or repeating iteration until convergence or a predefined stopping criterion is met…”), (see Chen, paragraph [0063]), wherein the generating comprises:
(see Chen, paragraph [0005]);
selecting representative sub-modules (see Chen, paragraph [0086], “In some embodiments, a method may also comprise receiving one or more computer code samples. Receiving, as used herein, may refer to requesting, accessing, obtaining, acquiring, accepting, identifying, selecting, highlighting, and/or collecting.”), (see Chen et al., paragraph [0005]);
(see Chen, paragraph [0005]);
generating, (see Chen., paragraph [0005]).
Chen does not appear to distinctly disclose:
wherein each sub-module in the plurality of sub-modules comprises source code representing a sub-task of the problem description;
grouping
augmenting the problem description
generating a program from the new sub-modules.
However, Buliani discloses:
generating, using one or more pre-trained large language models (LLMs), a plurality of sub-modules from a problem description in a natural language (see Buliani , col 28 line 66-67, “…Block 1904 can also include augmenting the prompt to the LLM…”), (see Buliani, col 5 lines 32-51, “…a development task agent of the SDS can cause an LLM to subdivide a given software development task into discrete actions… In some examples, the detailed actions include generated code snippets…”);
wherein each sub-module in the plurality of sub-modules comprises source code representing a sub-task of the problem description (see Buliani, col 5 lines 32-51);
generating new sub-modules from the plurality of sub-modules (see Buliani, col 5 lines 32-51)
augmenting the problem description with the representative sub-modules (see Buliani, col 28 lines 66-67);
generating new sub-modules from the augmented problem description (see Buliani., col 5 lines 32-51), (see Buliani, col 28 lines 66-67); and
generating a program from the new sub-modules (see Buliani, col 32 lines 64-67, “…once the changes are accepted, the SDS 110 can generate a final system map incorporating all of the changes and cause the deployment of the software system…”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified a system for generating computer code, as taught by Chen, to include an augmented problem description and the generation of a program, as taught by Buliani, for the result of generating a final output based on refined code.
Chen as modified does not appear to distinctly disclose:
grouping
However, Ye discloses:
grouping (see Ye, paragraph [0026], “…For example, the identifier 310 can use the code similarity system to translate code snippets to their vector forms for use by a clustering algorithm…”); and
(see Ye, paragraph [0026]);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified a system for generating computer code, as taught by Chen, to include the clustering of data, as taught by Ye, for the result of reducing redundancy among sub-modules used in subsequent code generation steps.
Regarding claim 2, Chen teaches: encoding, (see Chen, Paragraph [0071], “…the trained machine learning model may comprise… a transformer decoder architecture. A transformer decoder architecture, as used herein, may refer to a machine learning architecture comprising an encoder… wherein the encoder focuses on encoding the input sequence…”), (see Chen et al., paragraph [0005], “…in an embodiment, a method for… generating, using a trained machine-learning model and based on the docstring, one or more computer code samples…”); and (see Chen et al., paragraph [0005]), (see Chen et al., Paragraph [0071]);
Chen does not appear to distinctly disclose:
However, Buliani teaches: (see Buliani et al., col 28 line 66, “…Block 1904 can also include augmenting the prompt to the LLM…”);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified a system for generating computer code, as taught by Chen, to include a large language model (LLM), as taught by Buliani, for the result of refining output code.
Chen as modified does not appear to distinctly disclose: clustering
However, Ye teaches: clustering (see Ye et al., paragraph [0026], “…For example, the identifier 310 can use the code similarity system to translate code snippets to their vector forms for use by a clustering algorithm…”);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified a system for generating computer code, as taught by Chen, to include the clustering of data, as taught by Ye, for the result of reducing redundancy among sub-modules used in subsequent code generation steps.
Regarding claim 3, Chen teaches: selecting a subset of sub-module encodings (see Chen et al., paragraph [0086], “In some embodiments, a method may also comprise receiving one or more computer code samples. Receiving, as used herein, may refer to requesting, accessing, obtaining, acquiring, accepting, identifying, selecting, highlighting, and/or collecting.…”), (see Chen et al., paragraph [0005], “…in an embodiment, a method for… generating, using a trained machine-learning model and based on the docstring, one or more computer code samples…”), (see Chen et al., Paragraph [0071], “…the trained machine learning model may comprise… a transformer decoder architecture. A transformer decoder architecture, as used herein, may refer to a machine learning architecture comprising an encoder… wherein the encoder focuses on encoding the input sequence…”);
Chen does not appear to distinctly disclose: determining centroids in the plurality of clusters; and
However, Ye teaches: determining centroids in the plurality of clusters (see Ye et al., paragraph [0026], “…For example, the identifier 310 can define a centroid of the cluster as the reference copy of the code…”); and
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified a system for generating computer code, as taught by Chen, to include a centroid in a cluster of data, as taught by Ye, for the result of reducing redundancy among sub-modules used in subsequent code generation steps.
Regarding claim 5, Chen teaches:
generating the source code from the new sub-modules; (see Chen et al., paragraph [0005], “…in an embodiment, a method for… generating, using a trained machine-learning model and based on the docstring, one or more computer code samples…”), (see Chen et al., paragraph [0005], “…in an embodiment, a method for generating computer code based on natural language input…”), (see Chen et al., paragraph [0046], “As a result, the user may… create executable computer code which provides a solution to the problem…”);
Regarding claim 6, Chen teaches: executing the program to generate a solution for the problem description (see Chen et al., paragraph [0054], “In some embodiments, a method may further comprise causing each of the one or more computer code samples to be executed…”), (see Chen et al., paragraph [0046], “As a result, the user may… create executable computer code which provides a solution to the problem…”);
Regarding claim 9, Chen discloses: A system comprising: a memory storing (see Chen, paragraph [0016], “…a system for generating computer code based on natural language input may include at least one memory storing instructions and at least one processor configured to execute the instructions to perform operations…”), (see Chen et al., paragraph [0080], “The process shown in FIG. 1 or any of its constituent steps may be implemented using operating environment 700, system 800 (e.g., using at least one processor and at least one memory component), or any component thereof…”), the operations comprising:
receiving a problem description in a natural language (see Chen et al., paragraph [0005], “…in an embodiment, a method for generating computer code based on natural language input…”), wherein the problem description describes a programming task (see Chen et al., paragraph [0005]) and
includes test cases comprising input and output pairs (see Chen et al., paragraph [0063], “…Unit tests may be based on known input-output data combinations (e.g., test cases), known behavioral data (e.g., properties, invariants, edge data), or a combination of both…”);
generating, (see Chen et al., paragraph [0005], “…in an embodiment, a method for… generating, using a trained machine-learning model and based on the docstring, one or more computer code samples…”), (see Chen et al., paragraph [0005], “…in an embodiment, a method for generating computer code based on natural language input…”),
encoding, (see Chen et al., Paragraph [0071], “…the trained machine learning model may comprise… a transformer decoder architecture. A transformer decoder architecture, as used herein, may refer to a machine learning architecture comprising an encoder… wherein the encoder focuses on encoding the input sequence…”), (see Chen et al., paragraph [0005], “…in an embodiment, a method for… generating, using a trained machine-learning model and based on the docstring, one or more computer code samples…”);
(see Chen et al., paragraph [0005]), (see Chen et al., paragraph [0071]);
selecting a subset of sub-module encodings (see Chen et al., paragraph [0086], “In some embodiments, a method may also comprise receiving one or more computer code samples. Receiving, as used herein, may refer to requesting, accessing, obtaining, acquiring, accepting, identifying, selecting, highlighting, and/or collecting.”), (see Chen et al., paragraph [0005]), (see Chen et al., paragraph [0071]);
generating representative sub-modules from the subset of sub-module encodings (see Chen et al., paragraph [0005]), (see Chen et al., paragraph [0071]); and
generating, (see Chen et al., paragraph [0005]);
Chen does not appear to distinctly disclose:
wherein each sub-module in the plurality of sub-modules comprises source code representing a sub-task of the problem description;
clustering
However, Buliani discloses: A system comprising: a memory storing one or more pre-trained large language models (LLMs) (see Buliani, col 36 lines 46-51, “…program instructions and data implementing one or more desired functions, such as those methods, techniques, and data described above, are shown stored within the system memory 3020 as SDS code 3025 (e.g., executable to implement, in whole or in part, an LLM service such as those described herein) and data 3026”);
generating, using the one or more pre-trained LLMs, a plurality of sub-modules from the problem description (see Buliani et al., col 28 line 66-67, “…Block 1904 can also include augmenting the prompt to the LLM…”), (see Buliani et al., col 5 lines 32-51, “…a development task agent of the SDS can cause an LLM to subdivide a given software development task into discrete actions… In some examples, the detailed actions include generated code snippets…”),
wherein each sub-module in the plurality of sub-modules comprises source code representing a sub-task of the problem description (see Buliani et al., col 5 lines 32-51);
(see Buliani et al., col 28 line 66-67); and
(see Buliani et al., col 28 line 66-67);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified a system for generating computer code, as taught by Chen, to include a large language model (LLM) and an augmented problem description, as taught by Buliani, for the result of refining output code.
Chen as modified does not appear to distinctly disclose:
clustering
However, Ye discloses:
clustering (see Ye et al., paragraph [0026], “…For example, the identifier 310 can use the code similarity system to translate code snippets to their vector forms for use by a clustering algorithm…”);
(see Ye et al., paragraph [0026]);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified a system for generating computer code, as taught by Chen, to include the clustering of data, as taught by Ye, for the result of reducing redundancy among sub-modules used in subsequent code generation steps.
Regarding claim 10, Chen teaches: selecting sub-module encodings into the subset of sub-module encodings (see Chen et al., paragraph [0086], “In some embodiments, a method may also comprise receiving one or more computer code samples. Receiving, as used herein, may refer to requesting, accessing, obtaining, acquiring, accepting, identifying, selecting, highlighting, and/or collecting…”), (see Chen et al., paragraph [0005], “…in an embodiment, a method for… generating, using a trained machine-learning model and based on the docstring, one or more computer code samples…”), (see Chen et al., Paragraph [0071], “…the trained machine learning model may comprise… a transformer decoder architecture. A transformer decoder architecture, as used herein, may refer to a machine learning architecture comprising an encoder… wherein the encoder focuses on encoding the input sequence…”).
Chen does not appear to distinctly disclose:
However, Ye teaches: (see Ye et al., paragraph [0026], “…For example, the identifier 310 can define a centroid of the cluster as the reference copy of the code…”);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified a system for generating computer code, as taught by Chen, to include a centroid in a cluster of data, as taught by Ye, for the result of reducing redundancy among sub-modules used in subsequent code generation steps.
Regarding claim 11, Chen teaches: selecting one sub-module encoding (see Chen et al., paragraph [0086], “In some embodiments, a method may also comprise receiving one or more computer code samples. Receiving, as used herein, may refer to requesting, accessing, obtaining, acquiring, accepting, identifying, selecting, highlighting, and/or collecting…”), (see Chen et al., paragraph [0005], “…in an embodiment, a method for… generating, using a trained machine-learning model and based on the docstring, one or more computer code samples…”), (see Chen et al., Paragraph [0071], “…the trained machine learning model may comprise… a transformer decoder architecture. A transformer decoder architecture, as used herein, may refer to a machine learning architecture comprising an encoder… wherein the encoder focuses on encoding the input sequence…”);
Chen does not appear to distinctly disclose:
However, Ye teaches: (see Ye et al., paragraph [0026], “…For example, the identifier 310 can define a centroid of the cluster as the reference copy of the code…”);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified a system for generating computer code, as taught by Chen, to include a centroid in a cluster of data, as taught by Ye, for the result of reducing redundancy among sub-modules used in subsequent code generation steps.
Regarding claim 12, Chen teaches: selecting a pre-defined number of sub-modules from the plurality of sub-modules; and wherein the encoding further comprises, encoding the pre-defined number of sub-modules into the sub-module encodings (see Chen et al., paragraph [0086], “In some embodiments, a method may also comprise receiving one or more computer code samples. Receiving, as used herein, may refer to requesting, accessing, obtaining, acquiring, accepting, identifying, selecting, highlighting, and/or collecting…”), (see Chen et al., paragraph [0005], “…in an embodiment, a method for… generating, using a trained machine-learning model and based on the docstring, one or more computer code samples…”), (see Chen et al., Paragraph [0071], “…the trained machine learning model may comprise… a transformer decoder architecture. A transformer decoder architecture, as used herein, may refer to a machine learning architecture comprising an encoder… wherein the encoder focuses on encoding the input sequence…”);
Regarding claim 17, Chen discloses: A non-transitory computer readable medium storing instructions thereon, that when executed by a processor, cause the processor to perform operations (see Chen et al., paragraph [0017], “According to some disclosed embodiments, a non-transitory computer-readable medium may include instructions that are executable by one or more processors to perform operations…”), the operations comprising:
receiving a problem description in a natural language (see Chen et al., paragraph [0005], “…in an embodiment, a method for generating computer code based on natural language input…”);
generating, (see Chen et al., paragraph [0005], “…in an embodiment, a method for… generating, using a trained machine-learning model and based on the docstring, one or more computer code samples…”),
generating new sub-modules from the plurality of sub-modules over a plurality of iterations until a program generates expected results specified in the problem description (see Chen, paragraph [0084], “…step 530 may include initializing the machine learning model with random weights and biases, iterating over the training data (e.g., performing a forward pass, calculating the loss/error between predicted output and true output, and performing a backward pass), and/or repeating iteration until convergence or a predefined stopping criterion is met…”), (see Chen et al., paragraph [0063]), wherein the generating comprises:
(see Chen et al., paragraph [0005]);
selecting representative sub-modules (see Chen et al., paragraph [0086], “In some embodiments, a method may also comprise receiving one or more computer code samples. Receiving, as used herein, may refer to requesting, accessing, obtaining, acquiring, accepting, identifying, selecting, highlighting, and/or collecting.”), (see Chen et al., paragraph [0005]);
(see Chen et al., paragraph [0005]);
generating, (see Chen et al., paragraph [0005]); and
generating the program from the new sub-modules.
Chen does not appear to distinctly disclose:
wherein each sub-module in the plurality of sub-modules comprises source code representing a sub-task of the problem description;
grouping
augmenting the problem description
generating the program from the new sub-modules.
However, Buliani discloses:
generating, using one or more pre-trained large language models (LLMs), a plurality of sub-modules from the problem description (see Buliani et al., col 28 line 66-67, “…Block 1904 can also include augmenting the prompt to the LLM…”), (see Buliani et al., col 5 lines 32-51, “…a development task agent of the SDS can cause an LLM to subdivide a given software development task into discrete actions… In some examples, the detailed actions include generated code snippets…”),
wherein each sub-module in the plurality of sub-modules comprises source code representing a sub-task of the problem description (see Buliani et al., col 5 lines 32-51);
augmenting the problem description with the representative sub-modules (see Buliani et al., col 28 lines 66-67);
generating, using the one or more pre-trained LLMs, the new sub-modules from the augmented problem description (see Buliani et al., col 5 lines 32-51), (see Buliani et al., col 28 lines 66-67); and
generating the program from the new sub-modules (see Buliani et al., col 32 lines 64-67, “…once the changes are accepted, the SDS 110 can generate a final system map incorporating all of the changes and cause the deployment of the software system…”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified a system for generating computer code, as taught by Chen, to include a large language model (LLM) and an augmented problem description, as taught by Buliani, for the result of refining output code.
Chen as modified does not appear to distinctly disclose:
grouping
However, Ye discloses:
grouping (see Ye et al., paragraph [0026], “…For example, the identifier 310 can use the code similarity system to translate code snippets to their vector forms for use by a clustering algorithm…”);
(see Ye et al., paragraph [0026]);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified a system for generating computer code, as taught by Chen, to include the clustering of data, as taught by Ye, for the result of reducing redundancy among sub-modules used in subsequent code generation steps.
Regarding claim 18, Chen teaches: encoding, (see Chen et al., Paragraph [0071], “…the trained machine learning model may comprise… a transformer decoder architecture. A transformer decoder architecture, as used herein, may refer to a machine learning architecture comprising an encoder… wherein the encoder focuses on encoding the input sequence…”), (see Chen et al., paragraph [0005], “…in an embodiment, a method for… generating, using a trained machine-learning model and based on the docstring, one or more computer code samples…”); and (see Chen et al., paragraph [0005]), (see Chen et al., Paragraph [0071]);
Chen does not appear to distinctly disclose:
However, Buliani teaches: (see Buliani et al., col 28 line 66, “…Block 1904 can also include augmenting the prompt to the LLM…”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified a system for generating computer code, as taught by Chen, to include a large language model (LLM), as taught by Buliani, for the result of refining output code.
Chen as modified does not appear to distinctly disclose: clustering
However, Ye teaches: clustering (see Ye et al., paragraph [0026], “…For example, the identifier 310 can use the code similarity system to translate code snippets to their vector forms for use by a clustering algorithm…”);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified a system for generating computer code, as taught by Chen, to include the clustering of data, as taught by Ye, for the result of reducing redundancy among sub-modules used in subsequent code generation steps.
Regarding claim 19, Chen teaches: generating source code from the new sub-modules (see Chen et al., paragraph [0005], “…in an embodiment, a method for generating computer code based on natural language input…”), (see Chen et al., paragraph [0005], “…in an embodiment, a method for… generating, using a trained machine-learning model and based on the docstring, one or more computer code samples…”);
Regarding claim 20, Chen teaches: selecting sub-module encodings (see Chen et al., paragraph [0086], “In some embodiments, a method may also comprise receiving one or more computer code samples. Receiving, as used herein, may refer to requesting, accessing, obtaining, acquiring, accepting, identifying, selecting, highlighting, and/or collecting.…”), (see Chen et al., paragraph [0005], “…in an embodiment, a method for… generating, using a trained machine-learning model and based on the docstring, one or more computer code samples…”), (see Chen et al., Paragraph [0071], “…the trained machine learning model may comprise… a transformer decoder architecture. A transformer decoder architecture, as used herein, may refer to a machine learning architecture comprising an encoder… wherein the encoder focuses on encoding the input sequence…”);
Chen does not appear to distinctly disclose:
However, Ye teaches: (see Ye et al., paragraph [0026], “…For example, the identifier 310 can use the code similarity system to translate code snippets to their vector forms for use by a clustering algorithm…”);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified a system for generating computer code, as taught by Chen, to include the clustering of data, as taught by Ye, for the result of reducing redundancy among sub-modules used in subsequent code generation steps.
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Chen, Buliani, and Ye as applied to claim 3 above, and further in view of Fells et al. (GB2507551, Fells hereinafter).
Regarding claim 4, Chen teaches: (see Chen et al., paragraph [0005], “…in an embodiment, a method for… generating, using a trained machine-learning model and based on the docstring, one or more computer code samples…”), (see Chen et al., Paragraph [0071], “…the trained machine learning model may comprise… a transformer decoder architecture. A transformer decoder architecture, as used herein, may refer to a machine learning architecture comprising an encoder… wherein the encoder focuses on encoding the input sequence…”);
Chen does not appear to distinctly disclose: converting
However, Fells teaches: converting the subset of sub-module encodings into the representative sub-modules (see Fells, paragraph [30], “The mapper 132 converts the embedded code…”);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified a system for generating computer code, as taught by Chen, to include the conversion of data, as taught by Fells, for the result of reducing redundancy among sub-modules used in subsequent code generation steps.
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Chen, Buliani, and Ye as applied to claim 1 above, and further in view of Yo et al. (KR20230065017, Yo hereinafter).
Regarding claim 7, Chen teaches: (see Chen et al., paragraph [0005], “…in an embodiment, a method for… generating, using a trained machine-learning model and based on the docstring, one or more computer code samples…”); and generating source code (see Chen et al., paragraph [0005], “…in an embodiment, a method for generating computer code based on natural language input…”);
Chen does not appear to distinctly disclose: generating outlines describing
However, Yo teaches: generating outlines describing (see Yo et al., Abstract, “The present invention relates to an apparatus and a method for generating a summary of a program source code based on artificial intelligence analysis…”);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified a system for generating computer code, as taught by Chen, to include the generation of an outline, as taught by Yo, for the result of decomposing complex tasks into smaller and more easily manageable steps.
Claims 8 is rejected under 35 U.S.C. 103 as being unpatentable over Chen, Buliani, Ye, and Yo as applied to claim 7 above, and further in view of Yoneyama (JP3736308).
Regarding claim 8, Chen as modified does not appear to distinctly disclose: wherein an outline in the outlines includes a function header and a description statement.
However, Yo teaches: wherein an outline in the outlines includes (see Yo et al., paragraph [0041], “1 is a diagram illustrating a system for generating a summary sentence according to the present invention.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified a system for generating computer code, as taught by Chen, to include the generation of an outline containing a description, as taught by Yo, for the result of decomposing complex tasks into smaller and more easily manageable steps.
Chen as modified does not appear to distinctly disclose:
However, Yoneyama teaches: (see Yoneyama, paragraph [0057], “…This process corresponds to a process for generating a standard header corresponding to the acquired component name 521.”);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified a system for generating computer code, as taught by Chen, to include the generation of an outline containing a header, as taught by Yoneyama, for the result of decomposing complex tasks into smaller and more easily manageable steps.
Claims 13 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Chen, Buliani, and Ye as applied to claim 9 above, and further in view of Di et al. (CN116401091, Di hereinafter).
Regarding claim 13, Chen teaches: (see Chen et al., paragraph [0005], “…in an embodiment, a method for… generating, using a trained machine-learning model and based on the docstring, one or more computer code samples…”), (see Chen et al., paragraph [0005], “…in an embodiment, a method for generating computer code based on natural language input…”);
Chen does not appear to distinctly disclose: combining
However, Di teaches: combining the representative sub-modules into source code (see Di et al., Abstract, “Step for combining the embedded code data”);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified a system for generating computer code, as taught by Chen, to include the combination of data, as taught by Di, for the result of outputting a practical solution to the problem description.
Regarding claim 14, Chen teaches: executing the source code to generate a solution to the problem description (see Chen et al., paragraph [0054], “In some embodiments, a method may further comprise causing each of the one or more computer code samples to be executed…”), (see Chen et al., paragraph [0046], “As a result, the user may… create executable computer code which provides a solution to the problem…”);
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Chen, Buliani, and Ye as applied to claim 9 above, and further in view of Yo et al. (KR20230065017, Yo hereinafter).
Regarding claim 15, Chen teaches: (see Chen et al., paragraph [0005], “…in an embodiment, a method for… generating, using a trained machine-learning model and based on the docstring, one or more computer code samples…”), (see Chen et al., paragraph [0005], “…in an embodiment, a method for generating computer code based on natural language input…”);
Chen does not appear to distinctly disclose: generating outlines describing
However, Yo teaches: generating outlines describing (see Yo et al., Abstract, “The present invention relates to an apparatus and a method for generating a summary of a program source code based on artificial intelligence analysis…”);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified a system for generating computer code, as taught by Chen, to include the generation of an outline, as taught by Yo, for the result of decomposing complex tasks into smaller and more easily manageable steps.
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Chen, Buliani, Ye, and Yo as applied to claims 15 above, and further in view of Yoneyama (JP3736308).
Regarding claim 16, Chen as modified does not appear to distinctly disclose: wherein an outline in the outlines includes a function header and a description statement.
However, Yo teaches: wherein an outline in the outlines includes (see Yo et al., paragraph [0041], “1 is a diagram illustrating a system for generating a summary sentence according to the present invention.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified a system for generating computer code, as taught by Chen, to include the generation of an outline containing a description, as taught by Yo, for the result of decomposing complex tasks into smaller and more easily manageable steps.
Chen as modified does not appear to distinctly disclose:
However, Yoneyama teaches: (see Yoneyama, paragraph [0057], “…This process corresponds to a process for generating a standard header corresponding to the acquired component name 521.”);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified a system for generating computer code, as taught by Chen, to include the generation of an outline containing a header, as taught by Yoneyama, for the result of decomposing complex tasks into smaller and more easily manageable steps.
Response to Arguments
Applicant’s argument: The claims are not directed towards an abstract idea.
Examiner’s response: The applicant’s argument is considered but is not persuasive. The applicant states that a human cannot practically perform the operations of a pre-trained large language model in their mind or using pen and paper because such models contain billions of parameters and perform complex matrix operations. However, the claims fail to cite any of these billions of parameters or complex matrix operations. Additionally, the specification is not to be imported into the claims. Therefore, the claims are still directed to an abstract idea.
Applicant’s argument: The prior art does not teach problem description, test cases, and sub-modules.
Examiner’s response: The applicant’s argument is considered but is not persuasive. Chen teaches a docstring representing natural language text. Considering broadest reasonable interpretation, this is functionally equivalent to a problem description which is described in a natural language. Additionally, Chen states in paragraph [0053] that additional unit tests may be created based on examples found in the problem statement. Furthermore, Buliani teaches subdividing a software development task into discrete actions, and each discrete action can include generated code snippets. Considering broadest reasonable interpretation, this reads on sub-modules as source code representing sub-tasks. Therefore, the prior art teaches problem description, test cases, and sub-modules.
Applicant’s argument: The prior art does not teach iteratively generating sub-modules until the program generates expected results.
Examiner’s response: The applicant’s argument is considered but is not persuasive. Chen states in paragraph [0084] that the machine learning model iterates over training data, calculates the loss/error between the predicted output and true output, and iterates until a predefined criterion is met. Considering broadest reasonable interpretation, this reads on generating new sub-modules from the plurality of sub-modules over a plurality of iterations until the program generates expected results specified by the output pairs for the problem description. Therefore, the prior art teaches iteratively generating sub-modules until the program generates expected results.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Chen et al. (“Teaching Large Language Models to Self-Debug” Apr 11th 2023) disclose prompting a large language model to generate code from a problem description that includes unit tests in the form of input-output pairs, executing the generated code against those tests, and then iteratively re-prompting the model with execution feedback to generate revised code over successive turns until the code passes all unit tests or a maximum number of debugging turns is reached.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Joshua Tran whose telephone number is (571)272-5460. The examiner can normally be reached on M-F 9-5.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hyung Sough can be reached on (571)272-6799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JOSHUA TRAN/Examiner, Art Unit 2192
/S. Sough/SPE, Art Unit 2192
Prosecution Insights