Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-20 are presented for examination.
Claim Objections
Claims 1 and 11 are objected to because of the following informalities. Each of claims 1 and 11 recites “cause a neural network, to generate feedback to the query,” which includes an extraneous comma following the phrase “a neural network.” The Examiner suggests amending the phrase to read “cause a neural network to generate feedback to the query.” Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
Claim 1 as drafted, recite a process that, under its broadest reasonable interpretation, covers steps that could reasonably be performed in the mind, including with the aid of pen and paper, but for the recitation of generic computer components. That is, the limitation “generate, based at least on one or more requirements for a software product and one or more criteria related to the one or more requirements, a prompt representative of the one or more requirements, the one or more criteria, a query, and at least a portion of the software product” as drafted, is a process that, under its broadest reasonable interpretation, recite the abstract idea of mental processes. A reviewer of software requirements can formulate, in the mind or with pen and paper, a review question that recites the requirement under review, the writing standard the requirement must satisfy, and the software component to which the requirement pertains. These limitations encompass a human mind carrying out these functions through observation, evaluation, judgment and /or opinion, or even with the aid of pen and paper. Thus, these limitations recite and fall within the “Mental Processes” grouping of abstract ideas.
This judicial exception is not integrated into a practical application. The claims recites the following additional elements “one or more processors,” “one or more circuits,” “a neural network,” “the neural network configured based at least on training data comprising a plurality of examples of requirements and associated criteria, and a plurality of examples of feedback corresponding to the examples of requirements and associated criteria,” and “cause a presentation of the feedback.” The additional elements “one or more processors,” “one or more circuits,” “a neural network,” and the recitation that the neural network is configured on training data are merely instructions to implement an abstract idea on a computer, or merely using a generic computer or computer components as a tool to perform the abstract idea. See MPEP 2106.05(f). The additional element “cause a presentation of the feedback” does nothing more than add insignificant extra solution activity to the judicial exception, such as outputting the results of the abstract idea, to perform a task. See MPEP 2106.05(g). Accordingly, the additional elements recited in the claims do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea, thus fail to integrate the abstract idea into a practical application.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements “one or more processors,” “one or more circuits,” and “a neural network” are generic computer components and instructions used as the tools to perform the abstract idea. See MPEP 2106.05(f). As to the additional element “cause a presentation of the feedback,” the courts have identified gathering data and displaying the output of the abstract idea is well-understood, routine, conventional activity. See MPEP 2106.05(d). Accordingly, the additional elements recited in the claims cannot provide an inventive concept. Thus, the claims are not patent eligible.
Claim 2 further defines the “feedback” as “at least one of an indication of a modification of text of the one or more requirements or the modification of text” as part of the generating function set forth in the claim from which it depends, thus, is also considered to recite a mental process that can be reasonably carried out through observation, evaluation, judgment and /or opinion, or even with the aid of pen and paper. A human reviewer can mark up a requirement statement and write the corrected wording. No additional element is recited that integrates the abstract idea into a practical application or provides an inventive concept.
Claim 3 recites the additional element “wherein the plurality of examples of feedback comprise at least one of: a first example of feedback indicating that a first example of requirements … meets a first criterion … or a fourth example of feedback indicating that a fourth example of requirements … does not meet the second criterion” which does nothing more than add insignificant extra solution activity to the judicial exception, such as data gathering to perform a task. See MPEP 2106.05(g). Accordingly, the additional elements recited in the claim do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea, thus fail to integrate the abstract idea into a practical application. Further, the courts have identified gathering data for use by the abstract idea is well-understood, routine, conventional activity. See MPEP 2106.05(d). Accordingly, the additional elements recited in the claim cannot provide an inventive concept. Thus, the claim is not patent eligible.
Claim 4 recites the additional elements “a prompt tuning of the neural network” and “updating one or more parameters of the neural network based at least on one or more annotations of the plurality of examples of requirements or the plurality of examples of feedback,” which are merely instructions to implement an abstract idea on a computer, or merely using a generic computer or computer components as a tool to perform the abstract idea. See MPEP 2106.05(f). Accordingly, the additional elements recited in the claim do not integrate the abstract idea into a practical application, nor do they amount to significantly more than the judicial exception. Thus, the claim is not patent eligible.
Claim 5 recites the additional elements “one or more language models” and “trained using natural language processing (NLP) to model the one or more requirements and generate the feedback,” which are merely instructions to implement an abstract idea on a computer, or merely using a generic computer or computer components as a tool to perform the abstract idea. See MPEP 2106.05(f). Accordingly, the additional elements recited in the claim do not integrate the abstract idea into a practical application, nor do they amount to significantly more than the judicial exception. Thus, the claim is not patent eligible.
Claim 6 recites the additional element “a transformer architecture, the transformer architecture transforming the prompt representative of the one or more criteria into the feedback in a human-readable format,” which is merely an instruction to implement an abstract idea on a computer, or merely using a generic computer or computer components as a tool to perform the abstract idea. See MPEP 2106.05(f). The recitation of a particular model architecture at this level of generality does not impose a meaningful limit on practicing the abstract idea. Accordingly, the additional element recited in the claim does not integrate the abstract idea into a practical application, nor does it amount to significantly more than the judicial exception. Thus, the claim is not patent eligible.
Claim 7 as drafted, recite a process that, under its broadest reasonable interpretation, covers steps that could reasonably be performed in the mind, including with the aid of pen and paper, but for the recitation of generic computer components. That is, the limitation “generate, based at least on the one or more requirements, a second prompt representative of the one or more criteria” as drafted, is a process that, under its broadest reasonable interpretation, recite the abstract idea of mental processes. These limitations encompass a human mind carrying out these functions through observation, evaluation, judgment and /or opinion, or even with the aid of pen and paper. Thus, these limitations recite and fall within the “Mental Processes” grouping of abstract ideas. The claim further recites the additional element “retrieve a second text subsequent to output of the first feedback,” which does nothing more than add insignificant extra solution activity to the judicial exception, such as data gathering, to perform a task. See MPEP 2106.05(g). The courts have identified gathering data for use by the abstract idea is well-understood, routine, conventional activity. See MPEP 2106.05(d). Accordingly, the additional elements recited in the claim cannot provide an inventive concept. Thus, the claim is not patent eligible.
Claim 8 further defines the “prompt” as being “further generated based at least on a feedback level” as part of the generating function set forth in the claim from which it depends, thus, is also considered to recite a mental process that can be reasonably carried out through observation, evaluation, judgment and /or opinion, or even with the aid of pen and paper. A human reviewer can decide how detailed a review comment should be before writing it. No additional element is recited that integrates the abstract idea into a practical application or provides an inventive concept.
Claim 9 recites the additional element “wherein the training data comprises a plurality of feedback level examples corresponding with the plurality of examples of requirements and the plurality of examples of feedback,” which does nothing more than add insignificant extra solution activity to the judicial exception, such as data gathering, to perform a task. See MPEP 2106.05(g). Further, the courts have identified gathering data for use by the abstract idea is well-understood, routine, conventional activity. See MPEP 2106.05(d). Accordingly, the additional elements recited in the claim cannot provide an inventive concept. Thus, the claim is not patent eligible.
Claim 10 recites that the one or more processors are comprised in “a system comprising one or more large language models (LLMs) … a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources,” each of which is a generic computing environment recited at a high level of generality and used merely as a tool to perform the abstract idea. See MPEP 2106.05(f). Reciting sixteen alternative deployment environments does not confine the claim to a particular machine or to a particular technological improvement. Accordingly, the additional elements recited in the claim do not integrate the abstract idea into a practical application, nor do they amount to significantly more than the judicial exception. Thus, the claim is not patent eligible.
Claim 11 is a system claim that recites the same abstract idea “generate, based at least on one or more requirements for a software product and one or more criteria related to the one or more requirements, a prompt representative of the one or more requirements, the one or more criteria, a query, and at least a portion of the software product” set forth in claim 1, together with the additional elements “one or more processors to execute operations,” “a neural network,” and “cause a presentation of the feedback.” For the reasons set forth above in the rejection of claim 1, the additional elements are generic computer components used as tools to perform the abstract idea (see MPEP 2106.05(f)) and insignificant extra solution activity (see MPEP 2106.05(g)), and therefore neither integrate the abstract idea into a practical application nor provide an inventive concept. Thus, the claim is not patent eligible.
Claims 12-18 are system claims corresponding to the claims analyzed above, namely claim 12 corresponding to claim 2, claim 13 corresponding to claim 3, claim 14 corresponding to claim 4, claim 15 corresponding to claims 5 and 6, claim 16 corresponding to claim 7, claim 17 corresponding to claim 8, and claim 18 corresponding to claim 10, and are rejected for the same reasons set forth above in the rejections of the corresponding claims.
Claim 19 is a method claim that recites the same abstract idea “generating, using one or more processors based at least on one or more requirements for a software product and one or more criteria related to the one or more requirements, a prompt representative of the one or more requirements, the one or more criteria, a query, and at least a portion of the software product” set forth in claim 1, together with the additional elements “one or more processors,” “a neural network,” and “causing, using the one or more processors, a presentation of the feedback.” For the reasons set forth above in the rejection of claim 1, the additional elements are generic computer components used as tools to perform the abstract idea (see MPEP 2106.05(f)) and insignificant extra solution activity (see MPEP 2106.05(g)). Thus, the claim is not patent eligible.
Claim 20 further defines the “feedback” and the “prompt” as part of the generating function set forth in the claim from which it depends, and is rejected for the same reasons set forth above in the rejections of claims 2 and 8. Thus, the claim is not patent eligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-3, 5-7, 10-13, 15, 16, 18 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Parrish (US 2022/0382977 A1) in view of Vaughn (US 2024/0111498 A1)
Regarding Claim 1, Parrish teaches
One or more processors comprising: one or more circuits to:
generate, based at least on one or more requirements for a software product and one or more criteria related to the one or more requirements, (Parrish, Para. 0041, “the capability to scan requirements documents for violation of INCOSE standards (comprising 50+ rules) for well-written requirements, including identifying word(s) or phrases that violate a given rule”) Examiner Comments: The requirements documents scanned by Parrish are the claimed one or more requirements, and the INCOSE rules for well-written requirements against which those documents are scanned are the claimed one or more criteria related to the one or more requirements, because the rules govern how the requirements themselves must be written.
wherein the one or more requirements are requirements for a software product, (Parrish, Para. 0013, “the engineering documents of the second list comprise system or software requirement documents, system or software specification documents, system or software design documents, system or software architecture documents”) Examiner Comments: Parrish expressly identifies the requirement documents it processes as software requirement documents, which establishes that the requirements are requirements for a software product.
cause a neural network, to generate feedback to the query in view of the one or more requirements, the one or more criteria, (Parrish, Para. 0013, “the one or more artificial intelligence (AI)-based natural language processing algorithms comprise a Support Vector Machine (SVM) algorithm, a Bayesian Network algorithm, a Maximum Entropy algorithm, a Conditional Random Field algorithm, a Neural Network or Deep Learning algorithm, a Bidirectional Encoder Representations from Transformers (BERT) algorithm”) Examiner Comments: The neural network and BERT algorithms that Parrish applies to the requirements text are the claimed neural network, and the rule violations that the analyzer returns for a requirement evaluated against the INCOSE criteria are the claimed feedback generated in view of the requirements and the criteria.
the neural network configured based at least on training data comprising a plurality of examples of requirements and associated criteria, and a plurality of examples of feedback corresponding to the examples of requirements and associated criteria; and (Parrish, Para. 0011, “the one or more artificial intelligent (AI)-based natural language processing algorithms may be trained on training data comprising domain-specific vocabulary and acronyms”) Examiner Comments: Parrish configures its neural network on a training data set drawn from the requirements domain, and Parrish further supplies, at paragraph 0131, custom corpuses of words and phrases that violate individual INCOSE rules, which are examples of requirements paired with the criteria they offend and with the corresponding violation feedback.
cause a presentation of the feedback. (Parrish, Para. 0078, “another window 800 of the GUI allows users to review and edit RAAM analysis results for engineering requirement documents, and to automatically apply INCOSE rules. In some instances, a listing of specific requirements derived from a requirements document may be displayed, e.g., in panel 808 at the left side of the GUI”) Examiner Comments: Displaying the analysis results and the applicable INCOSE rules in the graphical user interface is the claimed causing of a presentation of the feedback.
Parrish did not specifically teach
a prompt representative of the one or more requirements, the one or more criteria, a query, and at least a portion of the software product; and
generating the feedback to the query in view of at least the portion of the software product.
However, Vaughn teaches
a prompt representative of the one or more requirements, (Vaughn, Para. 0080, “The system is used by the developer specifying requirements for a new API or to connect new function (calls) to an existing API. The developer prompts the LLM”) Examiner Comments: Vaughn forms the prompt out of the requirements that the developer specifies, so the prompt is representative of the one or more requirements.
a query, and at least a portion of the software product; (Vaughn, Para. 0081, “FIG. 10 shows an example of how to create a prompt that uses the user-developed question and chunks of code and data values as input”) Examiner Comments: The user-developed question incorporated into the prompt is the claimed query, and the chunks of code and data values incorporated alongside it are at least a portion of the software product.
cause a neural network to generate feedback to the query in view of at least the portion of the software product, (Vaughn, Para. 0027, “The processor circuitry 14 or means for processing 14 is to provide the information on the existing application architecture and the prompt as input for a Large Language Model (LLM) 5 . The processor circuitry 14 or means for processing 14 is to obtain an output of the LLM”) Examiner Comments: Vaughn supplies both the prompt and the description of the existing application architecture to the large language model and obtains the model output responsive to that combined input, which is generating a response to the query in view of the portion of the software product.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Vaughn's teaching of assembling a prompt from a developer requirement, a user question, and chunks of the existing code base and submitting that prompt to a large language model with Parrish's requirements quality analyzer in order to evaluate requirements against the INCOSE writing criteria without being confined to the fixed regular expression and corpus matching that Parrish describes at paragraph 0131. Parrish already trains neural network and BERT models on requirements text, and Vaughn teaches at paragraph 0031 that supplying structured context about the application to the model produces responses that are better grounded in the actual system. Substituting a prompted language model for the pattern matching engine is the combination of prior art elements according to known methods to yield the predictable result of a requirements review that accounts for the software component a requirement pertains to.
Regarding Claim 2, Parrish and Vaughn teach
The one or more processors of Claim 1.
Parrish further teaches
wherein the feedback comprises at least one of an indication of a modification of text of the one or more requirements or the modification of text. (Parrish, Para. 0041, “including the ability for users to inspect the specific word(s) or phrases that violated that rule), and allows users to edit the requirement text and re-run the INCOSE analyzer to determine whether the violation has been corrected”) Examiner Comments: Identifying to the user the specific words or phrases in the requirement that violate a rule tells the user what text must be changed, which is an indication of a modification of the text of the requirement.
Regarding Claim 3, Parrish and Vaughn teach
The one or more processors of Claim 1.
Parrish further teaches
wherein the plurality of examples of feedback comprise at least one of: a first example of feedback indicating that a first example of requirements of the plurality of examples of requirements meets a first criterion of the one or more criteria; a second example of feedback indicating that a second example of requirements of the plurality of examples of requirements does not meet the first criterion; (Parrish, Para. 0131, “custom corpuses of words and phrases that violate individual INCOSE or IEEE rules are also compared against requirement descriptions. The presence of any of these words or phrases indicates a rule violation as well”) Examiner Comments: Each corpus entry is an example of requirement wording paired with the individual rule it offends and with the resulting violation determination, and the absence of such wording yields the converse determination that the requirement satisfies that rule, so Parrish supplies examples of feedback in both the meets and does not meet forms recited in the claim.
a third example of feedback indicating that a third example of requirements of the plurality of examples of requirements meets a second criterion of the one or more criteria; or a fourth example of feedback indicating that a fourth example of requirements of the plurality of examples of requirements does not meet the second criterion. (Parrish, Para. 0041, “the capability to scan requirements documents for violation of INCOSE standards (comprising 50+ rules) for well-written requirements”) Examiner Comments: Because Parrish applies more than fifty separate rules, the same meets and does not meet determinations are produced for a second criterion as for the first, which satisfies the third and fourth alternatives of the claim.
Regarding Claim 5, Parrish and Vaughn teach
The one or more processors of Claim 1.
Parrish further teaches
wherein the neural network comprises one or more language models, the one or more language models trained using natural language processing (NLP) to model the one or more requirements and generate the feedback. (Parrish, Para. 0063, “FIG. 4 provides a non-limiting schematic illustration of a machine learning-based process 400 for performing natural language processing and semantic text matching using, e.g., a BERT neural network. Document text is input on a word-by-word basis”) Examiner Comments: The BERT model that Parrish applies to the requirements text is a language model, and Parrish expressly states that it is used to perform natural language processing on that text.
Regarding Claim 6, Parrish and Vaughn teach
The one or more processors of Claim 1.
Parrish further teaches
wherein the neural network comprises a transformer architecture, (Parrish, Para. 0063, “Document text is input on a word-by-word basis (e.g., Word 1 ( 402 ), Word 2 ( 404 ), Word 3 ( 406 ), Word 4 ( 408 ), and Word 5 ( 410 )) via input layer 412 and processed by a transformer encoder 414 before being output”) Examiner Comments: The transformer encoder recited in Parrish is the claimed transformer architecture.
Parrish did not specifically teach
the transformer architecture transforming the prompt representative of the one or more criteria into the feedback in a human-readable format.
However, Vaughn teaches
the transformer architecture transforming the prompt representative of the one or more criteria into the feedback in a human-readable format. (Vaughn, Para. 0045, “The processor circuitry may then obtain an output of the LLM, with the output comprising a portion of the code for implementing the additional component. However, initially, the output may comprise additional questions of the LLM”) Examiner Comments: The large language model of Vaughn returns natural language questions and explanations to the developer in response to the prompt, which is output in a human-readable format.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Vaughn's teaching of returning the model output as natural language readable by the developer with Parrish's transformer based analyzer in order to communicate the reason a requirement violates a criterion rather than merely flagging it, which reduces the interpretation effort Parrish otherwise leaves to the reviewing engineer at paragraph 0078.
Regarding Claim 7, Parrish and Vaughn teach
The one or more processors of Claim 1.
Parrish further teaches
wherein text of the one or more requirements is a first text, the prompt is a first prompt, and the feedback is a first feedback, and the one or more circuits are to: retrieve a second text subsequent to output of the first feedback; (Parrish, Para. 0041, “allows users to edit the requirement text and re-run the INCOSE analyzer to determine whether the violation has been corrected or if new violations have been created”) Examiner Comments: The edited requirement text that Parrish accepts after the first analysis results have been displayed is the claimed second text retrieved subsequent to output of the first feedback.
cause the neural network, based at least on the first feedback, the second text, and the second prompt, to generate a second feedback regarding the second text. (Parrish, Para. 0074, “The analysis results may be exported at step 518 , or may be reviewed and corrected (e.g., via the user interface) at step 520 , followed by export of an improved requirements document 524 at step 522 if the user successfully addressed the writing mistakes in the GUI”) Examiner Comments: Re-running the analyzer on the corrected requirement produces the claimed second feedback regarding the second text, and that second analysis is necessarily informed by the first feedback because the correction was made in response to it.
Parrish did not specifically teach
generate, based at least on the one or more requirements, a second prompt representative of the one or more criteria.
However, Vaughn teaches
generate, based at least on the one or more requirements, a second prompt representative of the one or more criteria; (Vaughn, Para. 0045, “the processor circuitry may obtain a refined prompt or follow-up prompt after presenting the output of the LLM to the user and provide at least the refined prompt or follow-up prompt as input to the LLM”) Examiner Comments: The refined or follow-up prompt that Vaughn constructs after the first output has been presented is the claimed second prompt, and it is generated from the same requirement context that produced the first prompt.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Vaughn's teaching of assembling a prompt from a developer requirement, a user question, and chunks of the existing code base and submitting that prompt to a large language model with Parrish's requirements quality analyzer in order to evaluate requirements against the INCOSE writing criteria without being confined to the fixed regular expression and corpus matching that Parrish describes at paragraph 0131. Parrish already trains neural network and BERT models on requirements text, and Vaughn teaches at paragraph 0031 that supplying structured context about the application to the model produces responses that are better grounded in the actual system. Substituting a prompted language model for the pattern matching engine is the combination of prior art elements according to known methods to yield the predictable result of a requirements review that accounts for the software component a requirement pertains to.
Regarding Claim 10, Parrish and Vaughn teach
The one or more processors of Claim 1.
Parrish further teaches
wherein the one or more processors is comprised in at least one of: a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources. (Parrish, Para. 0050, “the term ‘cloud computing platform’ refers to the operating system and hardware of a computer server (or distributed network of computer servers connected via, e.g., the Internet) that is configured to transfer, process, and/or store electronic data in the cloud”) Examiner Comments: Parrish deploys the requirements analyzer as a containerized web application running on a cloud computing platform, which satisfies the cloud computing resources alternative of this claim. Because the claim recites the alternatives in the at least one of form, satisfying a single alternative satisfies the limitation.
Parrish did not specifically teach
a system comprising one or more large language models (LLMs).
However, Vaughn teaches
a system comprising one or more large language models (LLMs); (Vaughn, Para. 0027, “provide the information on the existing application architecture and the prompt as input for a Large Language Model (LLM) 5”) Examiner Comments: Vaughn's system incorporates a large language model, which satisfies the first alternative of the claim.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Vaughn's teaching of assembling a prompt from a developer requirement, a user question, and chunks of the existing code base and submitting that prompt to a large language model with Parrish's requirements quality analyzer in order to evaluate requirements against the INCOSE writing criteria without being confined to the fixed regular expression and corpus matching that Parrish describes at paragraph 0131. Parrish already trains neural network and BERT models on requirements text, and Vaughn teaches at paragraph 0031 that supplying structured context about the application to the model produces responses that are better grounded in the actual system. Substituting a prompted language model for the pattern matching engine is the combination of prior art elements according to known methods to yield the predictable result of a requirements review that accounts for the software component a requirement pertains to.
Regarding Claim 11, is a system claim corresponding to the claim above (Claim 1) and, therefore, is rejected for the same reasons set forth in the rejection of claim 1.
Regarding Claim 12, is a system claim corresponding to the claim above (Claim 2) and, therefore, is rejected for the same reasons set forth in the rejection of claim 2.
Regarding Claim 13, is a system claim corresponding to the claim above (Claim 3) and, therefore, is rejected for the same reasons set forth in the rejection of claim 3.
Regarding Claim 15, is a system claim corresponding to the claims above (Claims 5 and 6) and, therefore, is rejected for the same reasons set forth in the rejections of claims 5 and 6.
Regarding Claim 16, is a system claim corresponding to the claim above (Claim 7) and, therefore, is rejected for the same reasons set forth in the rejection of claim 7.
Regarding Claim 18, is a system claim corresponding to the claim above (Claim 10) and, therefore, is rejected for the same reasons set forth in the rejection of claim 10.
Regarding Claim 19, is a method claim corresponding to the claim above (Claim 1) and, therefore, is rejected for the same reasons set forth in the rejection of claim 1.
Claims 4 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Parrish in view of Vaughn, further in view of Lester (US 2023/0325725 A1).
Regarding Claim 4, Parrish, and Vaughn teach
The one or more processors of Claim 1.
Parrish and Vaughn did not specifically teach
wherein the configuration of the neural network using the training data comprises a prompt tuning of the neural network, wherein the prompt tuning comprises updating one or more parameters of the neural network based at least on one or more annotations of the plurality of examples of requirements or the plurality of examples of feedback.
However, Lester teaches
wherein the configuration of the neural network using the training data comprises a prompt tuning of the neural network, (Lester, Para. 0030, “The systems and methods can include processing one or more training examples of the plurality of training examples with a pre-trained machine-learned model to generate a training output. In some implementations, the pre-trained machine-learned model can process the one or more training examples and a prompt”) Examiner Comments: Lester configures the model for a target task by processing training examples together with a prompt, which is the claimed prompt tuning of the neural network using training data.
wherein the prompt tuning comprises updating one or more parameters of the neural network (Lester, Para. 0032, “One or more prompt parameters of a prompt can then be adjusted based on the prompt gradient. In some implementations, the prompt can be trained for a particular task associated with the one or more training examples and the one or more training labels”) Examiner Comments: Adjusting the prompt parameters is the claimed updating of one or more parameters, and Lester confirms at paragraph 0034 that those parameters are input with the data into the model such that only those parameters are updated.
based at least on one or more annotations of the plurality of examples of requirements or the plurality of examples of feedback. (Lester, Para. 0031, “A prompt gradient can then be determined based at least in part on a comparison between the training output and one or more training labels associated with the one or more training examples”) Examiner Comments: The training labels that Lester associates with each training example are the claimed annotations, and the parameter update is derived from the comparison against those labels.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Lester's prompt tuning with the prompted requirements analyzer of Parrish and Vaughn in order to specialize the language model to the requirements review task without retraining the underlying model, which Lester identifies at paragraph 0001 as the object of prompt tuning. Vaughn expressly identifies the cost and delay of finetuning a large language model on an entire application corpus as an obstacle at paragraph 0056, and Lester supplies the recognized solution to that obstacle. This is the use of a known technique to improve a similar device in the same way.
Regarding Claim 14, is a system claim corresponding to the claim above (Claim 4) and, therefore, is rejected for the same reasons set forth in the rejection of claim 4.
Claims 8, 9, 17 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Parrish in view of Vaughn, further in view of Gardner et al. (US 2025/0061290 A1).
Regarding Claim 8, Parrish, and Vaughn teach
The one or more processors of Claim 1.
Parrish and Vaughn did not specifically teach
wherein the prompt is further generated based at least on a feedback level, the feedback level causes the neural network to generate the feedback according to predefined compliance of the feedback level.
However, Gardner teaches
wherein the prompt is further generated based at least on a feedback level, (Gardner, Para. 0027, “A level of abstraction is determined for the content item. A prompt is automatically engineered for providing to the one or more LLMs. The prompt includes a reference to the first content item and the level of the abstraction for the first content item”) Examiner Comments: The level of abstraction that Gardner incorporates into the engineered prompt is the claimed feedback level, and the prompt is generated based on it.
the feedback level causes the neural network to generate the feedback according to predefined compliance of the feedback level. (Gardner, Para. 0027, “A response to the prompt is received from the LLM. The response includes a second content item. The second content item includes a representation of the first content item that is generated by the LLM. The representation omits or simplifies one or more of the set of sub-content items based on the level of abstraction”) Examiner Comments: The model response in Gardner conforms to the level carried in the prompt, which is the claimed generation of the response according to the predefined compliance of that level, and Gardner confirms at paragraph 0147 that the level is predefined through user input, an API parameter, a configuration file or a preset.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Gardner's practice of carrying a predefined output level in the prompt with the prompted requirements analyzer of Parrish and Vaughn in order to match the depth of the review comment to the requirement under review, which addresses the problem Gardner identifies at paragraph 0014 of existing approaches lacking sufficient precision and user configurability over the depth of a generated response. Adding a level parameter to a prompt that already carries the requirement, the criteria and the code context is the combination of prior art elements according to known methods to yield predictable results.
Regarding Claim 9, Parrish, Vaughn and Gardner teach
The one or more processors of Claim 8.
Parrish and Vaughn did not specifically teach
wherein the feedback satisfies the predefined compliance, and wherein the training data comprises a plurality of feedback level examples corresponding with the plurality of examples of requirements and the plurality of examples of feedback.
Gardner further teaches
wherein the feedback satisfies the predefined compliance, and wherein the training data comprises a plurality of feedback level examples corresponding with the plurality of examples of requirements and the plurality of examples of feedback. (Gardner, Para. 0155, “The system may employ advanced machine learning techniques to optimize prompt engineering for precision control over the abstraction capabilities of large language models (LLMs). On innovation is a prompt engineering model that is trained to dynamically construct prompts tailored to the nuances of the input content and desired level of abstraction”) Examiner Comments: Training a model to construct prompts tailored to both the input content and the desired level requires training data in which each content example is paired with a corresponding level, which is the claimed plurality of feedback level examples corresponding with the examples of requirements and of feedback.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Gardner's practice of carrying a predefined output level in the prompt with the prompted requirements analyzer of Parrish and Vaughn in order to match the depth of the review comment to the requirement under review, which addresses the problem Gardner identifies at paragraph 0014 of existing approaches lacking sufficient precision and user configurability over the depth of a generated response. Adding a level parameter to a prompt that already carries the requirement, the criteria and the code context is the combination of prior art elements according to known methods to yield predictable results.
Regarding Claim 17, is a system claim corresponding to the claim above (Claim 8) and, therefore, is rejected for the same reasons set forth in the rejection of claim 8.
Regarding Claim 20, is a method claim corresponding to the claims above (Claims 2 and 8) and, therefore, is rejected for the same reasons set forth in the rejections of claims 2 and 8.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AMIR SOLTANZADEH whose telephone number is (571)272-3451. The examiner can normally be reached M-F, 9am - 5pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Wei Mui can be reached at (571) 272-3708. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AMIR SOLTANZADEH/Examiner, Art Unit 2191
/WEI Y MUI/Supervisory Patent Examiner, Art Unit 2191
Prosecution Insights