DETAILED ACTION
Claims 1-20 are pending.
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Examiner’s Notes
Examiner has cited particular columns and line numbers, paragraph numbers, or figures in the references as applied to the claims below for the convenience of the applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested from the applicant, in preparing the responses, to fully consider the references in entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the examiner.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 20 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 20 recites the limitation "presenting a plurality of different fix suggestion the each comprise different suggested changes to the source-code" in lines 2-3. There is insufficient antecedent basis for this limitation in the claim regarding “the each”. Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Jin et al. (InferFix: End-to-End Program Repair with LLMs over Retrieval-Augmented Prompts), March 13, 2023, hereinafter Jin, in further view of Liguori et al. (US-PAT-NO: 12,524,214 B1) hereinafter Liguori.
As per claim 1, Jin teaches a method implemented by a computing system for generating and processing fix suggestions for alerts produced by a code analysis tool processing source-code (see Jin Section 1, “In this paper, we propose InferFix: a transformer-based program repair framework paired with a state-of-the-art static analyzer to fix critical security and performance bugs”), the method comprising the computing system: receiving an alert produced by a code analysis tool in response to source-code being processed by the code analysis tool (see Jin Section 5, “(i) a static analysis tool that detects, localizes, and classified bugs”); parsing the alert to identify at least a problem type, an alert message, and a source-code location that are each associated with the alert (see Jin Section 2, “The context preprocessing module uses the information provided by the analyzer to extract the buggy method, and retains surrounding context most relevant to fixing the bug – import statements , class signature, body of the getResourceResolver method which is invoked at buggy line” and Section 3, “each bug in the dataset is associated with several pieces of metadata…Bug Type…Bug Location””); generating a prompt for a LLM, the prompt including:(i) a general description of the problem type, (ii) a description of the alert message (see Jin Section 2, “The retrieval augmentation engine then searches for semantically similar buggy code snippets in the historic database, prepending similar bug-fix to the prompt”), (iii) the source-code location, the source-code location comprising a subset of the source-code including a code snippet of lines of the source-code with corresponding line reference numbers of the source-code (see Jin Section 3.2 FIG. 2, “A buggy commit is detected by the Infer static analyzer, which is used to craft a prompt carrying the bug type annotation, location information, relevant syntax hierarchies (eWASH), and similar fixes retrieved from the historic database”), and (iv) a request for a proposed fix to the alert, the request including at least one of (a) a request for a natural language description of a proposed fix to the alert, (b) a request for a formal description of recommended code edits to apply in the proposed fix, and/or (c) a request for a list of dependencies to be added during implementation of the proposed fix (see Jin Section 4, “In the following , we explore several program repair baselines which are constructed around powerful LLMs (code-cushman-001 and text-davinci-003) for tasks of completing code, filling code in the middle, or generating a fix following a natural language instruction”); sending the prompt to the LLM (see Jin Section 2, “the augmented prompt is sent to the finetuned Codex model for inference”); obtaining a response to the prompt from the LLM comprising a proposed fix that includes a set of edits to be applied to the source-code and/or configuration files associated with the source-code (see Jin Section 3.2 FIG 2, “A LLM – finetuned 12B Codex model – generates a patch”); processing the response to the prompt (see Jin Section 2, “The predicted patch is then validated by executing the Infer static analyzer and unit tests as part of the CI pipeline to ensure the error is indeed fixed and no regressions are introduced in the code base”); validating the response to the prompt (see Jin Section 8, “the PR containing the candidate patch is validated through: (i) build – checking that the candidate patch is syntactically and semantically correct w.r.t. the source project; (ii) testing – ensuring that the candidate patch does not introduce regressions (failing tests)”); and assembling and providing a fix suggestion based on the response, the fix suggestion including (i) a natural language explanation of the proposed fix, (ii) suggested source-code changes (see Jin Section 4.3, “Davinci outputs a natural language summary of the proposed fix followed by a code snippet”).
Jin teaches in section 8, “The fix is then provided to the developer within the feature branch of the developer’s PR. This process allows the discard of invalid fixes suggested by the model (i.e., false positives), reducing the effort required by the developer to validate the candidates. The complexity of these stages is abstracted away from the developer, who will simple receive a PR comment within the system they are using (e.g., GitHub or Azure DevOps). We implemented a GitHub action which receives a validated patch from InferFix and surfaces it to the developer in form of a comment in the PR. The comment provides detailed information about the bug (i.e., extracted by Infer), and the resolution (i.e., served by InferFix) but does not explicitly teach (iii) selectable options for accepting, rejecting and editing the fix suggestion. However, Liguori teaches (iii) selectable options for accepting, rejecting and editing the fix suggestion (see Liguori [column 2, lines 42-51], “As described above, the SDS can help developers fix errors, bugs, and vulnerabilities in a software program. Specifically, the SDS can identify errors, propose a course of action to mitigate those errors, and mitigate those errors on behalf of the developer such as by recommending specific code changes. In some scenarios, a developer may get an error notification from another service or program and be able to select an option to resolve the error, which can call the SDS to propose the steps to fix the error and/or the mitigation for the error”).
Jin and Liguori are analogous art because they are in the same field of endeavor of software development. Therefore, it would have been obvious to one of ordinary skills before the effective filing date of the claimed invention to modify Jin’s teaching of InterFix an end-to-end solution for detection, classification, and localization of bugs, as well as fixing and validation of candidate patches, integrated in the continuous integration pipeline to automate the software development workflow with Liguori’s teaching of leveraging a large language model in software development to be prompted with an analysis of an error message to incorporate giving a developer an option to resolve an error when a software development service provides a fix to an error found when developing an application in order to propose a course of action to mitigate said error.
As per claim 2, Jin modified with Liguori teaches wherein in response to detecting user input selecting an option for accepting the fix suggestion, the method further includes automatically propagating changes to the source code that are identified in the fix suggestion (see Liguori [column 27, lines 29-39], “At operation 1712, the error resolution agent sends a prompt including the error message, the analysis received at operation 1706, and the suggested resolution received at operation 1710 to the LLM 1799, the prompt requesting descriptions of one or more source code changes to resolve the error. This prompt can be considered an “implementation” prompt to implement a change to correct a cause of the error message. Again, the prompt can further include various software system data such as a system map and source code of the software system as generated or obtained by context agents”).
As per claim 3, Jin modified with Liguori teaches the request for the proposed fix comprising a command for the LLM to provide a list of replacement blocks or replacement descriptors that show how snippets of the source code are to be replaced with new snippets of code (see Jin Section 2, “The retrieval augmentation engine then searches for semantically similar buggy code snippets in the historic database, prepending similar bug fixes to the prompt”).
As per claim 4, Jin modified with Liguori teaches the prompt also including supplementary source-code locations identified during the parsing and that are referenced in the alert message (see Jin Section 3.2 FIG 2, “A buggy commit is detected by the Infer static analyzer, which is used to craft a prompt carrying the bug type annotation, location information, relevant syntax hierarchies (eWASH) and similar fixed retrieved from the historic database” and also see Jin Sections 6-6.6).
As per claim 5, Jin modified with Liguori teaches the prompt also including a flow path identified during the parsing and that describes the flow of data of interest as an ordered list of source-code locations tagged with explanatory text, the flow path including the source code location (see Jin Section 5.1, “Our bug detection, localization, and classification module is powered by the Infer, which performs program analysis via Separation Logic. Although Infer’s Pulse framework has recently been released, for the purposes of this paper we examine bugs generated by Infer’s biabduction framework. Compiler frontends for Infer, such as InferSharp, translate source code into the control-flow-graph intermediate representation understood by Infer, known as the Smallfoot Intermediate Language. Infer performs automated program analysis over this graph and produces compositional method summaries in order to determine whether there are defects present in the source code” and also see Jin Sections 6-6.6).
As per claim 6, Jin modified with Liguori teaches the prompt also including documentation for the alert, identified during the parsing, explaining an issue being flagged and how to fix it in general terms (see Jin Section 5.3, “Specifically, the input to the model is the buggy code augmented with additional information such as bug localization and categorization, hierarchical extended context, and retrieved similar fixes” and also see Jin Sections 6-6.6).
As per claim 7, Jin modified with Liguori teaches the documentation for the alert including examples of how to fix a problem identified by the alert (see Jin Section 5.3, “Specifically, the input to the model is the buggy code augmented with additional information such as bug localization and categorization, hierarchical extended context, and retrieved similar fixes” and also see Jin Sections 6-6.6).
As per claim 8, Jin modified with Liguori teaches wherein the LLM that has not been specifically trained to fix vulnerabilities in code associated with the alert (see Jin Section 5.3, “Our generator is based on Codex Cushman, a 12B parameter decoder-only transformer language model developed by OpenAI, which is a descendant of GPT-3”; Particularly given the wide availability of LLMs such as GPT-3.5 at little or no cost, we believe our experimental results are reproducible with lesser access to high-end computing architecture, and with other LLMs”).
As per claim 9, Jin modified with Liguori teaches the response to the prompt also comprising a natural- language explanation of the proposed fix (see Jin section 8, “The fix is then provided to the developer within the feature branch of the developer’s PR. This process allows the discard of invalid fixes suggested by the model (i.e., false positives), reducing the effort required by the developer to validate the candidates. The complexity of these stages is abstracted away from the developer, who will simple receive a PR comment within the system they are using (e.g., GitHub or Azure DevOps). We implemented a GitHub action which receives a validated patch from InferFix and surfaces it to the developer in form of a comment in the PR. The comment provides detailed information about the bug (i.e., extracted by Infer), and the resolution (i.e., served by InferFix).
As per claim 10, Jin modified with Liguori teaches the response to the prompt also comprising metadata and/or links for third-party libraries that are recommended to use as dependencies in the proposed fix (see Liguori [column 16, lines 3-8, “Common formats for the map include JSON, YAML, UML, etc. In an example map, each component can include an identification and description of the APIs it vends as well as an identification of the APIs it calls. Thus, a component vending an API “product_order( )” would be a dependency to other components making calls to that API”).
As per claim 11, Jin modified with Liguori teaches the processing of the response comprising changing line numbers referenced in the set of edits in order to promote syntactic correctness of the suggested source-code changes (see Jin Section 8, “(i) build – checking that the candidate patch is syntactically and semantically correct w.r.t. the source project”).
As per claim 12, Jin modified with Liguori teaches the processing of the response comprising identifying configuration files to which to add dependencies, either entirely or partially, and adding edits to the identified configuration files to the fix suggestion (see Liguori [column 2, lines 52-65], “Further, as described above, the SDS can use AI such as an LLM to understand an application's architecture and the changes that developers want to make. As a first step, a developer can inform the SDS of where their application source code is located. The SDS then analyzes the source code, any documentation about the application, and information about the resources that comprise the application (ex: specific compute and storage resources) to understands the application code and architecture, for example by querying the APIs of other services to gather information about the application's resources and configuration. If there are areas of the task which are uncertain, the SDS can identify these areas and request clarification from the user before generating the development plan and proposed changes”).
As per claim 13, Jin modified with Liguori teaches wherein the processing of the response includes modifying the natural language explanation of the proposed fix (see Liguori [column 15, lines 21-35], “Response definitions 214 define how the SDS 110 will expect the response from the LLM to be formatted. Response definitions 214 can be used to regularize the responses from LLMs to improve the ability of the SDS 110 to parse those responses such that they can be stored, trigger follow on actions, etc. Example response definitions include instructing the LLM to respond in natural language forms such as with a Yes or No, a list of items, an enumerated list of items, etc. and also to respond with more structured forms (e.g., with Python code, with an SQL query, with a JSON object, etc.) Note that the interpretation of responses pursuant to response definitions is typically contingent on the phrasing of a prompt, tailored within a given agent (e.g., a negative response might indicate a pass for one prompt, a failure for another)”).
As per claim 14, Jin modified with Liguori teaches the method further comprising: obtaining dependency metadata for any suggested dependency included in the response, the metadata comprising version information, security vulnerability information, and statistic information corresponding to quality and dependability of the suggested dependency (see Liguori [column 31, lines 44-53], “FIGS. 22 through 26 illustrate a progression of changes to a software system with the software development service according to some examples. The graphical user interface 2100 of FIG. 21 can be used to display visual differences between states (or versions) of a software system. To illustrate these visual differences, FIGS. 22 through 26 illustrate various visual differences associated with the incremental actions. Such actions may be created by the user or as part of an action plan, such as the one obtained from an LLM by a development task agent as described above”); and including the dependency metadata in the fix suggestion as at least one of (i) a natural language description of the metadata or (ii) a selectable link to the metadata that is operable, when selected at an interface, to redirect the interface to a source of the metadata (see Liguori [column 24, lines 31-42], “For example, a link to a code repository having a format identifying the code repository (e.g., github.com/project-name), which correlates with the code repository context aggregator. As another example, a provided reference may be of a certain file or resource type uniquely associated with a particular context aggregator. The development task agent can send a prompt to an LLM requesting that the responses be matched to the API(s) of the context aggregators in the form of API calls. The development task agent can try to invoke context aggregators with those LLM-provided calls, iteratively prompting the LLM to refine a call if the call failed”).
As per claim 15, Jin modified with Liguori teaches wherein the request for the proposed fix is provided in a plurality of different prompts, each of the plurality of different prompts including a request for a different one of (a) a natural language description of a proposed fix to the alert, (b) a formal description of recommended code edits to apply in the proposed fix, and/or (c) the list of dependencies to be added during implementation of the proposed fix (see Liguori [column 22, lines 13-23], “At operation 1032, the system design agent sends a prompt including the task prompt (from operation 1002) and the stored questions and answers (whether inferred obtained data about software systems or solicited from the client) the prompt requesting the LLM to generate a proposed solution (or multiple proposed solutions). At operation 1034, the system design agent receives the response. At operation 1036, the system design agent sends the proposed solution(s) to the client 1001, which can be part of a prompt that includes a solicitation of additional questions about the proposed solution(s)”).
As per claim 16, Jin modified with Liguori teaches wherein the method further includes: obtaining, processing and validating a plurality of responses from the LLM corresponding to different prompts and prior to assembling and presenting the fix suggestion that is further based at least in part on the plurality of responses (see Liguori [column 22, lines 13-23], “At operation 1032, the system design agent sends a prompt including the task prompt (from operation 1002) and the stored questions and answers (whether inferred obtained data about software systems or solicited from the client) the prompt requesting the LLM to generate a proposed solution (or multiple proposed solutions). At operation 1034, the system design agent receives the response. At operation 1036, the system design agent sends the proposed solution(s) to the client 1001, which can be part of a prompt that includes a solicitation of additional questions about the proposed solution(s)”).
As per claim 17, Jin modified with Liguori teaches wherein providing the fix suggestion comprises displaying the fix suggestion at an interface (see Liguori [column 3, lines 1-5], “With this information, the SDS can suggest code and architectural changes and may present these graphically using an interactive canvas to visually convey the ramifications of a change, enabling developers to complete their tasks in a fraction of the time”).
As per claim 18, Jin modified with Liguori teaches wherein providing the fix suggestion comprises sending the fix suggestion as a file to a remote system (see Jin Section 3, “We executed Infer and InferSharp over the change histories of approximately 6.2k Java and C# open-source repositories (2.9k Java, 3.3k C#) hosted on GitHub, analyzing more than 1 million commits”).
As per claim 19, Jin modified with Liguori teaches wherein the LLM comprises a generative language model (see Liguori [column 4, lines 22-28], “Accordingly, disclosed are methods, apparatus, systems, and non-transitory computer-readable storage media for a SDS. The SDS can assist with a variety of software development efforts, including for complex tasks that involve multi-step reasoning or require large amounts of user-specific context, by leveraging generative artificial intelligence (GAI) models such as large language models (LLMs)”).
As per claim 20, Jin modified with Liguori teaches wherein the response comprises a plurality of proposed fixes and the assembling and providing the fix suggestion comprises presenting a plurality of different fix suggestions the each comprise different suggested changes to the source-code (see Liguori [column 22, lines 13-23], “At operation 1032, the system design agent sends a prompt including the task prompt (from operation 1002) and the stored questions and answers (whether inferred obtained data about software systems or solicited from the client) the prompt requesting the LLM to generate a proposed solution (or multiple proposed solutions). At operation 1034, the system design agent receives the response. At operation 1036, the system design agent sends the proposed solution(s) to the client 1001, which can be part of a prompt that includes a solicitation of additional questions about the proposed solution(s)”).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Ou et al. (US-PGPUB-NO: 2025/0245349 A1) teaches custom AI co-pilot for software security pen-testing.
Vembu et al. (US-PGPUB-NO: 2024/0281224 A1) teaches permissions and notifications for construct-modification tags.
Arumugam Selvaraj et al. (US-PGPUB-NO: 2023/0418565 A1) teaches validating and providing proactively generated code suggestions.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LENIN PAULINO whose telephone number is (571)270-1734. The examiner can normally be reached Week 1: Mon-Thu 7:30am - 5:00pm Week 2: Mon-Thu 7:30am - 5:00pm and Fri 7:30am - 4:00pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Bradley Teets can be reached at (571) 272-3338. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/LENIN PAULINO/Examiner, Art Unit 2197
Prosecution Insights