Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments and amendments filed August 14, 2025 regarding objections to the claims, see C. Claim Objections, have been fully considered and are persuasive. As argued, the claims have been amended to correct minor informalities indicated in the previous Office Action. The objections to the claims are withdrawn. However, a new objection is made regarding new claims added in amendments.
Applicant's arguments filed August 14, 2025 regarding rejections under 35 U.S.C. 101 and 103 have been fully considered but they are not persuasive.
Regarding the arguments presented in D. Rejection Under 35 U.S.C. § 101, applicant submits that the claims, as amended, are not directed to an abstract idea. As indicated in the interview held July 14, 2025, the amendments to the claim do not overcome the prior rejection under 35 U.S.C. § 101. Applicant argues that the amended claims include a series of acts that, when considered as a whole, provide an ordered combination of limitations directed to overcoming technical problems as a result of using LLMs in generating reliable and valid modeled components of a protocol. Particularly, Applicant notes the generating of the symbolic harness and application of the symbolic execution engine. The examiner respectfully disagrees. The claims do not recite any limitations which represent an improvement on the claimed technologies.
Regarding the limitation of applying a symbolic execution engine based on the symbolic harness, the claims merely recite details on the particular task on which the technology is to be applied, and does not indicate technical problems purported to be overcome by the claimed invention. The limitation “the symbolic harness being configured to initialize each of the one or more symbolic function parameters by constructing symbolic values based on identified inputs for functions of the modeled component of the protocol” introduced in the amendment is a conventional, routine, well-understood aspect of symbolic execution as taught by Song pages 3 – 5 (NPL, cited in previous action). As such, the amended claims do not represent an improvement on the claimed technologies.
Regarding the limitation of generating the symbolic harness, as indicated in the previous rejection, generating the symbolic harness is a mental process abstract idea which can be practically performed in the human mind. The amendment does not introduce a limitation which would preclude this generation from being performed in the human mind, as the symbolic harness is, under the broadest reasonable interpretation, code in the form of text. The amendment introduces a limitation on the expected behavior of the generated code when it is run, but that does not affect the feasibility of generating the code in the human mind. In arguments, Applicant submits that the examiner’s assertion that a human mind with sufficient skill in writing code could easily perform the limitations of claim 1 is inaccurate, and cites page 2 of the USPTO memo regarding subject matter eligibility. The examiner respectfully disagrees. The cited portion of the memo is a reminder that a claim does not recite a mental process when it contains limitations that cannot be practically performed in the human mind. The claims as written do not recite any limitations on generating the symbolic harness which cannot be practically performed in the human mind, except that the abstract idea is performed by the LLM, which, as indicated in the previous rejection, amounts to mere instructions to perform the abstract idea on a computer. Drawings of an example symbolic harness in Fig. 3 clearly indicate that generating the symbolic harness can be practically performed in the human mind.
Furthermore, the arguments fail to directly address the recited abstract idea of generating a model generation prompt. The claims do not provide an indication that a human being could not write a prompt which would cause the LLM to produce the desired symbolic harness. The prompts include a description and instructions for a task. Even if the claimed task could not be practically be performed by a human, as argued regarding the generation of the symbolic harness, a human would still be able to describe instructions to perform it. Drawings of example prompts in Figs. 5A and 5B clearly indicate that the prompt could practically be generated in the human mind. The claims do not appear to integrate this abstract idea into a practical application or amount to significantly more.
An analysis of the amended claims regarding 35 U.S.C. § 101 is included below.
Regarding the arguments presented in E. Prior Art Rejections, regarding rejections under 35 U.S.C. § 103, applicant submits that, while Song appears to describe the concept of symbolic execution, it does not teach or suggest the entire limitation regarding generating a symbolic harness based on the model generation prompt. Applicant further submits that Kanazawa describes LLM code generation at a general level, and does not suggest the claimed subject matter, and as such the combination of Song in view of Kanazawa do not suggest the recited subject matter. The examiner respectfully disagrees. The test for obviousness is not whether the features of a secondary reference may be bodily incorporated into the structure of the primary reference; nor is it that the claimed invention must be expressly suggested in any one or all of the references. Rather, the test is what the combined teachings of the references would have suggested to those of ordinary skill in the art. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981). The invention as claimed does not contain limitations that would not be obvious to one of ordinary skill in the art before the effective filing date of the invention given knowledge of the particular software engineering task taught by Song in view of the market incentive to apply LLM coding to software engineering tasks presented in Kanazawa.
A rejection of the amended claims regarding 35 U.S.C. § 103 is included below.
Specification
The use of the term ‘docker’, which is a trade name or a mark used in commerce, has been noted in this application. The term should be accompanied by the generic terminology; furthermore the term should be capitalized wherever it appears or, where appropriate, include a proper symbol indicating use in commerce such as ™, SM , or ® following the term.
Although the use of trade names and marks used in commerce (i.e., trademarks, service marks, certification marks, and collective marks) are permissible in patent applications, the proprietary nature of the marks should be respected and every effort made to prevent their use in any manner which might adversely affect their validity as commercial marks.
Claim Objections
Claim 22 objected to because of the following informalities: “docker” is not capitalized. Appropriate correction is required.
Claim Rejections - 35 USC § 101
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claims 1 – 17 and 21 – 23 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Below is an evaluation using the 2019 Revised Patent Subject Matter Eligibility Guidance.
Claim 1
Step 1:
Claim 1 is to a method.
Step 2A Prong 1: Abstract Idea
Claim 1 recites
generating a model generation prompt including a description of a protocol and instructions associated with generating protocol models
obtaining a modeled component of the protocol, the modeled component of the protocol including executable code generated … based on the model generation prompt
generating a symbolic harness based on the model generation prompt and one or more validity constraints associated with inputs to be handled by the modeled component output
which are abstract ideas of mental processes that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III). As clearly shown in Fig. 3, under the broadest reasonable interpretation of the claims, the claimed prompt, modeled component, and symbolic harness are all code in the form of text which can be generated and obtained in the human mind. As stated in the previous rejection, while the use of an LLM in the claims itself is not an abstract idea, its presence indicates that the functions performed by an LLM can practically be performed in the human mind, as LLMs are designed to performed natural language processing tasks.
Step 2A Prong 2: Additional elements
Claim 1 recites
providing the model generation prompt as an input to a large language model (LLM);
…generated by the LLM based on the model generation prompt provided as input to the LLM;
…by the LLM…
which are additional elements that amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP §§ 2106.04(d), 2106.05(f)(1). As recited, the claims are at a high level of generality regarding the LLM itself such that they do not amount to more than the words “apply it”.
Claim 1 further recites
the symbolic harness being configured to initialize each of one or more symbolic function parameters by constructing symbolic values based on identified inputs for functions of the modeled component of the protocol; and
applying a symbolic executing engine to the modeled component of the protocol based on the symbolic harness to generate a plurality of protocol tests associated with the modeled component of the protocol and which satisfy the one or more validity constraints of the symbolic harness
which are additional elements that generally link the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
They are also additional elements that amount to adding insignificant extra-solution activity to the judicial exception. Applying a symbolic execution engine to a symbolic test harness is a necessary part of its intended use. See MPEP §§ 2106.04(d), 2106.05(g). The additional elements are well‐understood, routine, and conventional with respect to symbolic execution as taught by Song (Pages 3 – 5) and as indicated by the Applicant (Paragraphs 0047 – 0050 of the specification cite KLEE and LLVM without prior explanation, indicating that they are presumed to be well-known in the art). See MPEP § 2106.05(d).
Step 2B: Significantly more
Claim 1 recites
providing the model generation prompt as an input to a large language model (LLM);
…generated by the LLM based on the model generation prompt provided as input to the LLM;
…by the LLM…
which are additional elements that amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP § 2106.04(d). As recited, the claims are at a high level of generality regarding the LLM itself such that they do not amount to more than the words “apply it”.
Claim 1 further recites
the symbolic harness being configured to initialize each of one or more symbolic function parameters by constructing symbolic values based on identified inputs for functions of the modeled component of the protocol; and
applying a symbolic executing engine to the modeled component of the protocol based on the symbolic harness to generate a plurality of protocol tests associated with the modeled component of the protocol and which satisfy the one or more validity constraints of the symbolic harness
which are additional elements that generally link the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.04(d).
They are also additional elements that amount to adding insignificant extra-solution activity to the judicial exception. Applying a symbolic execution engine to a symbolic test harness is a necessary part of its intended use. See MPEP § 2106.04(d). The additional elements are well‐understood, routine, and conventional with respect to symbolic execution as taught by Song (Pages 3 – 5) and as indicated by the Applicant (Paragraphs 0047 – 0050 of the specification cite KLEE and LLVM without prior explanation, indicating that they are presumed to be well-known in the art). See MPEP § 2106.05(d).
Claim 2
Step 1:
Claim 2 is to a method.
Step 2A Prong 1: Abstract Idea
Claim 2 recites the abstract ideas of Claim 1 by dependency.
Step 2A Prong 2: Additional elements
Claim 2 recites
further comprising executing the plurality of protocol tests using a plurality of protocol implementations to determine if one or more of the plurality of protocol implementations has protocol-specific issues.
This is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g). The additional element is well‐understood, routine, and conventional with respect to symbolic execution as taught by Song (Sections 5 and 6, testing interoperability and multiple protocols is performed in order to confirm functionality for real-life applications which may utilize a plurality of protocol implementations). See MPEP § 2106.05(d).
Step 2B: Significantly more
Claim 2 recites
further comprising executing the plurality of protocol tests using a plurality of protocol implementations to determine if one or more of the plurality of protocol implementations has protocol-specific issues.
This is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP § 2106.04(d). The additional element is well‐understood, routine, and conventional with respect to symbolic execution as taught by Song (Sections 5 and 6, testing interoperability and multiple protocols is performed in order to confirm functionality for real-life applications which may utilize a plurality of protocol implementations). See MPEP § 2106.05(d).
Claim 3
Step 1:
Claim 3 is to a method.
Step 2A Prong 1: Abstract Idea
Claim 3 recites the abstract ideas of Claim 1 by dependency.
Claim 3 further recites
wherein the model generation prompt includes definitions and function arguments of the protocol.
This is a continuation of the generating a model generation prompt abstract idea of claim 1. A human being could easily include a description of a protocol and associated instructions under the broadest reasonable interpretation of the claims.
Step 2A Prong 2: Additional elements
Claim 3 does not recite additional elements.
Step 2B: Significantly more
Claim 3 does not recite additional elements.
Claim 4
Step 1:
Claim 4 is to a method.
Step 2A Prong 1: Abstract Idea
Claim 4 recites the abstract ideas of Claim 1 by dependency.
Step 2A Prong 2: Additional elements
Claim 4 recites
wherein the plurality of protocol tests includes an exhaustive suite of test cases that follow a plurality of paths through the modeled component of the protocol based on information contained within the symbolic harness
This is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g). The additional element is well‐understood, routine, and conventional with respect to symbolic execution as taught by Song (Section 2.2, when execution reaches a branch depending on symbolic input, both paths are followed). See MPEP § 2106.05(d).
Step 2B: Significantly more
Claim 4 recites
wherein the plurality of protocol tests includes an exhaustive suite of test cases that follow a plurality of paths through the modeled component of the protocol based on information contained within the symbolic harness
This is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP § 2106.04(d). The additional element is well‐understood, routine, and conventional with respect to symbolic execution as taught by Song (Section 2.2, when execution reaches a branch depending on symbolic input, both paths are followed). See MPEP § 2106.05(d).
Claim 5
Step 1:
Claim 5 is to a method
Step 2A Prong 1: Abstract Idea
Claim 5 recites the abstract ideas of Claim 1 by dependency.
Claim 5 recites
based on input and output definitions and descriptions included within the model generation prompt.
This is a continuation of the model generation abstract idea of claim 1. As stated with respect to claim 1, given the drawings of the model in Fig. 3, a human being could, under the broadest reasonable interpretation of the claims, write code based on input and output definitions and descriptions.
Step 2A Prong 2: Additional elements
Claim 5 recites
wherein the modeled component is executable C-code
This is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g). The examiner takes official notice that C code is a well-known programming language commonly used in the art. The additional element is well‐understood, routine, and conventional. See MPEP § 2106.05(d).
This may alternatively be interpreted as a continuation of the model generation abstract idea of claim 1, as C is a human-readable programming language which can be practically represented in the human mind.
Step 2B: Significantly more
Claim 5 recites
wherein the modeled component is executable C-code
This is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP § 2106.04(d). The examiner takes official notice that C code is a well-known programming language commonly used in the art. The additional element is well‐understood, routine, and conventional. See MPEP § 2106.05(d).
This may alternatively be interpreted as a continuation of the model generation abstract idea of claim 1, as C is a human-readable programming language which can be practically represented in the human mind.
Claim 6
Step 1:
Claim 6 is to a method
Step 2A Prong 1: Abstract Idea
Claim 6 recites the abstract ideas of Claim 1 by dependency.
Step 2A Prong 2: Additional elements
Claim 6 recites
wherein the symbolic harness is a type-specific symbolic harness in C-code in which symbolic inputs are generated for the modeled component of the protocol.
This is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g). The additional element is well‐understood, routine, and conventional, as taught by Song (Section 4 teaches that rules and variables in the harness can be type-specific in order to follow certain requirements; The introduction teaches that the harness is in C code; Section 2.2 and 3 teach that the inputs are symbolic inputs generated for the modeled protocol). See MPEP § 2106.05(d).
Step 2B: Significantly more
Claim 6 recites
wherein the symbolic harness is a type-specific symbolic harness in C-code in which symbolic inputs are generated for the modeled component of the protocol.
This is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP § 2106.04(d). The additional element is well‐understood, routine, and conventional, as taught by Song (Section 4 teaches that rules and variables in the harness can be type-specific in order to follow certain requirements; The introduction teaches that the harness is in C code; Section 2.2 and 3 teach that the inputs are symbolic inputs generated for the modeled protocol). See MPEP § 2106.05(d).
Claim 7
Step 1:
Claim 7 is to a method.
Step 2A Prong 1: Abstract Idea
Claim 7 recites the abstract ideas of Claim 1 by dependency.
Claim 7 further recites
wherein the model generation prompt includes a library that provides restrictions on one or more types of input that the model is configured to receive and a type of output that the modeled component of the protocol is configured to output.
This is a continuation of the generating a model generation prompt abstract idea of claim 1. A human being could easily describe the desired input and output behavior of the model under the broadest reasonable interpretation of the claims.
Step 2A Prong 2: Additional elements
Claim 7 does not recite additional elements.
Step 2B: Significantly more
Claim 7 does not recite additional elements.
Claim 8
Step 1:
Claim 8 is to a method.
Step 2A Prong 1: Abstract Idea
Claim 8 recites the abstract ideas of Claim 1 by dependency.
Claim 8 further recites
wherein the model generation prompt includes a completion prompt that frames an implementation task as a completion problem and a system prompt that guides behavior of the LLM in generating the modeled component of the protocol.
This is a continuation of the generating a model generation prompt abstract idea of claim 1. A human being could easily frame a task as a problem and describe the desired behavior of the LLM under the broadest reasonable interpretation of the claims.
Step 2A Prong 2: Additional elements
Claim 8 does not recite additional elements.
Step 2B: Significantly more
Claim 8 does not recite additional elements.
Claim 9
Step 1:
Claim 9 is to a method.
Step 2A Prong 1: Abstract Idea
Claim 9 recites the abstract ideas of Claim 1 by dependency.
Step 2A Prong 2: Additional elements
Claim 9 recites
wherein the modeled component of the protocol includes a subset of protocol behaviors of the protocol.
This is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g). The additional element is well‐understood, routine, and conventional as admitted by the Applicant (Paragraph 0015 of the specification describes protocol testing and specifically model-based testing as conventional. Paragraphs 0022 and 0023 note the definitions for protocol and protocol model used, which include a subset of behaviors of the protocol). See MPEP § 2106.05(d).
This may alternatively be interpreted as a continuation of the model generation abstract idea of claim 1, as the example protocol model in Fig. 3 is a subset of behaviors which can be created in the human mind.
Step 2B: Significantly more
Claim 9 recites
wherein the modeled component of the protocol includes a subset of protocol behaviors of the protocol.
This is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP § 2106.04(d). The additional element is well‐understood, routine, and conventional as admitted by the Applicant (Paragraph 0015 of the specification describes protocol testing and specifically model-based testing as conventional. Paragraphs 0022 and 0023 note the definitions for protocol and protocol model used, which include a subset of behaviors of the protocol). See MPEP § 2106.05(d).
This may alternatively be interpreted as a continuation of the model generation abstract idea of claim 1, as the example protocol model in Fig. 3 is a subset of behaviors which can be created in the human mind.
Claim 10
Step 1:
Claim 10 is to a method.
Step 2A Prong 1: Abstract Idea
Claim 10 recites the abstract ideas of Claim 1 by dependency.
Step 2A Prong 2: Additional elements
Claim 10 recites
wherein the protocol is one or more of Quick UDP Internet Connections (QUIC) protocol, Domain Names System (DNS) protocol, or Border Gateway Patrol (BGP) protocol.
which is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h). The claims do not link the protocols to the abstract idea, and the additional element is well‐understood, routine, and conventional as taught by Song (Sections 1 and 2 indicate DNS is a commonly-used network protocol) and as indicated by the Applicant (Paragraph 0022 of the specification lists these protocols as examples without further explanation). See MPEP § 2106.05(d).
Step 2B: Significantly more
Claim 10 recites
wherein the protocol is one or more of Quick UDP Internet Connections (QUIC) protocol, Domain Names System (DNS) protocol, or Border Gateway Patrol (BGP) protocol.
which is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.04(d). The claims do not link the protocols to the abstract idea, and the additional element is well‐understood, routine, and conventional as taught by Song (Sections 1 and 2 indicate DNS is a commonly-used network protocol) and as indicated by the Applicant (Paragraph 0022 of the specification lists these protocols as examples without further explanation). See MPEP § 2106.05(d).
Claims 11 – 15
Step 1:
Claims 11 – 15 are to a machine.
Step 2A Prong 1: Abstract Idea
Claims 11 – 15 recite similar language to claims 1 – 5, respectively, and as such recite similar abstract ideas.
Step 2A Prong 2: Additional elements
Claim 11, and 12 – 15 by dependency, recite
at least one processor;
memory in electronic communication with the at least one processor; and
instructions stored in the memory, the instructions being executable by the at least one processor
which are additional elements that amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP §§ 2106.04(d), 2106.05(f)(1).
Claims 11 – 15 otherwise recite similar language to claims 1 – 5, respectively, and as such do not recite additional elements which integrate the abstract ideas into a practical application.
Step 2B: Significantly more
Claim 11, and 12 – 15 by dependency, recite
at least one processor;
memory in electronic communication with the at least one processor; and
instructions stored in the memory, the instructions being executable by the at least one processor
which are additional elements that amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP § 2106.04(d).
Claims 11 – 15 otherwise recite similar language to claims 1 – 5, respectively, and as such do not recite amount to significantly more.
Claims 16 and 17
Step 1:
Claims 16 and 17 are to a non-transitory computer-readable storage medium.
Step 2A Prong 1: Abstract Idea
Claims 16 and 17 recite similar language to claim 1, and claims 6 and 7, respectively, and as such recite similar abstract ideas.
Step 2A Prong 2: Additional elements
Claim 16, and claim 17 by dependency, recite
A non-transitory computer readable medium storing instructions thereon that, when executed by at least one processor,
which are additional elements that amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP §§ 2106.04(d), 2106.05(f)(1).
Claims 16 and 17 otherwise recite similar language to claim 1, and claims 6 and 7, respectively, and as such do not recite additional elements which integrate the abstract ideas into a practical application.
Step 2B: Significantly more
Claim 16, and claim 17 by dependency, recite
A non-transitory computer readable medium storing instructions thereon that, when executed by at least one processor,
which are additional elements that amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP § 2106.04(d).
Claims 16 and 17 otherwise recite similar language to claim 1, and claims 6 and 7, respectively, and as such do not recite amount to significantly more.
Claim 21
Step 1:
Claim 21 is to a method.
Step 2A Prong 1: Abstract Idea
Claim 21 recites the abstract ideas of Claim 1 by dependency.
Step 2A Prong 2: Additional elements
Claim 21 recites
wherein generating the plurality of protocol tests includes invoking, by the symbolic execution engine, a compiler and a plurality of commands to instruct the symbolic execution engine in generating code for extracting protocol test cases for testing functionality of the modeled component of the protocol.
This is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g). The additional element is well‐understood, routine, and conventional aspect of symbolic execution testing as taught by Song (Section 3) and as indicated by the Applicant (Paragraph 0048 provides a C compiler, KLEE commands and LLVM bytecode as examples without further explanation, indicating they are presumed to be known in the art). See MPEP § 2106.05(d).
Step 2B: Significantly more
Claim 21 recites
wherein generating the plurality of protocol tests includes invoking, by the symbolic execution engine, a compiler and a plurality of commands to instruct the symbolic execution engine in generating code for extracting protocol test cases for testing functionality of the modeled component of the protocol.
This is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP § 2106.04(d). The additional element is well‐understood, routine, and conventional aspect of symbolic execution testing as taught by Song (Section 3) and as indicated by the Applicant (Paragraph 0048 provides a C compiler, KLEE commands and LLVM bytecode as examples without further explanation, indicating they are presumed to be known in the art). See MPEP § 2106.05(d).
Claim 22
Step 1:
Claim 22 is to a method.
Step 2A Prong 1: Abstract Idea
Claim 22 recites the abstract ideas of Claim 1 by dependency.
Step 2A Prong 2: Additional elements
Claim 22 recites
wherein a first task of compiling protocol test cases and a second task of executing the plurality of protocol tests are performed in separate docker containers
This is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g). The additional element is well‐understood, routine, and conventional as taught by Delimat (NPL, A complete guide to end-to-end API testing with Docker), which describes Docker, and separating tests and builds within Docker, as an advantageous method for testing. KLEE, which is used in Song and the application, is commonly used through Docker as taught by Korczynski (NPL, Symbolic execution with KLEE: From installation and introduction to bug-finding in open source software). See MPEP § 2106.05(d). Additionally the limitation modifies the extra-solution activity of executing test cases in a way that does not integrate the abstract ideas of writing the prompts and writing the harness code.
Step 2B: Significantly more
Claim 22 recites
wherein a first task of compiling protocol test cases and a second task of executing the plurality of protocol tests are performed in separate docker containers
This is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP § 2106.04(d). The additional element is well‐understood, routine, and conventional as taught by Delimat which describes Docker, and separating tests and builds within Docker, as an advantageous method for testing. KLEE, which is used in Song and the application, is commonly used through Docker as taught by Korcynski. See MPEP § 2106.05(d).
Claim 23
Step 1:
Claim 23 is to a method.
Step 2A Prong 1: Abstract Idea
Claim 23 recites the abstract ideas of Claims 1 and 21 by dependency.
Claim 23 recites
updating the LLM in generating subsequent modeled components of the protocol based on the logged one or more compiler errors.
In light of paragraph 0048 of the Specification, particularly “Users may use this feedback to update models of descriptions to the LLM”, this limitation is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III). The support in the specification for performing this update to the LLM is this recitation of updating the description, which is the mental process of rewriting the description.
Step 2A Prong 2: Additional elements
Claim 23 recites
logging one or more compiler errors from the compiler based on executing the plurality of protocol tests
This is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g). The additional element is well‐understood, routine, and conventional aspect of software engineering, as taught by the Wikipedia article for Compilation error (NPL, most recent version before effective filing date) in the header, wherein compilation error messages are known to help programmers debug source code. See MPEP § 2106.05(d).
Step 2B: Significantly more
Claim 23 recites
logging one or more compiler errors from the compiler based on executing the plurality of protocol tests
This is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP § 2106.04(d). The additional element is well‐understood, routine, and conventional aspect of software engineering, as taught by the Wikipedia article for Compilation error, in the header, wherein compilation error messages are known to help programmers debug source code. See MPEP § 2106.05(d).
Claim Rejections - 35 USC § 103
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claims 1 – 17 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Song (cited in previous action) in view of Kanazawa (cited in previous action).
Regarding claim 1, Song teaches:
A method for generating protocol tests (Introduction), the method comprising:
obtaining a modeled component of a protocol, the modeled component of the protocol including executable code (Introduction, the network protocol implementation including C source code);
generating a symbolic harness based on one or more validity constraints (3.1 SymbexNet Overview, creation of packet rules and generation of test packets), the symbolic harness being configured to initialize each of one or more symbolic function parameters by constructing symbolic values based on identified inputs for functions of the modeled component of the protocol (3.2 Symbolic Execution and Exploration, the symbolic packets are injected into the network protocol);
and applying a symbolic execution engine to the modeled component of the protocol based on the symbolic harness to generate a plurality of protocol tests associated with the modeled component of the protocol and which satisfy the one or more validity constraints of the symbolic harness (3.1 SymbexNet Overview step 2, a symbolic execution engine is applied to the modelled protocol. This applies a plurality of test packets which, referring to steps 1 and 4, satisfy the validity constraints of the symbolic harness).
Song does not explicitly recite generating a model generation prompt, providing the prompt to an LLM, nor that the modeled component is generated by the LLM (Song is focused on the application of symbolic execution to protocol code, rather than the creation of the protocol code itself).
Kanazawa teaches generating a generation prompt including a description of code and instructions associated with generating code (Tell It: Guide the Model with a High Level Task Description);
providing the generation prompt as an input to a large language model (LLM) (Page 1);
obtaining code, the modeled code including executable code generated by the LLM based on the generation prompt provided as the input to the LLM (Pages 1 and 2); and
generating code based on the generation prompt and one or more validity constraints associated with inputs to be handled by the code output by the LLM (Pages 6 and 7, the prompt can include constraints, and the LLM will generate code following those constraints).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention that the protocol code being tested by symbolic execution and the symbolic harness of Song could be generated by the LLM of Kanazawa. There is a clear market force prompting adoption of LLM-generated code in the field of software engineering, as demonstrated by the promotional nature of Kanazawa and endorsement by a head of AI at a major company (Kanazawa, Prompt Engineering as “Software 3.0”/Getting Access to Codex, Copilot, and OpenAI). The differences between Song and the claimed invention are encompassed by supplementing the existing network protocol symbolic testing task by replacing human-written code with LLM-generated code. One of ordinary skill in the art could easily prompt an LLM to generate code for any describable task, including generation of protocol code and a test harness, with predictable results. In addition to the market force prompting use of LLM code generation, one of ordinary skill in the art would also be motivated to use LLMs to reduce the amount of coding they would perform themselves.
Regarding claim 2, Song in view of Kanazawa teaches the method of claim 1, further comprising executing the plurality of protocol tests using a plurality of protocol implementations to determine if one or more of the plurality of protocol implementations has protocol specific issues (Song, 5. Inoperability Testing and 6. Evaluation, multiple implementations of the protocol are tested to find flaws in the particular protocols, as well as interoperability issues).
Regarding claim 3, Song in view of Kanazawa teaches the method of claim 1, wherein the model generation prompt includes definitions and functional arguments of the protocol (Kanazawa pages 5 and 7, examples of definitions and functional arguments are provided as part of the prompt, and an API including definitions and functional arguments is provided to it; Song Fig. 5, the protocol implementation includes definitions and functional arguments).
Regarding claim 4, Song in view of Kanazawa teaches the method of claim 1, wherein the plurality of protocol tests includes an exhaustive set of test cases that follow a plurality of paths through the modeled component of the protocol based on information contained within the symbolic harness (Song page 1, final paragraph; “SymbexNet generates an exhaustive set of input packets that achieve a broad and deep exploration of the system space”).
Regarding claim 5, Song in view of Kanazawa teaches the method of claim 1, wherein the modeled component is executable C-code (Song Introduction, the protocol implementation is C code) based on the input and output definitions and descriptions included within the model generation prompt (Kanazawa, Show It: Guide the Model with Examples).
Regarding claim 6, Song in view of Kanazawa teaches the method of claim 1, wherein the symbolic harness is a type-specific (Song 4.1 Rule Extraction, 4.2 Rule-Based Packet Stream Language, rules and variables can be type-specific, as certain fields may need to follow certain rules/requirements) symbolic harness in C-code (Song introduction, the network code is in C; Song 3.2 Symbolic Execution and Exploration, 3.3 Single-Packet Exchange Symbolic Execution (SPE-SE), the symbolic execution engine used is KLEE, a symbolic execution engine for C) in which symbolic inputs are generated for the modeled component of the protocol (Song 2.2 Symbolic Execution/3 SymbexNet Design, the generated test input packets are symbolic inputs for the modeled protocol component).
Regarding claim 7, Song in view of Kanazawa teaches the method of claim 1, wherein the model generation prompt includes a library that provides restrictions in one or more types of input that the model is configured to receive and a type of output that the modeled component of the protocol is configured to output (Kanazawa page 7 includes a library/API with defined inputs and outputs; Song 3.1 SymbexNet Overview, protocol requirements include definition of input-output behavior).
Regarding claim 8, Song in view of Kanazawa teaches the method of claim 1, wherein the model generation prompt includes a completion prompt that frames an implementation task as a completion problem (Kanazawa Tell It: Guide the Model with a High Level Task Description, the high level task description) and a system prompt that guides behavior of the LLM in generating the modeled component of the protocol (Kanazawa pages 5 – 10, high level context, examples, and conversational history).
Regarding claim 9, Song in view of Kanazawa teaches the method of claim 1, wherein the modeled component of the protocol includes a subset of protocol behaviors of the protocol (Song 3.3 Single-Packet Exchange Symbolic Execution (SPE-SE) and 3.4 Multi-Packet Exchange Symbolic Execution (MPE-SE), methods for modelling two different subsets of protocol behavior are described).
Regarding claim 10, Song in view of Kanazawa teaches the method of claim 1, wherein the protocol is one or more of Quick UDP Internet Connections (QUIC) protocol, Domain Name System (DNS) protocol, or Border Gateway Patrol (BGP) protocol (Song Introduction, the protocol can be DNS).
Regarding claim 11, Song and Kanazawa teach at least one processor (Song 3.2 Symbolic Execution and Exploration/3.2 Single-Packet Exchange Symbolic Execution (SPE-SE), SymbexNet performs processing).
They do not explicitly recite:
memory in electronic communication with the at least one processor; and
instructions stored in the memory, the instructions being executable by the at least one processor (Song and Kanazawa are non-patent literature and are thus focused on describing the systems rather than the hardware which implements them).
The examiner takes official notice that computer systems, AI models, and computer processes such as those described in Song and Kanazawa are typically performed using a system containing a processor, memory in electronic communication with the processor, and instructions stored in the memory, and that this is well-known in the art.
It would be obvious to one of ordinary skill in the art before the effective filing date of the invention that, as computer programs, implementing the methods of Song or Kanazawa would involve memory in electronic communication with at least one processor and instructions stored in the memory, the instructions executable by the processor.
Claim 11 otherwise recites similar limitations to claim 1, and is similarly rejected over Song in view of Kanazawa.
Claim 12 recites similar language to claim 2, and is similarly rejected.
Claim 13 recites similar language to claim 3, and is similarly rejected.
Claim 14 recites similar language to claim 4, and is similarly rejected.
Claim 15 recites similar language to claim 5, and is similarly rejected.
Regarding claim 16, Song and Kanazawa teach a computer readable medium storing instructions thereon that, when executed by at least one processor, cause a computing device to perform a method (Song 3.2 Symbolic Execution and Exploration/3.2 Single-Packet Exchange Symbolic Execution (SPE-SE), SymbexNet performs processing, causing the method to be performed).
They do not explicitly recite that the computer readable medium is non-transitory (Song and Kanazawa are non-patent literature and are thus focused on describing the systems rather than the hardware which implements them).
The examiner takes official notice that computer readable mediums for instructions can be non-transitory, and that this is well-known in the art.
It would be obvious to one of ordinary skill in the art before the effective filing date of the invention that, as computer programs, implementing the methods of Song or Kanazawa would involve non-transitory computer readable media.
Claim 16 otherwise recites similar limitations to claim 1, and is similarly rejected over Song in view of Kanazawa.
Claim 17 recites similar language to claims 6 and 7, and is similarly rejected.
Regarding claim 21, Song in view of Kanazawa teaches the method of claim 1, wherein generating the plurality of protocol tests includes invoking, by the symbolic execution engine, a compiler (Song section 3.3 Single-Packet Exchange Symbolic Execution (SPE-SE), the source code is compiled to LLVM bitcode to execute a network daemon symbolically) and a plurality of commands to instruct the symbolic execution engine in generating code for extracting protocol test cases for testing the functionality of the modeled component of the protocol (Song section 3.3 Single-Packet Exchange Symbolic Execution (SPE-SE), the user provides instructions to mark certain fields as symbolic to explore execution paths).
Claim 22 is rejected under 35 U.S.C. 103 as being unpatentable over Song in view of Kanazawa as applied to claim 1 above, and further in view of Delimat (cited in 101 rejection).
Regarding claim 22, Song in view of Kanazawa teaches the method of claim 1.
Song in view of Kanazawa does not explicitly teach the method of claim 1 wherein a first task of compiling protocol test cases and a second task of executing the plurality of protocol tests are performed in separate Docker containers (Song and Kanazawa are not concerned with Docker containerization).
Delimat teaches a first task of compiling protocol (Overkill?/Our first test, the software being tested is an API on a server) test cases (Our first test, the myapp container builds the container image) and a second task of executing the plurality of protocol tests (Our first test, the myapp-tests container executes the tests) are performed in separate containers (The two containers are separate).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to, after generating the test code through the combination of Song and Kanazawa, compile and run the tests in separate Docker containers as taught by Delimat. They would be motivated to do so because it is an efficient testing strategy (Delimat Header) that containerizes instances so they do not interfere with one another, allowing for parallelization without losing accuracy (Delimat The painful alternatives), and is a highly adaptable approach (Delimat Conclusion and a few other thoughts).
Claim 23 is rejected under 35 U.S.C. 103 as being unpatentable over Song in view of Kanazawa as applied to claim 21 above, and further in view of the Wikipedia page for Compilation error (cited in 101 rejection), hereinafter Wikipedia Compilation.
Regarding claim 23, Song in view of Kanazawa teaches the method of claim 21, further comprising:
logging one or more errors based on executing the plurality of protocol tests (Song Section 6.7 Discovered Implementation Errors, SymbexNet detects and logs bugs in the code);
updating the LLM (Kanazawa pages 3 and 4, the user may need to fix the prompt in response to the LLM-generated code containing an error) in generating subsequent modeled components of the protocol based on the logged one or more errors (Song Section 8 Conclusions, SymbexNet detected and logged a number of errors, which were then fixed by developers).
Song in view of Kanazawa does not explicitly state that the error that was corrected is a compiler error (Song presumes the code can be run, which means any compiler errors would have been fixed before testing; Kanazawa’s examples are in Python, which is not a statically compiled language).
Wikipedia Compilation teaches that compile errors are a commonly known type of error (Header and Examples).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention that, when generating code with an LLM and correcting errors caused by the LLM, as taught by Song in view of Kanazawa, that one such error could be a compilation error as taught by Wikipedia Compilation. It would have been obvious because a compilation error is a commonly known type of error (Wikipedia Compilation Common C++ compilation errors), and given the presumption in Song that the code can be compiled, one of ordinary skill in the art would be clearly motivated to correct compilation errors in the code. Wikipedia Compilation teaches a common compilation error in which a common function is undeclared (Wikipedia Compilation Common C++ compilation errors). The conditions under which such a common function undeclared error occurs can be generated in LLM-generated code, and can be corrected by modifying the prompt (Kanazawa pages 2 and 3, the example issue is that the LLM incorrectly assumes the library is already declared/imported. The user notes this issue and corrects it by modifying the prompt to include instructions to import any needed libraries first). While the error described in Kanazawa would not typically be referred to as a compilation error due to Python not being statically compiled, it would be clear to one of ordinary skill in the art that the methods of identifying and correcting the errors of Kanazawa and Wikipedia Compilation are the same.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory a