Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Detailed Action
Office Action is in response to the instant Application 19/221,525 filed on 5/29/2025. Claims 1-20 are pending. This Office Action is Non-Final.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 USC 101 as being directed to an abstract idea without being integrated into a practical application or being significantly more.
Regarding claim 1 and similarly in claims 12 and 13, the claim recites the limitations “obtaining evaluation seed date;” and “generating a test …” Broadly interpreted, the aforementioned steps are directed to mental processes as said steps could be performed in the human mind. Therefore, the claims recite an abstract idea. Said abstract idea and/or judicial exception is not integrated into a practical application as the claim does not recite any other active steps that could be considered that the abstract idea is being integrated into a practical application.
However, said operations are not sufficient to consider that the abstract idea is being interpreted into a practical application. Said operations are recited at a high level of generality in gathering/processing/storing information, which are a form of insignificant extra-solution activity.
It’s also noted that the claims recite additional limitation/elements (i.e., system, processing circuitry, processor, memory, etc.,). However, said additional elements are recited at a high-level of generality (i.e., as a generic computing device performing a generic computer functions) such that it amounts no more than mere instructions to apply the exception or abstract idea using generic computer components. Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
The claims do not include additional elements/limitations/embodiments that are sufficient to amount to significantly more than the judicial exception because the additional elements when considered both individually and as an ordered combination do not amount to significantly more than the abstract idea. As mentioned above, although the claims recite additional elements, said elements taken individually or as a combination, do not result in the claim amounting to significantly more than the abstract idea because as the additional elements perform generic computer content distributing functions routinely used in information technology field. As discussed above, the additional elements recited at a high-level of generality such that they amount no more than mere instructions to apply the exception using a generic computer component. Therefore, the claim is directed to non-statutory subject matter.
Regarding claims 2-11 and 14-20, claims 2-11 and 14-20 are also rejected under 35 U.S.C. 101 as being directed to non-statutory subject matter for the same reasons addressed above as the claims recite an abstract idea and the claims do not positively recite any other operations that could be considered as the abstract idea is being integrated into a practical application or significantly more. Therefore, claims 2-11 and 14-20 are also rejected under 35 U.S.C. 101 as being directed to non-statutory subject matter.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-5 and 9-17 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Glynn et al. (US 2025/0315533).
As per claim 1, Glynn discloses a test case generation method, wherein the method comprises: obtaining evaluation seed data (Glynn, Paragraph 0045 recites “For example, the first computing device may input a first prompt to the LLM as: “Based on the OWASP standards, please identify what insecurities might be present in an API with this schema.” The first computing device may also send the API with the scheme referred to in the first prompt. The first computing device may send a description to the API, to the LLM. For example, the API and the description the first computing device sends to the LLM may be as below:”);
and generating a test case set and a case label of each test case in the test case set based on the evaluation seed data by using a trained generative large model and at least one induced attack technique (Glynn, Paragraph 0054 recites “At step 420, the first computing device (or the LLM) may generate a first version of the threat model based on the first information for generating the threat model. The first version of the threat model may be a preliminary version of the threat model. The first computing device may generate a version of a threat model by combining the threats identified by the LLM and additional information associated with the computing system that may facilitate an evaluation team to understand the threats. For example, the threat model may comprise an overview of the computing system.”).
As per claim 2, Glynn discloses the test case generation method according to claim 1, Glynn further discloses wherein the test case set comprises a first test case set and a second test case set; and generating a test case set and a case label of each test case in the test case set based on the evaluation seed data by using a trained generative large model and at least one induced attack technique comprises: inputting the evaluation seed data to the trained generative large model, to obtain the first test case set; obtaining a generation control condition of the generative large model; determining the at least one induced attack technique based on the generation control condition by using an adaptive attack strategy generator; and performing transformation processing on the first test case set by using each induced attack technique, to generate the second test case set and a case label of each test case in the second test case set (Glynn, Paragraph 0055-0056 recites “At step 425, the first computing device may determine whether additional data is to be provided to the LLM. The additional data may be used to improve the threat model. The additional data may be the data requested by the LLM (e.g., as described in step 420 above). Additionally or alternatively, the additional data may be obtained without the explicit request from the LLM. For example, the additional data may comprise a weakness in the threat model that is identified by an engineer reviewing the threat model, an update to the computing system, and/or a new threat identified to a software module similar to the one in the computing system, as described in greater detail in FIG. 5 below. If the first computing device determines that no additional data is to be provided to the LLM, the first computing device may wait until new data is received. If the first computing device determines that additional data is to be provided to the LLM, the method may proceed to step 430. At step 430, the first computing device may further input, to the LLM, the new data. The first computing device may also input the previous output received from the LLM, so that the LLM may refine the LLM's previous output in view of the new data. The first computing device may input a second prompt requesting second information for generating a new version (e.g., a second version) of the threat model. The new version of the threat model may be a refined version based on the previous version (e.g., the preliminary version). For example, in the previous output, the LLM may identify a potential threat without identifying the precise impact of the threat on the computing system. The LLM may request a penetration test to determine what impact the potential threat may have on the computing system. The first computing device may input the test result (e.g., a test log) to the LLM. The LLM may refine the previous output by providing a precise assessment regarding the impact of the potential threat and/or a priority of fixing the potential threat (e.g., a higher priority may be determined if the impact is likely to be greater), for example, based on the test result.”).
As per claim 3, Glynn discloses the test case generation method according to claim 2, Glynn further discloses wherein the induced attack technique comprises an initial induced attack technique and a target induced attack technique; and determining the at least one induced attack technique based on the generation control condition by using an adaptive attack strategy generator comprises: selecting the initial induced attack technique; and performing transformation processing on a target test case in the first test case set by using the initial induced attack technique, and upon determining that a transformation result meets the generation control condition, adjusting the initial induced attack technique by using the adaptive attack strategy generator until the target induced attack technique is determined when the transformation result of the target test case does not meet the generation control condition (Glynn, Paragraph 0051 recites “The first output may also comprise a request for additional data. The additional data may be requested by the LLM to further improve the threat model. For example, the additional data may comprise information related to another portion of the computing system that interacts with the one or more software modules analyzed by the LLM. For example, if the one or more software modules analyzed by the LLM retrieve data from a database, the LLM may request information associated with the database in order to further identify potential threats. In another example, the additional data may comprise a result of a penetration test on the one or more software modules. The penetration test may simulate a cyber-attack against the computing system to further determine the vulnerabilities of the computing system. The LLM may generate the test instruction (e.g., the scope of the test, the payload of the simulated attack, etc.) and/or send the test instruction to the first computing device. The first computing device may conduct the test (e.g., either directly or via another computing device that communicates with the computing system) and obtain a test result (e.g., logs that reflect the computing system's reaction to the simulated attack). As described in further detail below, the test result may be used to improve the threat modeling (e.g., by generating a refined version of the threat model).”)
As per claim 4, Glynn discloses the test case generation method according to claim 2, Glynn further discloses wherein the evaluation seed data comprises text seed data; and inputting the evaluation seed data to the trained generative large model, to obtain the first test case set comprises: generating context information of the text seed data by using a context processor; and generating the first test case set based on the context information by using the trained generative large model (Glynn, Paragraph 0045 recites “For example, the first computing device may input a first prompt to the LLM as: “Based on the OWASP standards, please identify what insecurities might be present in an API with this schema.” The first computing device may also send the API with the scheme referred to in the first prompt. The first computing device may send a description to the API, to the LLM. For example, the API and the description the first computing device sends to the LLM may be as below:”).
As per claim 5, Glynn discloses the test case generation method according to claim 2, Glynn further discloses wherein the evaluation seed data comprises image seed data; and inputting the evaluation seed data to the trained generative large model, to obtain the first test case set comprises: performing element detection on the image seed data, to obtain an image element comprised in the image seed data and description information of the image element; and generating the first test case set based on the description information by using the trained generative large model (Glynn, Paragraph 0036 recites “Ultimately, the trained model may be provided with input beyond the training set and used to generate predictions regarding the likely results. Artificial neural networks may have many applications, including object classification, image recognition, speech recognition, natural language processing, text recognition, regression analysis, behavior modeling, and others.”).
As per claim 9, Glynn discloses the test case generation method according to claim 1, Glynn further discloses wherein the method further comprises: displaying the test case set and the case label of each test case in the test case set through a user interface; and receiving modification information input through the user interface, and adjusting the test case set and the corresponding case label based on the modification information (Glynn, Paragraph 0025 recites “As seen in FIG. 1, computing device 101 may include a processor 111, RAM 113, ROM 115, network interface 117, input/output interfaces 119 (e.g., keyboard, mouse, display, printer, etc.), and memory 121. Processor 111 may include one or more computer processing units (CPUs), graphical processing units (GPUs), or other processing units such as a processor adapted to perform computations associating converting information, routing copies of messages, or other functions described herein. I/O 119 may include a variety of interface units and drives for reading, writing, displaying, or printing data or files. I/O 119 may be coupled with a display such as display 120.”).
As per claim 10, Glynn discloses the test case generation method according to claim 1, Glynn further discloses wherein the method further comprises: inputting a target test case in the test case set to the generative large model, to obtain an output result corresponding to the target test case; and determining a risk evaluation result of the generative large model based on a case label and the output result that correspond to the target test case (Glynn, Paragraph 0050 recites “The first information may also comprise other information associated with the identified threat. For example, the first information may comprise a risk assessment indicating the potential impact of each of the identified threats, a likelihood regarding how probable each the threats may occur, and/or a recommendation of a remedial action. It is appreciated that the information for generating the threat model is merely an example, other information relevant to threat modeling is possible.”)
As per claim 11, Glynn discloses the test case generation method according to claim 10, Glynn further discloses wherein inputting a target test case in the test case set to the generative large model, to obtain an output result corresponding to the target test case comprises: inputting the target test case to the generative large model through a preset API interface, and obtaining the output result corresponding to the target test case through the API interface (Glynn, Paragraph 0046 recites “The LLM may receive the first prompt and/or information associated with the API. The LLM may identify the threats in the API based on the first prompt. The first prompt may also specify the format of the expected output from the LLM. For example, the first prompt may state: “Please provide a response that includes a subset of the code provided, or an answer of ‘nothing insecure’ if nothing is found to be insecure.”).
Regarding claims 12 and 13, claims 12 and 13 are directed to a non-transitory computer-readable storage medium and a computing device associated with the method of claim 1. Claims 12 and 13 are of similar scope to claim 1, and are therefore rejected under similar rationale.
Regarding claim 14, claim 14 is directed to a similar method associated with the method of claim 2 respectively. Claim 14 is similar in scope to claim 2, respectively, and are therefore rejected under similar rationale.
Regarding claim 15, claim 15 is directed to a similar method associated with the method of claim 3 respectively. Claim 15 is similar in scope to claim 3, respectively, and are therefore rejected under similar rationale.
Regarding claim 16, claim 16 is directed to a similar method associated with the method of claim 4 respectively. Claim 16 is similar in scope to claim 4, respectively, and are therefore rejected under similar rationale.
Regarding claim 17, claim 17 is directed to a similar method associated with the method of claim 5 respectively. Claim 17 is similar in scope to claim 5, respectively, and are therefore rejected under similar rationale.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 6 and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Glynn et al. (US 2025/0315533) in view of Sellars et al. (US 2024/0406210).
As per claim 6, Glynn discloses the test case generation method according to claim 1, but fails to teach wherein the induced attack technique comprises one or more of a contrastive technique, a role-playing technique, a backward induction technique, a text adversarial technique, a step-by-step technique, a target obfuscation technique, a forced consent technique, and a long-sentence overflow technique.
However, in an analogous art Sellars teaches wherein the induced attack technique comprises one or more of a contrastive technique, a role-playing technique, a backward induction technique, a text adversarial technique, a step-by-step technique, a target obfuscation technique, a forced consent technique, and a long-sentence overflow technique (Sellars, Paragraph 0039 recites “The text representation might say, for example, this device did this, then this cyber threat compromised this node initially and then migrated to this device, etc., etc. The textual sentence produced is a string of codes describing aspects of a cyber compromise. The string of codes, for example, further includes-edge type, from this node type to this node type, the indexes of those devices involved in the compromise, type of compromise. Next, the system trains the large language model on these sentences/textual representations derived from graphs describing a security compromise produced by the cyber security analyst, so that the LLM can deduce how these attacks are structured. Next, once the neural networks forming the artificial intelligence have learned how to understand graphs that describe a security compromise, then the synthetic cyberattack tool 125 can work the LLM to take the next step and train on how to generatively produce graphs that describe how to make a security compromise. Thus, cyberattack simulator 105 can use an autonomous agent LLM trained on cyber security analyst graphs describing a security compromise to then produce a graph for a proposed simulated/synthetic cyberattack and then evaluate each step in an ongoing synthetic cyberattack to the current conditions in the mimicked network.”).
It would have been obvious to a person of ordinary skill in the art, before the earliest effective filing date, to use Sellars’ Cyber security training tool that uses a large language model with Glynn’s Threat Model Generation Systems because it offers the advantage of providing analysis and an explanation as to why machine learning identified the synthetic cyberattack and/or the real cyberattack as a cyber threat for a purpose of providing cyber security training.
Regarding claim 18, claim 18 is directed to a similar method associated with the method of claim 6 respectively. Claim 18 is similar in scope to claim 6, respectively, and are therefore rejected under similar rationale.
Claim(s) 7 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Glynn et al. (US 2025/0315533) in view of Devaraj et al. (US 2024/0273395).
As per claim 7, Glynn discloses the test case generation method according to claim 1, but fails to teach wherein the case label of each test case comprises a case quality score, a case risk category, an induced attack technique, and case question difficulty.
However, in an analogous art Devaraj teaches wherein the case label of each test case comprises a case quality score, a case risk category, an induced attack technique, and case question difficulty (Devaraj, Paragraph 0075 recites “Model validation customizer system 401 may use these scores of relative importance to identify difficulty levels at which to cut off tests for each of the quality pillar performance categories. Cutting off the tests may include cutting the tests short, or applying a completion to the tests, e.g., at the identified levels corresponding to the relative importance of each of the quality pillar performance categories, which may include to override other completion conditions to apply completion to the tests, for example.”).
It would have been obvious to a person of ordinary skill in the art, before the earliest effective filing date, to use Devaraj’s automated customized machine learning model validation flow with Glynn’s Threat Model Generation Systems because it offers the advantage of providing data to increase the accuracy of tests.
Regarding claim 19, claim 19 is directed to a similar method associated with the method of claim 7 respectively. Claim 19 is similar in scope to claim 7, respectively, and are therefore rejected under similar rationale.
Claim(s) 8 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Glynn et al. (US 2025/0315533) in view of Ramarao (US 2025/0173556).
As per claim 8, Glynn discloses the test case generation method according to claim 1, but fails to teach wherein obtaining evaluation seed data comprises: obtaining a historical seed data set and evaluation data corresponding to each piece of historical seed data in the historical seed data set; performing feature extraction on each piece of historical seed data, to obtain a seed feature; encoding each piece of evaluation data to obtain an evaluation feature, and obtaining an input feature of each piece of historical seed data based on the seed feature and the evaluation feature; and determining the evaluation seed data from the historical seed data set based on each input feature by using a trained reinforcement learning model.
However, in an analogous art Ramarao teaches wherein obtaining evaluation seed data comprises: obtaining a historical seed data set and evaluation data corresponding to each piece of historical seed data in the historical seed data set; performing feature extraction on each piece of historical seed data, to obtain a seed feature; encoding each piece of evaluation data to obtain an evaluation feature, and obtaining an input feature of each piece of historical seed data based on the seed feature and the evaluation feature; and determining the evaluation seed data from the historical seed data set based on each input feature by using a trained reinforcement learning model (Ramarao, Paragraph 0020 recites “In one embodiment, modifying a seed statement includes replacing an original seed statement with a historical seed statement or an organization-authorized seed statement. For example, an organization may store a set of seed statements that have been used in the past to generate description content. Replacing the original seed statement may include applying a nearest-neighbor type algorithm to the original seed statement and the additional seed statements to determine, by a computer, which of the additional seed statements is the closest, semantically, to the original seed statement. Replacing the description content may include providing a sentence to an LLM with a prompt to generate a replacement sentence or set of sentences. For example, the system may generate a prompt to an LLM. The prompt includes the following elements: (a) a particular sentence that corresponds to a particular output segment and a relevance score that is below a threshold, (b) a particular input segment paired with the particular sentence in the maximal match, and (c) an instruction to modify the sentence to be more relevant to the particular input segment. In one example, the prompt further includes a weight value as metadata, representing the degree to which the LLM should modify the sentence based on a degree to which the sentence falls short of the relevance threshold.”).
It would have been obvious to a person of ordinary skill in the art, before the earliest effective filing date, to use Ramarao’s Relevance-Based Filtering Of Machine-Learning-Generated Descriptions with Glynn’s Threat Model Generation Systems because it offers the advantage of filtering content from ML-generated descriptions based on a relevance of sentences within the descriptions to sub-components of a seed statement used to generate the description.
Regarding claim 20, claim 20 is directed to a similar computing device associated with the method of claim 8 respectively. Claim 20 is similar in scope to claim 8, respectively, and are therefore rejected under similar rationale.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RODERICK TOLENTINO whose telephone number is (571)272-2661. The examiner can normally be reached Mon- Fri 8am-4pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Luu Pham can be reached at 571-270-5002. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
RODERICK . TOLENTINO
Examiner
Art Unit 2439
/RODERICK TOLENTINO/Primary Examiner, Art Unit 2439
Prosecution Insights