DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-2 and 11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al., US 12627689 B2 (hereafter referred to as Zhang) in view of Feng et al., US 20250080556 A1 (hereafter referred to as Feng).
Claim 11, Zhang teaches a system comprising: processing circuitry (column 2, lines 10-18; “The invention can be implemented in numerous ways, including as a process; an apparatus; a system; a composition of matter; a computer program product embodied on a computer readable storage medium; and/or a processor, such as a processor configured to execute instructions stored on and/or provided by a memory coupled to the processor.”) configured to:
obtain an Application Programming Interface (API) call that is associated with a Large Language Models (LLM) (column 10, lines 36-43; “prediction engine 174 determines whether a newly received sample is malicious based at least in part on characteristics/attributes pertaining to the sample (e.g., regex statements, information obtained from a file header, calls to libraries, APIs, etc.).” );
generate a feature vector that numerically represents data included in the API call associated with the LLM (column 10, lines 5-22; “In response to receiving a sample for which malicious sample detector 170 is to determine whether the sample is malicious (or a likelihood that the sample is malicious), malicious sample detector 170 determines the one or more feature vectors (e.g., individual feature vectors corresponding to a set of predefined regex statements, individual feature vectors corresponding to attributes or patterns obtained using an algorithmic-based analysis of exploits, and/or a combined feature vector of both, etc.).” And column , lines ; “As will be further described below with respect to FIG. 3A, a Large Language Model (LLM) (e.g., a commercially available LLM solution from OpenAI, Microsoft, Google, and/or other vendors can be used for the LLMs described herein) is trained to understand whether a given network session is malicious or not, and specifically, it is trained to provide an explanation as to the result. For example, the trained LLM can provide a reason for the determination that a given session was found to be malicious. The training is generally performed automatically using samples of malicious and benign session data (e.g. and using various prompts to fine tune the LLM model, such as further described below with respect to FIGS. 3A and 3B). The output of the trained LLM model provides reasoning with fine grained details (e.g., a human understandable explanation and report, such as further described herein), such as a vulnerable parameter, an injected meta-character, a malicious command, etc.”);
provide the feature vector to a security LLM trained to detect security threats to the LLM (column 21, lines 56-62; “At 504, sending the sample to a security service to generate a Large Language Model (LLM) powered detection and reason is performed. For example, the LLM can be trained for providing detection reasoning for samples associated with malicious HTTP request headers with command injection, such as similarly described above with respect to FIGS. 3A-4D.”);
obtain an output from the security LLM that indicates a security threat to the LLM (column 21, lines 62-67 and column 22, lines 1-3; “The trained LLM can then be prompted to automatically generate a malware or benign verdict and a reason for explaining the verdict. At 506, reporting the LLM powered detection and reason is performed. For example, the LLM powered detection and reason can be generated as a report that is sent back to an inline security service and/or to an offline security reporting service, such as similarly described above with respect to FIGS. 3A-4D.”);
determine a security policy based on the security threat (column 10, lines 39-46; “According to various embodiments, prediction engine 174 determines whether a newly received sample is malicious based at least in part on characteristics/attributes pertaining to the sample (e.g., regex statements, information obtained from a file header, calls to libraries, APIs, etc.). For example, prediction engine 174 applies a machine learning model to determine whether the newly received sample is malicious.” ); and
provide the security policy to a security proxy that screens (column 11, lines 6-28; “Firewalls typically deny or permit network transmission based on a set of rules. These sets of rules are often referred to as policies (e.g., network policies or network security policies). For example, a firewall can filter inbound traffic by applying a set of rules or policies to prevent unwanted outside traffic from reaching protected devices. A firewall can also filter outbound traffic by applying a set of rules or policies (e.g., allow, block, monitor, notify or log, and/or other actions can be specified in firewall rules or firewall policies, which can be triggered based on various criteria, such as described herein). A firewall can also filter local network (e.g., intranet) traffic by similarly applying a set of rules or policies.”). Zhang does not specifically teach provide the security policy to a security proxy that screens the API call. However, in the same field of endeavor, Feng teaches provide the security policy to a security proxy that screens the API call (p. 113, “Profile 420 corresponds to the profile configuration from which the actual traffic is generated during the C2 session. In the example shown, profile 420 sets the URI for its http request with the value “/api/v1/users”.” The profile of the sample includes API calls. And p. 113, “… the system handles the network traffic based at least in part on (i) the traffic classification, and (ii) a security policy. For example, a security policy may define how the system is to handle C2 traffic (e.g., block the traffic, etc.).” See also p. 133, “process 1000 may be implemented by a security entity (e.g., a firewall) such as in connection with enforcing a security policy with respect to files communicated across a network or in/out of the network. In some implementations...”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Zhang by incorporating policy and rules for screening API calls from Feng for the screening traffic from applications and thereby improve effectiveness. The motivation would have been to improve effectiveness by further defining the training traffic to include more attributes and thereby improve predictions.
Claim 1 is a method comprising steps similar to the operations of the system of claim 11 above. Claim 1 is rejected on a similar rationale.
Claim 2, Zhang-Feng teaches the method of claim 1 further comprising:
obtaining training data that indicates historical security threats to the LLM (Zhang, column 8, lines 25-30, 37-40; “system 100 (e.g., malicious sample detector 170, security service 140, etc.) trains a detection model to detect exploits (e.g., malicious samples), malicious traffic, and/or other malicious/nefarious/undesirable activity/behavior, etc.” “…[T]he model(s) trained by system 100 (e.g., security service 140) are obtained using a machine learning process (e.g., implementing various machine learning techniques (MLT)).”);
generating one or more training feature vectors that numerically represent the historical security threats to the LLM (Zhang, column 9, lines 53-58; “Malicious sample detector 170 can use the set of one or more feature vectors to determine the one or more attributes of patterns that are to be used in connection with training or implementing the model (e.g., attributes for which fields are to be populated in the feature vector, etc.). The model may be trained using a set of features that are obtained based at least in part on sample malicious traffic, such as a set of features corresponding to predefined regex statements and/or a set of feature vectors determined based on an algorithmic-based feature extraction.” The “API calls” are optionally substituted for the “regex statements”.);
providing the one or more training feature vectors to the security LLM to train the security LLM to detect the security threats to the LLM (Zhang, column 9, lines 53-58; “Malicious sample detector 170 can use the set of one or more feature vectors to determine the one or more attributes of patterns that are to be used in connection with training or implementing the model (e.g., attributes for which fields are to be populated in the feature vector, etc.). The model may be trained using a set of features that are obtained based at least in part on sample malicious traffic, such as a set of features corresponding to predefined regex statements and/or a set of feature vectors determined based on an algorithmic-based feature extraction.” The “API calls” are optionally substituted for the “regex statements”.);
obtaining a training output from the security LLM that includes a prediction of the historical security threats (Zhang, column 10, lines 59-63; “For example, prediction engine 174 receives a result of a determination or analysis by machine learning model 176. In some embodiments, prediction engine 174 receives from machine learning model 176 an indication of a likelihood that the sample is malicious.”); and
determining a training state of the security LLM based on an accuracy of the prediction (column 18, lines 41-45; “… the training process can include an automated or at least a semi-automated offline validation system 330 that is performed to verify whether the LLM's output reasoning is correct or not (e.g., using a data pipeline 312 and scheduled analytics jobs 316 as shown in FIG. 3A).” Validated training process indicates that the state of the model is correct.).
Claim(s) 3, 12, 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang and Feng as applied to claims 1 and 11 above, and further in view of Wang et al., US 20240430282 A1 (hereafter referred to as Wang).
Claim 12, Zhang-Feng teaches the system of claim 11, as cited above. Zhang-Feng teaches wherein the security threat comprises a sensitive data leak. However, in the same field of use, Wang teaches wherein the security threat comprises a sensitive data leak (p. 51, “The behavior analytics system 110 may also employ one or more Machine Learning (ML) models for various purposes, such as a transformer model for API learning and understanding, a large language model for sensitive data classification, and a graph neural network for user behavior correlation.” And p. 64, “the behavior analytics system 110 may be utilized to provide data protection by detecting sensitive data leaks (such as data breaches, unintended partners, and internal attacks) and volumetric data exfiltration.”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Zhang- Feng by incorporating Wang’s sensitive leak into the malicious data to from Zhang-Feng to expand the type of malicious data detected. The motivation would have been to improve effectiveness by further defining the training traffic to include more attributes and thereby improve predictions.
Claim 3 is a method comprising steps similar to the operations of the system of claim 12 above. Claim 3 is rejected on a similar rationale.
Claim 20, Zhang-Feng teaches one or more computer-readable storage media having program instructions stored thereon, wherein the program instructions, when executed by a computing system, direct the computing system to perform operations, the operations comprising similar to claim 11 above (Zhang, p. 15, “The invention can be implemented in numerous ways, including as a process; an apparatus; a system; a composition of matter; a computer program product embodied on a computer readable storage medium; and/or a processor, such as a processor configured to execute instructions stored on and/or provided by a memory coupled to the processor. In this specification, these implementations, or any other form that the invention may take, may be referred to as techniques. In general, the order of the steps of disclosed processes may be altered within the scope of the invention.”). Which respect to claim 11 above, claim 20 further comprises wherein the security threat comprises at least one of a sensitive data leak, a prompt injection attack, data poisoning, insecure output handling, a denial-of-service attack, a permission issue, excessive agency, or an insecure plugin.
Claim 20 is computer readable storage media comprising steps similar to the operations of the system of claim 12 above. Claim 20 is rejected on a similar rationale.
Claim(s) 4-5 and 13-14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang and Feng as applied to claims 1 and 11 above, and further in view of Mantin et al., US 20250111051 A1 (hereafter referred to as Mantin).
Claim 13, Zhang-Feng teaches the system of claim 11 as cited above. However, Zhang-Feng does not specifically teach wherein the security threat comprises a prompt injection attack. However, in the same field of endeavor, Mantin teaches wherein the security threat comprises a prompt injection attack (p. 93, “Step 304 includes training, iteratively, the machine learning model using the first outputs and a second subset of the queries until convergence to generate a trained machine learning model which, when executed, is trained to detect prompt injection cyberattacks in monitored outputs of the large language model.”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Zhang- Feng by incorporating Mantin’s prompt injection into the malicious data to from Zhang-Feng to expand the type of malicious data detected. The motivation would have been to improve effectiveness by further defining the training traffic to include more attributes and thereby improve predictions.
Claim 4 is a method comprising steps similar to the operations of the system of claim 13 above. Claim 4 is rejected on a similar rationale.
Claim 14, Zhang-Feng teaches the system of claim 11 as cited above. However, Zhang-Feng does not specifically teach wherein the security threat comprises data poisoning. However, in the same field of use, Mantin teaches wherein the security threat comprises data poisoning (p. 92, “The large language model takes, as input, at least some of the queries and generates, as output, text. The outputs may be normal outputs (i.e., not affected by a prompt injection cyberattack) or may be poisoned outputs (i.e., influenced in some way by the prompt injection cyberattack).” And p. 93, “… a trained machine learning model which, when executed, is trained to detect prompt injection cyberattacks in monitored outputs of the large language model.”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Zhang- Feng by incorporating Mantin’s poisonous data into the malicious data to from Zhang-Feng to expand the type of malicious data detected. The motivation would have been to improve effectiveness by further defining the training traffic to include more attributes and thereby improve predictions.
Claim 5 is a method comprising steps similar to the operations of the system of claim 14 above. Claim 5 is rejected on a similar rationale.
Claim(s) 6 and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang and Feng as applied to claims 6 and 15 above, and further in view of Keller et al., USPN 12430428 B2 (hereafter referred to as Keller).
Claim 15, Zhang-Feng teaches the system of claim 11 as recited above. Zhang-Keller does not specifically teach wherein the security threat comprises insecure output handling. However, in the same field of endeavor, Keller teaches wherein the security threat comprises insecure output handling (column 3, lines 16-25; “By processing only select communications between an application system and the corresponding LLM, the threat detection system safeguards itself from being compromised by indirect prompt injection attacks aimed at compromising detection and/or exploiting vulnerabilities of the detection-based large generative model. In particular, the threat detection system prevents external content from being input into the detection-based large generative model ensuring accurate classifications of LLM outputs without becoming compromised itself.”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Zhang- Feng by incorporating Keller’s insecure output handling for the malicious data to expand the type of malicious data detected. The motivation would have been to improve effectiveness by further defining the training traffic to include more attributes and thereby improve predictions.
Claim 6 is a method comprising steps similar to the operations of the system of claim 15 above. Claim 6 is rejected on a similar rationale.
Claim(s) 7-9 and 16-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang and Feng as applied to claims 1 and 11 above, and further in view of Williams et al., US 20240333746 A1 (hereafter referred to as Williams).
Claim 16, Zhang-Feng teaches the system of claim 11 as above. Zhang-Feng does not specifically teach wherein the security threat comprises a denial-of-service attack. However, in the same field of endeavor, Williams teaches wherein the security threat comprises a denial-of-service attack (p. 129, “The abnormal network traffic may be further classified into one or more categories of known security threats, including … , elevation of privilege, and denial of service.” And p. 132, “the ML engine 505 may process and/or analyze the training data 520 (e.g., via MLTM 142) to train the ML model 510 to generate the abnormal network traffic detection code 580. The ML model 510 may be trained to generate the abnormal network traffic detection code 580 via a large language model, …”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Zhang- Feng by incorporating Williams’ denial of service attach for the malicious data to expand the type of malicious data detected. The motivation would have been to improve effectiveness by further defining the training traffic to include more attributes and thereby improve predictions.
Claim 7 is a method comprising steps similar to the operations of the system of claim 16 above. Claim 7 is rejected on a similar rationale.
Claim 17, Zhang-Feng teaches the system of claim 11, as cited above. Zhang-Feng does not specifically teach wherein the security threat comprises a permission issue. However, in the same field of endeavor, Williams teaches the security threat comprises a permission issue (p. 129, “The abnormal network traffic may be further classified into one or more categories of known security threats, including … , elevation of privilege, and denial of service.” And p. 132, “the ML engine 505 may process and/or analyze the training data 520 (e.g., via MLTM 142) to train the ML model 510 to generate the abnormal network traffic detection code 580. The ML model 510 may be trained to generate the abnormal network traffic detection code 580 via a large language model, …”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Zhang- Feng by incorporating Williams’ denial of service attach for the malicious data to expand the type of malicious data detected. The motivation would have been to improve effectiveness by further defining the training traffic to include more attributes and thereby improve predictions.
Claim 8 is a method comprising steps similar to the operations of the system of claim 17 above. Claim 8 is rejected on a similar rationale.
Claim 18, Zhang-Feng teaches the system of claim 11 as cited above. Zhang-Feng does not specifically teach wherein the security threat comprises excessive agency. However, in the same field of endeavor, Williams teaches wherein the security threat comprises excessive agency (p. 83, “the training data 320 may labeled to aid in training, retraining and/or fine-tuning the ML model 310. The training data 320 may include security vulnerability documents. The security vulnerability documents may comprise descriptions of security vulnerabilities in source code, applications, or software. For example, security vulnerability documents may be in Common Weakness Enumeration format, Common Vulnerability Exposures format, or unformatted. The security vulnerabilities in the security vulnerability documents may include … race conditions, uncontrolled resource consumption …” And p. 88, “The ML model 310 may be trained to generate the identified security vulnerabilities 350 via a large language model, neural network …”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Zhang- Feng by incorporating Williams’ excessive agency attack for the malicious data to from Zhang-Feng to expand the type of malicious data detected. The motivation would have been to improve effectiveness by further defining the training traffic to include more attributes and thereby improve predictions.
Claim 9 is a method comprising steps similar to the operations of the system of claim 18 above. Claim 9 is rejected on a similar rationale.
Claim(s) 10 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang and Feng as applied to claims 1 and 11 above, and further in view of Oz et al., USPN 12468785 B2 (hereafter referred to as Oz).
Claim 19, Zhang-Feng teaches the system of claim 11 as cited above. Zhang-Feng does not specifically teach wherein the security threat comprises an insecure plugin. However, in the same field of endeavor Oz teaches the security threat include an insecure plugin (column 3, lines 4-12; “Where the application acts based on LLM output, the attack may have the consequence of sensitive data being exposed outside the application, or nefarious actions taken by the application. In this scenario, the user of the LLM is likely unaware of the attack and is an innocent bystander. As LLM systems become more flexible with plugins, skills, and capabilities, the dangers of indirect prompt injection become more severe.”). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Zhang- Feng by incorporating Oz’s insecure plugin attach for the malicious data to from Zhang-Feng to expand the type of malicious data detected. The motivation would have been to improve effectiveness by further defining the training traffic to include more attributes and thereby improve predictions.
Claim 10 is a method comprising steps similar to the operations of the system of claim 19 above. Claim 10 is rejected on a similar rationale.
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Smith et al., US 20250294033 A1, teaches extracting information concerning cybersecurity events (such as cybersecurity breaches) from received text descriptions of those events. extraction of this information can be automated using machine learning or artificial intelligence models such as Large Language Models (LLMs). A predetermined set of prompts designed to elicit specific types of information about data breaches can also be used in conjunction with the text data and provided to an LLM to extract the predefined details. Outputs received from LLMs can be further processed according to expert-defined rules which augment their output.
Parla, US 20250097237 A1, teaches when combined with penetration testing to identify additional variants of malware, the integration of a Large Language Model (LLM) brings an arsenal of capabilities to the table for thwarting potential cybersecurity attacks. Leveraging its code summarization processing capabilities, an LLM can meticulously analyze the patterns and attributes of these new malware variants. The LLM contributes to early detection and classification, ensuring that security teams can swiftly recognize potential threats and respond with targeted countermeasures.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PATRICE L WINDER whose telephone number is (571)272-3935. The examiner can normally be reached M-F 10am-6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, KAMAL B DIVECHA can be reached at (571)272-5863. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Patrice L Winder/Primary Examiner, Art Unit 2453
Prosecution Insights