DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 05/13/2026 has been entered.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-6, 8-13 and 15-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Boyer et al., US-20240045990-A1 (hereinafter “Boyer ‘990”) in view of Agarwal, US- 20200342116-A1 (hereinafter “Agarwal ‘116”) and Alayrac et al., US-20230350936-A1 (hereinafter “Alayrac ‘936”).
Per claim 1 (independent):
Boyer ‘990 discloses: A system, comprising:
at least one computing device comprising at least one processor and at least one memory; and
machine-readable instructions stored in the at least one memory that, when executed by the at least one processor, cause the at least one computing device to at least:
generate at least one user interface comprising instructions to provide audio data that describes, for a particular application, at least one of: threats, weaknesses, security controls, or any combination thereof;
generate LLM prompting data based at least in part on application architecture data and the audio data
(FIG. 5B, [0081], an interactive cyber security user interface (at least one user interface) having a LLM module; [0082], The interactive cyber security user interface 710 (e.g. a form a chatbot) receives supplied input from a user, whether it be via written or voice input (provide audio data) ... from the user from a number of different input sources, such as the UI of the local cyber security appliance 100 (para [0117] – [0119] disclose that the cybersecurity appliance collects and stores various types of cybersecurity-related data, including “comprehensive logs for network traffic,” “metrics and previous threat alerts associated with network traffic.” – describes at least one of: threats, weaknesses, security controls, or any combination thereof),... convert the speech to text ... into supplied text that is fed into a natural language processing module to generate both a specific question being asked as well as a dialog manager to keep track of the background contextual information associated with the question being asked (generate LLM prompting data) ... The query builder module and the user interaction module may be part of a generative artificial intelligence large language model (LLM) that has been trained. The query builder module of the interactive cyber security user interface 710 then executes that generated software code to query, for example, API's of any of the multiple different components/devices of the cyber security system; [0182], the cyber security restoration engine to restore the protected system can use historic source codebase information (application architecture data) and modelling from the AI models in the detection engine for development to revert commits and code changes that potentially introduce bad or compromised code; note that for example, the LLM prompting data may include a natural-language request specifying how to restore a system to a normal state by reversing changes caused by a cyberattack);
input, into a threat modeling Large Language Model (LLM), the LLM prompting data comprising: the audio data and LLM instructions for the threat modeling LLM to generate application security data using the audio data; and
receive, from the threat modeling LLM, the application security data comprising at least one of: threat data, weakness data, security control data, a security risk summarization, an application threat model, or any combination thereof
(FIG. 5B, [0081], The query builder module and the user interaction module may be part of a generative artificial intelligence large language model (a threat modeling Large Language Model (LLM)) that has been trained. The query builder module of the interactive cyber security user interface 710 then executes that generated software code (generated by inputting the LLM prompting data into the threat modeling Large Language Model) to query, for example, API's of any of the multiple different components/devices of the cyber security system (e.g. the cyber security restoration engine 140, the prediction/simulator engine 105, the autonomous response engine 140, the cyber threat detect engine 100 all locally as well as then tap into the secure communications ... The query builder module of the interactive cyber security user interface 710 can also execute the generated software code to query, for example, data and analysis...) to retrieve information as well as other cloud based resources to formulate an answer and explain in natural human language the answer to the specific question being asked by the user (receive, from the threat modeling LLM, the application security data); [0068], the interactive cyber security user interface 710 may process the responses received from each of the queried components of the cyber security system using the LLM module to collate and/or summarize the information received in the responses – comprising at least one of: threat data, weakness data, security control data, a security risk summarization, an application threat model, or any combination thereof);
Boyer ‘990 does not disclose but Agarwal ‘116 discloses: generate the at least one user interface comprising instructions to provide image data that describes, for the particular application, at least one of: the threats, the weaknesses, the security controls, or any combination thereof
(FIG. 25, [0182], the user may select the new selector 2402 to bring up import interface (interface) 2500 (generate the at least one user interface) as shown in FIG. 25; [0184], The IMPORT selector allows a user to use a preexisting data file from a second software application/program to generate a threat model ... a user may import a MICROSOFT VISIO file (image data) in VSD and/or VSDX format; [0185], The VISIO file imported by the user would be a VISIO diagram file on which the user has previously diagrammed a system, application, or process and now wishes to generate a threat model using that same diagram; [0197], The data file (image data) of the second software would also generally define relationships between elements, for example at least lines connecting one element or component with another ... The second software diagram data file (image data), such as VSD or VSDX file, will generally include data related to each component, such as location of the component, which other components it is connected to, the default title of the component, any user-defined name for the component, etc. The system 100 and methods described herein use that information to create the threat model – describes, for the particular application, at least one of: the threats, the weaknesses, the security controls, or any combination thereof).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified Boyer ‘990 with the import of a preexisting data file from a second software application in order to generate a threat model as taught by Agarwal ‘116 because such an approach advantageously allows an existing application or system architecture diagram to be reused. Additionally, Agarwal ‘116 is analogous to the claimed invention because it teaches threat modeling process which includes creating a threat model of the application, system or process of interest [0085].
Boyer ‘990 in view of Agarwal ‘116 does not disclose but Alayrac ‘936 discloses: generate multimodal LLM prompting data based at least in part on the audio data, and the image data;
input, into a multimodal Large Language Model (LLM), the multimodal LLM prompting data comprising: the audio data, the image data, and LLM instructions
(FIG. 2, [0078], the structure of the query processing system 200 is illustrated. The query processing system 200 is configured to receive as an input a query input 201 which includes one or more data items (indicating multimodal LLM) and an input token string (LLM instructions) – input, into a multimodal Large Language Model (LLM), the multimodal LLM prompting data, that is, the query input 201 is to be fed into the token processing layers of FIG. 1(b); [0080], the modality network 207 may be configured to receive data item(s) of the query input which are a single image (e.g. still image; the image data), and the modality network 209 may be con figured to receive data item(s) of the query input which are audio data (the audio data); FIG. 1(b), [0077], The token processing layers may constitute a "language model" (e.g. a "large language model") (the multimodal LLM), trained on a large database of data, e.g. natural language data, such that upon an input token string (sequence of tokens from the vocabulary) being input to the first token processing layer, the output token string is an appropriate response.).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified Boyer ‘990 in view of Agarwal ‘116 with the query processing based on a multimodal LLM as taught by Alayrac ‘936 because the architecture enables a language model to jointly process multiple input modalities by converting the non-text data into modality-specific representations that can be processed together with input tokens. Additionally, Alayrac ‘936 is analogous to the claimed invention because it teaches a system which receives a query input comprising an input token string and also at least one data item having a second, different modality [0005].
Per claim 2 (dependent on claim 1):
Boyer ‘990 in view of Agarwal ‘116 and Alayrac ‘936 discloses the elements detailed in the rejection of claim 1 above, incorporated herein by reference.
Boyer ‘990 discloses: The system of claim 1, wherein the LLM instructions comprise natural language instructions for the threat modeling multimodal LLM (FIG. 5B, [0081], convert the speech to text ... into supplied text that is fed into a natural language processing module (for natural language instructions) to generate both a specific question being asked as well as a dialog manager to keep track of the background contextual information associated with the question being asked ... The query builder module and the user interaction module may be part of a generative artificial intelligence large language model (the threat modeling LLM) that has been trained. The query builder module of the interactive cyber security user interface 710 then executes that generated software code (generated from the LLM prompting data including natural language instructions) to query, for example, API's of any of the multiple different components/devices of the cyber security system; note that “multimodal LLM” has been taught by Alayrac ‘936).
Per claim 3 (dependent on claim 1):
Boyer ‘990 in view of Agarwal ‘116 and Alayrac ‘936 discloses the elements detailed in the rejection of claim 1 above, incorporated herein by reference.
Boyer ‘990 in view of Agarwal ‘116 does not disclose but Alayrac ‘936 discloses: The system of claim 1, wherein the LLM instructions comprise a first LLM instruction subset for the audio data and a second LLM instruction subset for the image data (FIG. 2, [0078], the structure of the query processing system 200 is illustrated. The query processing system 200 is configured to receive as an input a query input 201 which includes one or more data items and an input token string (the LLM instructions); [0080], the modality network 207 may be configured to receive data item(s) of the query input which are a single image (e.g. still image; the image data), and the modality network 209 may be con figured to receive data item(s) of the query input which are audio data (the audio data); FIG. 1(b), [0077], The token processing layers may constitute a "language model" (e.g. a "large language model") (the multimodal LLM), trained on a large database of data, e.g. natural language data, such that upon an input token string (sequence of tokens from the vocabulary) being input to the first token processing layer, the output token string is an appropriate response; [0023], a captioning system for a data item which is a video item. The input token string may define the video item processing task as "provide captions for the video explaining what is happening" ... If the data item is a video item with a soundtrack, the captions may comprise a transcription of the soundtrack – a first LLM instruction subset for the audio data and a second LLM instruction subset for the image data).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified Boyer ‘990 in view of Agarwal ‘116 with the query processing based on a multimodal LLM for a video item with a sound track as taught by Alayrac ‘936 because the architecture enables a language model to jointly process multiple input modalities by converting the non-text data into modality-specific representations that can be processed together with input tokens.
Per claim 4 (dependent on claim 1):
Boyer ‘990 in view of Agarwal ‘116 and Alayrac ‘936 discloses the elements detailed in the rejection of claim 1 above, incorporated herein by reference.
Boyer ‘990 in view of Alayrac ‘936 does not disclose but Agarwal ‘116 discloses: The system of claim 1, wherein the application threat model comprises a data flow diagram that visually shows the threat data, the weakness data, and the security control data in a diagrammatic form (FIG. 25, [0182], the user may select the new selector 2402 to bring up import interface (interface) 2500 as shown in FIG. 25; [0184], The IMPORT selector allows a user to use a preexisting data file from a second software application/program to generate a threat model (the application threat model) ... a user may import a MICROSOFT VISIO file (a data flow diagram) in VSD and/or VSDX format; [0185], The VISIO file imported by the user would be a VISIO diagram file on which the user has previously diagrammed (in a diagrammatic form) a system, application, or process and now wishes to generate a threat model using that same diagram; [0197], The data file of the second software would also generally define relationships between elements, for example (visually shows) at least lines connecting one element or component with another ... The second software diagram data file, such as VSD or VSDX file, will generally include data related to each component, such as location of the component, which other components it is connected to, the default title of the component, any user-defined name for the component, etc. The system 100 and methods described herein use that information to create the threat model – the threat data, the weakness data, and the security control data).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified Boyer ‘990 in view of Alayrac ‘936 with the import of a preexisting data file, diagramming a system, application, or process, from a second software application in order to generate a threat model as taught by Agarwal ‘116 because such an approach advantageously allows an existing application or system architecture diagram to be reused.
Per claim 5 (dependent on claim 4):
Boyer ‘990 in view of Agarwal ‘116 and Alayrac ‘936 discloses the elements detailed in the rejection of claim 4 above, incorporated herein by reference.
Boyer ‘990 in view of Alayrac ‘936 does not disclose but Agarwal ‘116 discloses: The system of claim 4, wherein the data flow diagram comprises an interactive data flow diagram viewed using a threat modeling software (FIG. 25, [0182], the user may select the new selector 2402 to bring up import interface (interface) 2500 as shown in FIG. 25; [0184], The IMPORT selector allows a user to use a preexisting data file from a second software application/program (using a threat modeling software, e.g., VISIO) to generate a threat model ... a user may import a MICROSOFT VISIO file (the data flow diagram) in VSD and/or VSDX format; [0185], The VISIO file imported by the user would be a VISIO diagram file on which the user has previously diagrammed a system, application, or process and now wishes to generate a threat model using that same diagram; [0197], The data file of the second software would also generally define relationships between elements, for example (an interactive data flow diagram) at least lines connecting one element or component with another ... The second software diagram data file, such as VSD or VSDX file, will generally include data related to each component, such as location of the component, which other components it is connected to, the default title of the component, any user-defined name for the component, etc. The system 100 and methods described herein use that information to create the threat model).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified Boyer ‘990 in view of Alayrac ‘936 with the import of a preexisting data file, diagramming a system, application, or process, from a second software application in order to generate a threat model as taught by Agarwal ‘116 because such an approach advantageously allows an existing application or system architecture diagram to be reused.
Per claim 6 (dependent on claim 4):
Boyer ‘990 in view of Agarwal ‘116 and Alayrac ‘936 discloses the elements detailed in the rejection of claim 4 above, incorporated herein by reference.
Boyer ‘990 in view of Alayrac ‘936 does not disclose but Agarwal ‘116 discloses: The system of claim 4, wherein the data flow diagram comprises an image (FIG. 25, [0182], the user may select the new selector 2402 to bring up import interface (interface) 2500 as shown in FIG. 25; [0184], The IMPORT selector allows a user to use a preexisting data file from a second software application/program to generate a threat model ... a user may import a MICROSOFT VISIO file (the data flow diagram) in VSD and/or VSDX format; [0185], The VISIO file imported by the user would be a VISIO diagram file on which the user has previously diagrammed a system, application, or process and now wishes to generate a threat model using that same diagram; [0197], The data file of the second software would also generally define relationships between elements, for example (an image) at least lines connecting one element or component with another ... The second software diagram data file, such as VSD or VSDX file, will generally include data related to each component, such as location of the component, which other components it is connected to, the default title of the component, any user-defined name for the component, etc. The system 100 and methods described herein use that information to create the threat model).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified Boyer ‘990 in view of Alayrac ‘936 with the import of a preexisting data file, diagramming a system, application, or process, from a second software application in order to generate a threat model as taught by Agarwal ‘116 because such an approach advantageously allows an existing application or system architecture diagram to be reused.
Per claim 8 (independent):
Boyer ‘990 discloses: A method, comprising:
training a threat modeling Large Language Model (LLM) to use audio to generate application security data comprising at least one of: threat data, weakness data, security control data, a security risk summarization, an application threat model, or any combination thereof, wherein the threat modeling LLM is trained using an application security data training set
([0074], Labelled training data sets to train and fine tune LLMs (training a threat modeling Large Language Model (LLM)) can be time consuming and costly to generate. Therefore, the LLMs of the LLM module can be trained on automatically generated data sets (an application security data training set). The LLM responsible for analyzing the natural language input may be trained on data sets that model human inputs; FIG. 5B, [0082], The interactive cyber security user interface 710 (e.g. a form a chatbot) receives supplied input from a user, whether it be via written or voice input (to use audio) ... from the user from a number of different input sources, such as the UI of the local cyber security appliance 100; [0068], the interactive cyber security user interface 710 may process the responses received from each of the queried components of the cyber security system using the LLM module (the threat modeling Large Language Model (LLM)) to collate and/or summarize the information received in the responses – generate application security data comprising at least one of: threat data, weakness data, security control data, a security risk summarization, an application threat model, or any combination thereof).
Boyer ‘990 in view of Agarwal ‘116 does not disclose but Alayrac ‘936 discloses: training a multimodal Large Language Model (LLM) to use audio and images, wherein the multimodal LLM is trained using an audio input training set and an image input training set
(FIG. 2, [0078], the structure of the query processing system 200 is illustrated. The query processing system 200 is configured to receive as an input a query input 201 which includes one or more data items (audio and images to be used) and an input token string – input, into a multimodal Large Language Model (LLM), the multimodal LLM prompting data, that is, the query input 201 is to be fed into the token processing layers of FIG. 1(b); [0080], the modality network 207 may be configured to receive data item(s) of the query input which are a single image (e.g. still image; images), and the modality network 209 may be con figured to receive data item(s) of the query input which are audio data (audios); FIG. 1(b), [0077], The token processing layers may constitute a "language model" (e.g. a "large language model") (the multimodal LLM), trained on a large database of data, e.g. natural language data, such that upon an input token string (sequence of tokens from the vocabulary) being input to the first token processing layer, the output token string is an appropriate response; FIG. 8, [0114], The modality network(s) 207, 209, and the gated cross-attention layers 231, 232, ... 23j may be trained jointly (the multimodal LLM is trained) in the system shown in FIG. 8 ... each comprise at least one data item and a token string ("multi-mode" training examples)(using an audio input training set and an image input training set) ... The data item(s) are input to the modality network(s) 207, 209 of the query processing system 200 – para [0019] recites “the data items in a given query input may have different respective modalities, e.g. one of the data items may be an audio signal and another may be a (still or moving) image”).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have modified Boyer ‘990 in view of Agarwal ‘116 with the query processing based on a multimodal LLM which can be trained based on audio and image data as taught by Alayrac ‘936 because the architecture enables a language model to jointly process multiple input modalities by converting the non-text data into modality-specific representations that can be processed together with input tokens.
The remaining limitations of the claim(s) correspond(s) to features of claim 1 and the claim(s) is/are rejected for the reasons detailed with respect to claim 1.
Per claim 9 (dependent on claim 8):
Boyer ‘990 in view of Agarwal ‘116 and Alayrac ‘936 discloses the elements detailed in the rejection of claim 8 above, incorporated herein by reference.
The limitations of the claim(s) correspond(s) to features of claim 2 and the claim(s) is/are rejected for the reasons detailed with respect to claim 2.
Per claim 10 (dependent on claim 8):
Boyer ‘990 in view of Agarwal ‘116 and Alayrac ‘936 discloses the elements detailed in the rejection of claim 8 above, incorporated herein by reference.
The limitations of the claim(s) correspond(s) to features of claim 3 and the claim(s) is/are rejected for the reasons detailed with respect to claim 3.
Per claim 11 (dependent on claim 8):
Boyer ‘990 in view of Agarwal ‘116 and Alayrac ‘936 discloses the elements detailed in the rejection of claim 8 above, incorporated herein by reference.
The limitations of the claim(s) correspond(s) to features of claim 4 and the claim(s) is/are rejected for the reasons detailed with respect to claim 4.
Per claim 12 (dependent on claim 8):
Boyer ‘990 in view of Agarwal ‘116 and Alayrac ‘936 discloses the elements detailed in the rejection of claim 8 above, incorporated herein by reference.
The limitations of the claim(s) correspond(s) to features of claim 5 and the claim(s) is/are rejected for the reasons detailed with respect to claim 5.
Per claim 13 (dependent on claim 8):
Boyer ‘990 in view of Agarwal ‘116 and Alayrac ‘936 discloses the elements detailed in the rejection of claim 8 above, incorporated herein by reference.
The limitations of the claim(s) correspond(s) to features of claim 6 and the claim(s) is/are rejected for the reasons detailed with respect to claim 6.
Per claim 15 (independent):
The limitations of the claim(s) correspond(s) to features of claim 8 and the claim(s) is/are rejected for the reasons detailed with respect to claim 8.
Per claim 16 (dependent on claim 15):
Boyer ‘990 in view of Agarwal ‘116 and Alayrac ‘936 discloses the elements detailed in the rejection of claim 15 above, incorporated herein by reference.
The limitations of the claim(s) correspond(s) to features of claim 2 and the claim(s) is/are rejected for the reasons detailed with respect to claim 2.
Per claim 17 (dependent on claim 15):
Boyer ‘990 in view of Agarwal ‘116 and Alayrac ‘936 discloses the elements detailed in the rejection of claim 15 above, incorporated herein by reference.
The limitations of the claim(s) correspond(s) to features of claim 3 and the claim(s) is/are rejected for the reasons detailed with respect to claim 3.
Per claim 18 (dependent on claim 15):
Boyer ‘990 in view of Agarwal ‘116 and Alayrac ‘936 discloses the elements detailed in the rejection of claim 15 above, incorporated herein by reference.
The limitations of the claim(s) correspond(s) to features of claim 4 and the claim(s) is/are rejected for the reasons detailed with respect to claim 4.
Per claim 19 (dependent on claim 15):
Boyer ‘990 in view of Agarwal ‘116 and Alayrac ‘936 discloses the elements detailed in the rejection of claim 15 above, incorporated herein by reference.
The limitations of the claim(s) correspond(s) to features of claim 5 and the claim(s) is/are rejected for the reasons detailed with respect to claim 5.
Per claim 20 (dependent on claim 15):
Boyer ‘990 in view of Agarwal ‘116 and Alayrac ‘936 discloses the elements detailed in the rejection of claim 15 above, incorporated herein by reference.
The limitations of the claim(s) correspond(s) to features of claim 6 and the claim(s) is/are rejected for the reasons detailed with respect to claim 6.
Allowable Subject Matter
Claim(s) 7 and 14 is/are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 7, the prior art of record (Boyer ‘990 in view of Agarwal ‘116 and Alayrac ‘936) does not disclose: “receive, from the threat modeling multimodal LLM, the application security data comprising: the threat data, the weakness data, and the security control data; and
input, into an LLM, the threat data, the weakness data, the security control data, and instructions for the LLM to generate the security risk summarization corresponding to a predetermined length of text that describes the threat data, the weakness data, and the security control data for the particular application.” In the recited context.
In particular, Boyer ‘990 indicates that a particular LLM may summarize a response generated by the cyber security system, and thus the resulting output may be characterized as a natural-language summary of cybersecurity information, including ongoing threats and potential future risks. However, it is not clear how such a summary specifically corresponds to the particular outputs recited in the claim. Moreover, Boyer ‘990 appears to be silent as to the claimed requirement that the output comprises “a predetermined length of text.”
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Fach et al., US-20190028498-A1 – data characterizing a software system is received and processed using a machine-learning model trained on historically generated threat models to generate a threat model identifying cybersecurity threats and corresponding security measures. The generated threat model is then provided, for example, by displaying, transmitting, or storing the threat model.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SANGSEOK PARK whose telephone number is (571)272-4332. The examiner can normally be reached Monday-Friday 7:30-5:30 and Alternate Fridays 9:00 am-5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, PHILIP CHEA can be reached at (571)272-3951. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SANGSEOK PARK/Primary Examiner, Art Unit 2499
Prosecution Insights