Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 6/15/26 have been fully considered but they are not persuasive.
Applicant sets forth two arguments.
1) Applicant argues that Rozenberg teaches a runtime profile of a virtual container, and that the claims at issue teach a runtime profile of machine learning model.
Examiner asserts that the profile as taught by Rozenberg and in view of Lang are of a ML model instigated in a container. Thus the profile is a ML model. Lang teaches deviations from a “baseline” which is interpreted as a profile. Rozenberg teaches a runtime profile of a container. Said container should be loaded with the ML model and thus the profile is the standard behavior of the ML model.
2) Applicant argues that the invention relies on a list of system calls.
Examiner has incorporated Moyal US 2020/0026868 to meet the claims as amended.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-7, 13-16, 21-26 is/are rejected under 35 U.S.C. 103 as being unpatentable over Rozenberg US 2024/0004993 in view of Lang US 2024/0202405 in view of Moyal US 2020/0026868.
As per claim 1. Rozenberg teaches accessing a container image based at least in part on information associated with the ML model; loading a software container using the container image; loading the ML model within the container; detecting a plurality of system calls from the ML model within the container; categorizing one or more system calls of the plurality of system calls to be suspicious; responsive at least in part on categorizing the one or more system calls of the plurality of system calls to be suspicious, tagging the ML model as possibly being malicious; and outputting a result indicative of the ML model being tagged as possibly being malicious. [0005][0027][0028][0036][0049] (teaches a container mounting system and behavior profile of the container, to monitor the container in realtime, and detect system calls, and to isolate system calls that deviate from the behavior profile in order to detect suspicious and malicious activities)
Rozenberg fails to teach that the system calls are made by a ML model.
Lang teaches receiving a ML model. Lang teaches loading the ML model in a container, and tagging the ML model as suspicious. [0268] (deployment to container) [0274] (monitoring of AI) [0360] (model drift, malicious data, logging api calls, etc) [0366] (detecting anomalies in the model) Lang teaches outputting a result indicative of ML model being possibly malicious [0281] (alert)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the current invention to use the teaching of Lang with the prior art because it improves the security of the system.
Moyal teaches accessing a safe list of system calls and or a suspicious list of system calls representing expected behaviors of machine learning models; analyzing one or more of the plurality of system calls from the ML model in view of the safelist of system calls and/or the suspicious list of system calls, and categorizing one or more of the calls as suspicious based in part on the analyzing. [0012]-[0015] (monitors system calls from a program in a sandbox and compares the system calls to a white list or blacklist; investigates the suspicious code if the system calls are unknown)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the current invention to use the teaching of Moyal with the prior art because it helps prevent false positives.
As per claim 2. Lang teaches the non-transitory computer-readable medium of claim 1, wherein: receiving information associated with the ML model comprises receiving an identification of a software framework of the ML model; and accessing the container image comprises one of (i) selecting the container image from among a plurality of container images or (ii) building the container image, such that the selected or built container image supports the software framework of the ML model. [0268] (deployment to container)
As per claim 3. The non-transitory computer-readable medium of claim 1, Rozenberg teaches wherein the operations further include: defining a test for the ML model; and executing the test on the ML model, wherein the test on the ML model includes loading the ML model within the container; wherein detecting the plurality of system calls from the ML model comprises detecting the plurality of system calls from the ML model, while the test is being executed on the ML model. [0005][0027][0028][0036][0049] (teaches a container mounting system and behavior profile of the container, to monitor the container in Realtime, and detect system calls, and to isolate system calls that deviate from the behavior profile in order to detect suspicious and malicious activities)
Lang teaches receiving a ML model. Lang teaches loading the ML model in a container, and tagging the ML model as suspicious. [0268] (deployment to container) [0274] (monitoring of AI) [0360] (model drift, malicious data, logging Api calls, etc.) [0022][0120][0126] (detecting inputs and outputs and detecting abnormalities)
As per claim 4. Lang teaches the non-transitory computer-readable medium of claim 3, wherein the test on the ML model further includes the ML model to output an inference, based on an input. [0022][0120][0126] (detecting inputs and outputs and detecting abnormalities)
As per claim 5. The non-transitory computer-readable medium of claim 4, Lang teaches wherein the test on the ML model comprises one of: issuing a load command to load the ML model, and an inference command to output the inference; or issuing a pipeline command to load the ML model and to output the inference. [0120] [0137] (teaches analysis and inference inputs/outputs)
As per claim 6. The non-transitory computer-readable medium of claim 3, Lang teaches wherein defining the test for the ML model comprises: selecting the test from among a plurality of tests for the ML model, or receiving the test via an user interface. [0119][0120][0129] (teaches an interface where user chooses how to analyze the AI model)
As per claim 7. The non-transitory computer-readable medium of claim 1, Lang teaches wherein the operations further include: receiving the information associated with the ML model, wherein the information comprises one or more of (i) a software framework of the ML model, (ii) a path to a storage repository that stores the model, and (iii) a checksum of one or more files including the ML model. [0196] (framework files)
As per claim 13. The non-transitory computer-readable medium of claim 1, Rozenberg teaches wherein the operations further include: prior to loading the ML model within the container, configuring the container to use a subset of a plurality of resources accessible to the container. [0060][0062][0078] (providing the appropriate resource configuration)
As per claim 14. The non-transitory computer-readable medium of claim 1, Rozenberg teaches wherein the ML model is a pre-trained ML model. [0124][0143][0145] (teaches a plurality of models that are trained before deployment and storing the models in a database)
As per claim 15. The non-transitory computer-readable medium of claim 1, Rozenberg teaches wherein the operations further include: receiving the ML over the network from a model repository storing a plurality of pre-trained ML models. [0124][0143][0145] (teaches a plurality of models that are trained before deployment and storing the models in a database)
As per claim 16. The non-transitory computer-readable medium of claim 1, Rozenberg teaches wherein categorizing the one or more system calls of the plurality of system calls to be suspicious comprises: categorizing a first system call of the plurality of system calls to have a first level of suspicion; and categorizing a second system call of the plurality of system calls to have a second level of suspicion that is different from the first level of suspicion. [0023][0047][0049] (teaches a score compared to a threshold and determining whether benign or malware; and suspicious)
As per claim 21. Rozenberg teaches A computer implemented method comprising: accessing a container image based at least in part of information associated with the ML model; loading a software container using the container image; detecting a plurality of system calls from the ML model within the container; categorizing one or more system calls of the plurality of system calls to be suspicious; responsive at least in part to categorizing the one or more system calls of the plurality of system calls to be suspicious, tagging the ML model as possibly being malicious; and outputting a result indicative of the ML model being tagged as possibly being malicious.
[0005][0027][0028][0036][0049] (teaches a container mounting system and behavior profile of the container, to monitor the container in realtime, and detect system calls, and to isolate system calls that deviate from the behavior profile in order to detect suspicious and malicious activities)
Rozenberg fails to teach that the system calls are made by a ML model.
Lang teaches receiving a ML model. Lang teaches loading the ML model in a container, and tagging the ML model as suspicious. [0268] (deployment to container) [0274] (monitoring of AI) [0360] (model drift, malicious data, logging api calls, etc) [0366] (detecting anomalies in the model) Lang teaches outputting a result indicative of ML model being possibly malicious [0281] (alert)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the current invention to use the teaching of Lang with the prior art because it improves the security of the system.
Moyal teaches accessing a safe list of system calls and or a suspicious list of system calls representing expected behaviors of machine learning models; analyzing one or more of the plurality of system calls from the ML model in view of the safelist of system calls and/or the suspicious list of system calls, and categorizing one or more of the calls as suspicious based in part on the analyzing. [0012]-[0015] (monitors system calls from a program in a sandbox and compares the system calls to a white list or blacklist; investigates the suspicious code if the system calls are unknown)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the current invention to use the teaching of Moyal with the prior art because it helps prevent false positives.
As per claim 22. The method of claim 21, wherein: Rozenberg teaches receiving information associated with the ML model comprises receiving an identification of a software framework of the ML model; and accessing the container image comprises selecting the container image from among a plurality of container images, such that the selected or built container image supports the software framework of the ML model. [0025] (container orchestration)
Lang teaches receiving a ML model. Lang teaches loading the ML model in a container, and tagging the ML model as suspicious. [0268] (deployment to container) [0274] (monitoring of AI) [0360]
As per claim 23. The method of claim 21, Rozenberg teaches wherein categorizing the one or more system calls of the plurality of system calls to be suspicious comprises: determining that (i) a first system call of the plurality of system calls is within an expected behavior of the ML model, and (ii) a second system call of the plurality of system calls is outside the expected behavior of the ML model; and categorizing the second system call to be suspicious, without categorizing the first system call to be suspicious. [0005][0026] [0041] [0047] [0049](detecting both malicious and benign behaviors)
As per claim 24. Lang teaches receiving a first machine learning (ML) model and a second ML model over a network; loading the first ML model within a first container and the second ML model within a second container; [0268] (deployment to container) [0274] (monitoring of AI) [0360] (model drift, malicious data, logging api calls, etc) [0366] (detecting anomalies in the model) Lang teaches outputting a result indicative of ML model being possibly malicious [0281] (alert)
Examiner asserts that Lang teaches a system that loads ML models and is applicable to more than just a first ML model.
Rozenberg teaches detecting a first plurality of system calls from the first ML model within the first container and a second plurality of system calls from the second ML model within the second container; determining that (i) first one or more system calls of the first plurality of system calls from the first ML model deviate from an expected runtime behavior profile of machine learning models, and (ii) the second plurality of system calls from the second ML model adhere to the expected runtime behavior profile of machine learning models; responsive to the determining, tagging the first ML model as possibly being malicious and tagging the second ML model as possibly being safe; and outputting a result indicative of the first ML model being tagged as possibly being malicious and second ML model being tagged as possibly being safe.
[0005][0026][0027][0028][0036][0038][0049] (teaches a container mounting system and behavior profile of the container, to monitor the container in realtime, and detect system calls, and to isolate system calls that deviate from the behavior profile in order to detect suspicious and malicious activities, and marking as suspicious or safe) [0041] [0047] (detecting both malicious and benign behaviors)
Rozenberg fails to teach that the system calls are made by a ML model.
It would have been obvious to one of ordinary skill in the art before the effective priority date of the current application to use Rozenberg with the prior art because it increases security.
Moyal explicitly teaches labeling as safe or malicious. [0012]-[0015] (monitors system calls from a program in a sandbox and compares the system calls to a white list or blacklist; investigates the suspicious code if the system calls are unknown)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the current invention to use the teaching of Moyal with the prior art because it helps prevent false positives.
As per claim 25. The system of claim 24, wherein Lang teaches the ML model is a pretrained ML model [0124][0143][0145] (teaches a plurality of models that are trained before deployment and storing the models in a database)
Lang teaches code may be received over the internet from cloud based open source repository. [0299]
As per claim 26. (New) The non-transitory computer-readable medium of claim 1, Rozenberg teaches wherein the ML model is a first ML model, the plurality of system calls is a first plurality of system calls, the container is a first container, and the operations further include: receiving a second ML model; loading the second ML model within a second container; detecting a second plurality of system calls from the second ML model within the second container; and [0005][0027][0028][0036][0049] (teaches a container mounting system and behavior profile of the container, to monitor the container in realtime, and detect system calls, and to isolate system calls that deviate from the behavior profile in order to detect suspicious and malicious activities)
Moyal teaches accessing a safe list of system calls and or a suspicious list of system calls representing expected behaviors of machine learning models; analyzing one or more of the plurality of system calls from the ML model in view of the safelist of system calls and/or the suspicious list of system calls, and categorizing one or more of the calls as suspicious based in part on the analyzing. [0012]-[0015] (monitors system calls from a program in a sandbox and compares the system calls to a white list or blacklist; investigates the suspicious code if the system calls are unknown)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the current invention to use the teaching of Moyal with the prior art because it helps prevent false positives.
analyzing one or more of the second plurality of system calls from the second ML model in view of the safe list of system calls and/or the suspicious list of system calls; categorizing one or more system calls of the second plurality of system calls to be safe or suspicious;
Lang teaches outputting a result indicative of the categorization of one or more system calls of the second plurality of system calls. [0268] (deployment to container) [0274] (monitoring of AI) [0360] (model drift, malicious data, logging api calls, etc) [0366] (detecting anomalies in the model) Lang teaches outputting a result indicative of ML model being possibly malicious [0281] (alert)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the current invention to use the teaching of Lang with the prior art because it improves the security of the system.
Claim(s) 8-10, 12, 17-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Rozenberg US 2024/0004993 in view of Lang US 2024/0202405 in view of Yeung US 12,174,954.
As per claim 8. The non-transitory computer-readable medium of claim 1, Yeung teaches wherein categorizing one or more system calls of the plurality of system calls to be suspicious comprises: detecting (i) a first system call to access a first file including sensitive information, and (ii) a second system call to access a second file including non-sensitive information; and categorizing the first system call to be suspicious, without categorizing the second system call to be suspicious. (Column 4 lines 41-62) (Column 8 lines 9-41) (teaches classification as benign or suspicious or malware) (Column 9 line 24-30) (Column 11 lines 4-20; lines 43-55) (teaches detection of calls to leak sensitive information which is suspicious)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the current application to use the teaching of Yeung with the prior art because it improves security.
As per claim 9. The non-transitory computer-readable medium of claim 8, Yeung teaches wherein: the sensitive information comprises one or more of (i) a plurality of passwords, (ii) one or more private keys, and (iii) operating system level information stored in the first file that is a system file; and the non-sensitive information comprises data that is to be accessed by a legitimate ML model during its course of operation. (Column 11 lines 44-55) (attempting to leak passwords)
As per claim 10. The non-transitory computer-readable medium of claim 1, Yeung teaches wherein categorizing the one or more system calls of the plurality of system calls to be suspicious comprises: detecting the one or more system calls to at least one of (i) bind to a network port, (ii) establish a non-Transport Layer Security (non-TLS) connection with an Internet Protocol (IP) address, (iii) attempt to establish a connection with an IP address that is excluded from a list of predefined IP addresses, and (iiiv) requests access for more than a threshold number of times to computing resources outside a current directory of the ML model; and categorizing the one or more system calls of the plurality of system calls to be suspicious, responsive at least in part on detecting the one or more system calls to at least one of (i) bind to the network port, (ii) establish the non-TLS connection, (iii) attempt to establish a connection with an IP address that is excluded from a list of predefined IP addresses, and (iv) requests access for more than the threshold number of times. (Column 4 lines 41-62) (Column 8 lines 9-41) (teaches classification as benign or suspicious or malware) (Column 9 line 24-30) (Column 11 lines 4-20; lines 43-55) (teaches detection of calls to leak sensitive information which is suspicious) (Column 9 lines 60 to Column 10 line 11) (teaches upon attack detection, blocking IP address, and establishing a blacklist of IP addresses to filter further communications)
As per claim 12. The non-transitory computer-readable medium of claim 1, Yeung teaches wherein the operations further include: prior to loading the ML model within the container, configuring a network mode of the container to disable, thereby one or both of (i) disabling inbound traffic to the container, and (ii) disabling outbound traffic from the container. (Column 9 line 62- Column 10 line 8) (blocking inbound traffic)
As per claim 17. The non-transitory computer-readable medium of claim 16, Yeung teaches wherein the operations further include: assigning a score to the ML model, the score indicative of a degree of suspicion associated with the ML model being malicious. (Column 9 line 9-65) (score the result from malicious, to highly suspicious to benign)
Rozenberg teaches wherein the operations further include: assigning a score to the ML model, the score indicative of a degree of suspicion associated with the ML model being malicious. [0023][0047][0049] (teaches a score compared to a threshold and determining whether benign or malware)
As per claim 18. The non-transitory computer-readable medium of claim 17, Yeung teaches wherein the score is based at least in part on (i) a first number of system calls having the first level of suspicion, and (ii) a second number of system calls having the second level of suspicion. (Column 9 line 9-65) (score the result from malicious, to highly suspicious to benign)
Rozenberg teaches wherein the score is based at least in part on (i) a first number of system calls having the first level of suspicion, and (ii) a second number of system calls having the second level of suspicion. [0023][0047][0049] (teaches a score compared to a threshold and determining whether benign or malware)
As per claim 19. The non-transitory computer-readable medium of claim 17, Rozenberg teaches wherein tagging the ML model as possibly being malicious comprises: tagging the ML model as possibly being malicious, responsive at least in part on the score being higher than a threshold value. [0049] (score vs threshold label malicious)
Claim(s) 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Rozenberg US 2024/0004993 in view of Lang US 2024/0202405 in view of Yeung US 12,174,954 in view of Kim US 2020/0285733.
As per claim 20. The non-transitory computer-readable medium of claim 17, Kim teaches wherein detecting the plurality of system calls comprises: detecting the plurality of system calls using a strace utility. [0043] (teaches using a container system call monitor using strace)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the current invention to use teaching of Kim with the prior art because it improves security.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHRISTOPHER BROWN whose telephone number is (571)272-3833. The examiner can normally be reached M-F 8-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Luu Pham can be reached at (571) 270-5002. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHRISTOPHER J BROWN/Primary Examiner, Art Unit 2439
Prosecution Insights