DETAILED ACTION
Claims 1-20 remain for examination.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 7/13/26 have been fully considered but they are not persuasive. Regarding the Santana de Olivera reference, Applicant argues:
Santana describes a different arrangement. The penetration-testing environment 110 of Santana, which the Office relies on for disclosing the features related to the first set of command outputs, includes a test application 120 with its own runtime, user interface (UI), and separate test database. See Santana at para. [0044]. Santana expressly states that "[s]eparate from the penetration-testing environment 110 may be at least one production environment 128" (Santana at para. [0048], emphasis added). In that separate production environment, proxy 132 distinguishes malicious traffic from legitimate traffic and routes malicious traffic to production application clone 148. See Santana at para. [0050]. Santana further discloses that a runtime deceptive agent references a trained reinforcement-learning model and selects an appropriate response to the attacker's request. See Santana at para. [0071].
These disclosures in Santana are not equivalent to the claimed shared knowledge-base workflow of claim 1. The application-security testing logs and session traces in Santana are generated in a penetration-testing environment 110, which Santana describes as being "separate from" the production environment in which the attacker is routed to a production application clone. The Office action is attempting to map different components performing different functions in separate environments (i.e., the penetration-testing traces or logs for training, and the production application clone or deceptive agent for live attacker interaction) to the "knowledge base" recited in claim 1, which is the same knowledge base that is populated with an entry including the command and the first set of command outputs and also from which the command output selected from the second set of command outputs is output to the user.
This distinction between claim 1 and Santana is not merely a difference in terminology, but rather provides certain non-obvious advantages that are not provided by Santana. In particular, the architecture of claim 1 enables the honeypot entity to combine low-interaction and high-interaction honeypot behavior in a specific way (see para. [0010] of the present application). The honeypot entity can select known responses from a knowledge base in a lightweight manner, while also expanding that same response repertoire by executing commands in controlled backend systems when assessment is required. The separate penetration-testing and production-clone arrangement in Santana does not provide that architecture or the same advantages.
In response, the Examiner finds two flaws with this argument. First, the claims as presented make no requirement as to whether or not all the limitations of the claims are intended to apply to a live production environment, as opposed to what appears to be the preferred embodiment of Santana’s invention where a knowledge base is populated first via training in a simulated environment intended for penetration testing, wherein said knowledge base can then subsequently be used to process a second set of command outputs in the live production environment. Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). Second, Santana further teaches that training is not limited to knowledge gleaned from penetration testing in a separate environment, but alternate embodiments are permitted, including using data from actual attacks on the network, as per paragraph 0078:
[0078] As performed in 302, detection may also be based at least in part on machine learning or reinforcement learning based on training sets from penetration testing, actual attacks from previously, or a combination thereof. Training may be an iterative process, for example.
The remaining arguments regarding the Dowling reference are rebutted for substantially similar reasons as discussed supra.
Claim Rejections - 35 USC § 102
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claims 1-8 and 12-20 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Santana de Oliveira (U.S Patent Publication 2021/0377307; hereinafter, “Santana”).
Regarding claim 1:
Santana discloses a honeypot entity, comprising: a storage medium storing instructions (paragraphs 0097-0099, and Figure 4); and a processor configured to execute the instructions (paragraph 0075, and Figure 4) to cause the honeypot entity to: receive a command of a user (paragraphs 0044-0049, including ¶0045:“Pentester 104 may use browser 106 and/or any pentest tools 108 (which may be identical to any hacking tools 168) to conduct penetration testing on the test application 120 in the penetration testing environment 110 (also known as a security testing environment)” and ¶0049: “Attacker 104 may use browser 166 and/or any hacking tools 168 (which may be identical to any pentest tools 108) to conduct an attack, presumably for malicious purposes as opposed to pre-authorized, sanctioned penetration testing. However, in some use cases, attacker 164 may be a hired security researcher tasked with penetration testing from the outside…”); and determine whether an assessment of the command is required (paragraph 0050: “…proxy 132 may use additional logic, in some embodiments, to distinguish malicious traffic from legitimate traffic and to identify attacker 164, thereby routing legitimate traffic to production application 140 and routing malicious traffic to production application clone 148, at least for at least one identified attacker 164”); based on determining that the assessment of the command is required: arrange for execution of the command by a number of controlled backend systems (paragraph 0050: “In some embodiments, different attackers may be routed to separate instances of production application clone 148.”), retrieve a first set of command outputs associated with the command from the controlled backend systems, and populate a knowledge base with an entry including the command and the first set of command outputs (paragraphs 0068-0069, including ¶0068: “In some embodiments, a streamlined representation corresponding to the application responses may be saved by collecting logs during the application security testing process”); retrieve a second set of command outputs associated with the command from the knowledge base (paragraph 0071: “During runtime, provided that at least one honeytoken 138 has been triggered and that the corresponding session has been correctly re-routed by the proxy, a non-authenticated attacker may then interact only with an autonomous deceptive agent, according to some embodiments. Thus, the deceptive agent may autonomously reference a trained agent model (reinforcement-learning model 178) and accordingly select therefrom an appropriate response to the attacker's request(s) at a given step”; note the distinction between the initial pen-testing phase comprising the first set of inputs and outputs used for training, vs. live deployment of the honeypot against a real attacker); select a command output from the second set of command outputs based on a policy that seeks to maximize a value function (paragraph 0071, Ibid; see also paragraph 0051: “For real-time response to an actual live attack, model 178 may be used, e.g., by a given agent 158 to select a response action that has a greater probability of yielding a better reward for the given agent 158 based on input received from attacker 164”); output the command output selected from the second set of command outputs to the user (Ibid); determine an immediate reward associated with the command output selected from the second set of command outputs (see the examples of using reward functions to determine a reward for a particular response at paragraphs 0029, 0030, & 0032); and adapt the policy based on an interaction history associated with the user, the command output selected from the second set of command outputs, and the immediate reward (paragraph 0043: “Moreover, a defensive agent may learn how to craft HTTP response structures following a policy configured to improve future reward of any subsequent actions of the agent, following observed actions of an attacker. In a case of offensive automation of attacks, offensive agents may learn how to craft HTTP requests following a policy configured to increase a likelihood of exploitation of vulnerabilities, according to some embodiments”).
Regarding claims 15 and 18:
The rejection of claim 1 applies mutatis mutandis to claims 15 & 18.
Regarding claim 2: Santana further discloses wherein the processor executing the instructions further causes the honeypot entity to: adapt the policy based on reinforcement learning (paragraphs 0065-0067).
Regarding claim 3: Santana further discloses wherein the reinforcement learning comprises Q-learning (paragraphs 0059 & 0066).
Regarding claims 4, 16, and 19: Santana further discloses wherein the policy defines an expected value of a total reward for the command output selected from the second set of command outputs given the interaction history associated with the user; and wherein the value function comprises a sum of immediate rewards associated with the interaction history (paragraph 0066, including: “A Q-table may be defined as a list or equivalent representation of greatest expected future reward(s) for a given action at a given state…”).
Regarding claims 5, 17, and 20: Santana further discloses wherein the processor executing the instructions further causes the honeypot entity to: update the expected value of the total reward for the command output selected from the second set of command outputs based on the interaction history, the command output selected from the second set of command outputs, and the immediate reward (paragraph 0072).
Regarding claim 6: Santana further discloses wherein the interaction history comprises one or more most recent received commands associated with the user (see the various logs of paragraph 0079).
Regarding claim 7: Santana further discloses wherein the interaction history comprises the one or more most recent commands associated with the user and respective selected command outputs (Ibid).
Regarding claim 8: Santana further discloses wherein the immediate reward comprises at least one of: a negative first value upon discontinued interaction with the user; a positive second value upon continued interaction with the user; or a positive third value based on the command not being included in the knowledge base (e.g. paragraph 0058: “A deceptive agent (defensive agent) may be set to receive a positive reward for a repeated request from the attacker to exploit a parameter for a given web application page that is supposed to be vulnerable, to the extent that the attacker expends time or resources without becoming discouraged, giving up, or, in some use cases, exploiting what may otherwise be a critical vulnerability for a given application, service, system, or network, for example”)
Regarding claim 12: Santana further discloses wherein the processor executing the instructions further causes the honeypot entity to: classify the command (paragraph 0027); and arrange for execution of the command by the number of controlled backend systems based on classifying the command (see the examples of the invention processing different types of commands at paragraphs 0029-0032).
Regarding claim 13: Santana further discloses wherein the processor executing the instructions further causes the honeypot entity to classify the command into one or more of: a command of a UNIX computer system; a command of a Windows® computer system; a command of a network routing system; or a command of an Internet of Things (IoT) device (paragraph 0101).
Regarding claim 14: Santana further discloses wherein the processor executing the instructions further causes the honeypot entity to: suggest enhancing a functionality of the number of controlled backend systems based on the first set of command outputs comprising zero command outputs (paragraphs 0068-0069, including “Session traces may or may not contain the full input and output content from the application requests and responses”).
Claim Rejections - 35 USC § 103
Claims 9-11 are rejected under 35 U.S.C. 103 as being unpatentable over Santana as applied to claim 1 above, and further in view of “Using Reinforcement Learning to Conceal Honeypot Functionality” (hereinafter, “Dowling”).
Regarding claim 9: Santana appears to be silent regarding wherein determining that the assessment of the command is required comprises determining that the command is not included in the knowledge base. However, Dowling discloses a related invention for using reinforcement learning to train honeypots comprising this limitation (page 348, “3, Reinforcement Learning Honeypot Implementation”, including “Attacker commands can be categorized into the following…O – Other commands: Unhandled keystrokes such as ENTER and unknown commands”; see also Table 2 on page 351). It would have been obvious prior to the effective filing date of the instant application for Santana’s honeypot – which as discussed supra, is already known to employ reinforcement learning – to account for the possibility that it will receive an unknown command from an attacker, as doing so was a known option within the ability of a person of ordinary skill in the art, in order to train the honeypot to respond more effectively and overcome automated honeypot detection techniques (e.g. Dowling, page 346, 1st paragraph).
Regarding claim 10: The combination further discloses wherein determining that the assessment of the command is required comprises determining that the command is not associated with at least one command output for which the reward is in excess of a threshold (Dowling, page 346, “Markov decision processes”, including “q(.|s,a) defines a probability distribution governing the rewards received…”).
Regarding claim 11: The combination further discloses wherein determining that the assessment of the command is required comprises determining that at least one associated command output requires a refresh in accordance with a refresh probability (Dowling, page 352, “Boltzmann Softmax – A discrete explorer that executes actions with a probability that depends on their action values”).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Thomas A Gyorfi whose telephone number is (571)272-3849. The examiner can normally be reached 10:00am - 6:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amir Mehrmanesh can be reached at 571-270-3351. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
THOMAS A. GYORFI
Examiner
Art Unit 2435
/THOMAS A GYORFI/Examiner, Art Unit 2435 9/18/2026
/AMIR MEHRMANESH/Supervisory Patent Examiner, Art Unit 2435
