DETAILED ACTION
This non-final office action is responsive to application 18/382,763 as submitted on 23 October 2023.
Claim status is currently pending and under examination for claims 1-20 of which independent claims are 1, 10 and 16.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The following are the references relied upon in the rejections below:
Ali et al. (US 20250077916 A1)
Storhaug, André, et al. "Efficient Avoidance of Vulnerabilities in Auto-completed Smart Contract Code Using Vulnerability-constrained Decoding." arXiv preprint arXiv:2309.09826v2 (2023).
Claims 1-4, 7-8, 10-13, 16, 18 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Ali in view of Storhaug.
Regarding Claim 1, Ali teaches:
A processor, comprising one or more circuits to ([0095] processor, dedicated circuitry):
cause one or more random strings to be inserted into a prompt template ([0026] “The program segment in the instruction prompt template includes placeholders for program segment boundary tags and program instructions. The data segment in the instruction prompt template includes placeholders for data segment boundary tags and data instructions”
[0060] “The program segment includes the program instructions and is enclosed by the program segment boundary tags <207a0233-cda8-474f-aa96-10a99c5665a1>”
[0037] “The example python function below uses a Hugging Face transformers library to generate secure boundary segment tags for any token length”
At [0037] Ali discloses function random_tag_openai() (reproduced below) that generates boundary segment tags from randomized integers (token IDs). Boundary segment tags are comprised of letters and integers, therefore, boundary segment tags are comprised of random strings that are included (inserted) into a prompt template.
PNG
media_image1.png
614
1198
media_image1.png
Greyscale
);
assign, within a tokenizer, special token identifiers (IDs) to the one or more random strings ([0036] “The following example code illustrates how secure program segment boundary tags and data segment boundary tags can be generated for a GPT-2 LLM and a GPT 3 LLM. The vocabulary has a size of 50257 with a single special token <|endoftext|> of id 50256”
The function random_tag_openai() uses a tokenizer to decode token IDs generated from a special token ID into boundary segment tags comprised of random strings, therefore assigning within a tokenizer special token IDs to random strings.);
tokenize, using the tokenizer, text data in the prompt template ([0002] “The generative LLM relies on instruction prompts to perform novel or domain-specific tasks. The instruction prompts typically include program instructions and data instructions. The data instructions are often received from untrusted end-users. The data instructions are natural language instructions”
[0014] “The instruction prompt template also includes placeholders for the program segment boundary tags, the data segment boundary tags, the program instructions, and the data instructions”
Data instructions in the form of natural language instructions (‘text data’) are received from end-users. A generative LLM uses the data instructions to perform tasks. For a generative LLM to process data instructions to perform tasks, the natural language instructions must be tokenized to be understood by the LLM, therefore tokenizing text data in a prompt template is implied.),
wherein the one or more random strings are tokenized as the special token IDs (The function random_tag_openai() uses a tokenizer to decode token IDs generated from a special token ID into boundary segment tags comprised of random strings, therefore the random strings of a tag must have been tokenized as special token IDs.);
and send the tokenized data to a large language model (LLM) trained using [a special token] associated with the special token IDs ([0035-0036] “The generative LLM 104 does not typically use classic English alphabet characters or Unicode characters. The generative LLM 104 typically employs a vocabulary (set of tokens) learned from a large corpus of text data using algorithms, such as for example Byte Pair Encoding (BPE). Secure program segment boundary tag and data segment boundary tag creation typically take into account the vocabulary or token set utilized by each individual generative LLM 104. … The vocabulary has a size of 50257 with a single special token <|endoftext|> of id 50256.”
[0019] “the generative LLM implements the data instructions in the privilege based segmented instruction prompt and generates a response. In at least one embodiment, a privilege based segmented instruction prompt has N segments, each delineated with a unique segment boundary tag and assigned a distinct privilege level.”
A generative LLM uses data instructions (tokenized text data) in a privilege based segmented instruction prompt to generate a response. The privilege based segmented instruction prompt is delineated with boundary segment tags (random strings tokenized as special token IDs), therefore tokenized data is sent to the generative LLM for processing.
The generative LLM learns vocabulary (a set of tokens) that includes a special token. Vocabulary is used to generate boundary segment tags (which are assigned a randomized special token ID), therefore the generative LLM is trained using a special token associated with special token IDs.).
However, Ali does not teach sending tokenized data to a large language model (LLM) trained using multiple special tokens associated with special token IDs, which is taught by Storhaug:
and send the tokenized data to a large language model (LLM) trained using special tokens associated with the special token IDs ((P. 1, Abstract) “we fine-tune an LLM to include vulnerability labels when generating code, acting as an embedded classifier. Then, during decoding, we deny the model to generate these labels to avoid generating vulnerable code.”
(P. 6, Sec. III-B1, ¶2) “For each of the vulnerabilities described in section III-A4, we devise the following vulnerability labels: <DC>, <IOU>, <NC>, <RE>, <TD>, <TO>, <TOD>, <UcC>, <UpS>, and <FE>.”
(P. 6, Sec. III-B2, ¶1-2) “To perform vulnerability-tuning of the model on the labeled vulnerability dataset, we first add the vulnerability labels as special tokens to the tokenizer. This prevents the tokenizer from splitting the vulnerability labels into multiple already pre-trained tokens. For example, the “<IOU>” label is tokenized into four different tokens: “<”, “I”, “OU”, “>” with corresponding ids: 27, 40, 2606, 29. … the vulnerability labels are added as special tokens to the tokenizer, effectively expanding the vocabulary. We resize the model’s embedding matrix to accommodate the new tokens, adding randomly initialized vectors at the end.”
(P. 4, Sec. II-C, ¶1) “To restrict the model during decoding (generation), we provide the model with a list of forbidden tokens (labels)”).
Storhaug teaches fine-tuning an LLM to learn special tokens is a known method in the art. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to combine the method of Ali with the fine-tuning technique disclosed by Storhaug to train an LLM to learn multiple special tokens. By training an LLM to learn multiple special tokens, an LLM is able to learn how to delineate different types of data segments, thereby allowing the LLM to accurately parse a prompt and selectively choose which prompt instructions to execute.
Regarding Claims 2 and 11, the combined method of Ali/Storhaug teaches:
The processor of claim 1, wherein the one or more circuits are further to: encode the one or more special tokens with system instructions ([0060] “The program segment includes the program instructions and is enclosed by the program segment boundary tags <207a0233-cda8-474f-aa96-10a99c5665a1> and 207a0233-cda8-474f-aa96-10a99c5665a1> defined in the trusted segment. The program instructions have been provided by a programmer via a program device 108. The program instructions define the one or more tasks that the generative LLM 104 is to perform with respect to the data instructions received from an end-user.”
Program segment boundary tags are generated from a special token (see [0036-0037]) and are used to determine where program instructions begin and end, therefore program segment boundary tags are special tokens that are encoded with system instructions (since tags are able to differentiate program instructions from data instructions).).
Regarding Claims 3 and 12, the combined method of Ali/Storhaug teaches:
The processor of claim 2, wherein the system instructions encoded to at least one of the special tokens include semantic intent data ([0027] “the trusted instructions specify that if the program instructions in the program segment seek to extract, modify, or overrule the trusted instructions in the trusted segment of the privilege based segmented instruction prompt, the privilege based segmented instruction prompt is to be identified by the generative LLM 104 as an instruction injection attack. The trusted instructions have a higher privilege level than the program instructions. The program instructions are considered to be in conflict with the trusted instructions and in violation of the privilege levels associated with the trusted instructions and the program instructions if the program instructions seek to extract, modify, or overrule the trusted instructions.”
Program segment boundary tags are generated from a special token and are used to determine where program instructions begin and end, therefore program instructions written by a user attempting to extract, modify, or overrule trusted instructions is semantic intent data (and therefore program segment boundary tags are special tokens encoded with semantic intent data).).
Regarding Claims 4 and 13, the combined method of Ali/Storhaug teaches:
The processor of claim 1, wherein the one or more circuits are further to: train the LLM using the one or more special tokens ([0035-0036] “The generative LLM 104 typically employs a vocabulary (set of tokens) learned from a large corpus of text data using algorithms, such as for example Byte Pair Encoding (BPE). … The vocabulary has a size of 50257 with a single special token <|endoftext|> of id 50256.”).
Regarding Claim 7, the combined method of Ali/Storhaug teaches:
The processor of claim 1, wherein the one or more special tokens include a first token encoded with a first system instruction to demarcate a beginning of instructions in the prompt template ([0060] “The program segment includes the program instructions and is enclosed by the program segment boundary tags <207a0233-cda8-474f-aa96-10a99c5665a1> and 207a0233-cda8-474f-aa96-10a99c5665a1> defined in the trusted segment. The program instructions have been provided by a programmer via a program device 108. The program instructions define the one or more tasks that the generative LLM 104 is to perform with respect to the data instructions received from an end-user.”
[0014] “The instruction prompt template also includes placeholders for the program segment boundary tags, the data segment boundary tags, the program instructions, and the data instructions.”
Program segment boundary tags are generated from a special token (see [0036-0037]) and are used to determine where program instructions begin and end in a prompt template, therefore a program segment boundary tag that determines where program instructions begin is a special token that is encoded with a first system instruction to demarcate a beginning of instructions in a prompt template.)
and a second token encoded with a second system instruction to demarcate an end of the prompt template instructions (A program segment boundary tag that determines where program instructions end is a special token that is encoded with a second system instruction to demarcate an end of prompt template instructions.).
Regarding Claim 8, the combined method of Ali/Storhaug teaches:
The processor of claim 1, wherein the one or more special tokens include a first token encoded with a first system instructions to demarcate a beginning of a user input in the prompt template ([0061] “The data segment includes the data instructions provided by the customer and is enclosed by the data segment boundary tags <17d16563-4e16-4aa2-a9bf-d1939001aebd> and </17d16563-4e16-4aa2-a9bf-d1939001aebd> defined in the trusted segment. The data instructions are provided by the end-user.”
[0014] “The instruction prompt template also includes placeholders for the program segment boundary tags, the data segment boundary tags, the program instructions, and the data instructions.”
Data segment boundary tags are generated from a special token (see [0036-0037]) and are used to determine where data instructions provided by an end-user begin and end in a prompt template, therefore a data segment boundary tag that determines where data instructions begin is a special token that is encoded with a first system instruction to demarcate a beginning of user input in a prompt template.)
and a second token encoded with a second system instruction to demarcate an end of the user input (A data segment boundary tag that determines where data instructions end is a special token that is encoded with a second system instruction to demarcate an end of user input in a prompt template.).
Regarding Claim 10, the rejection of claim 1 is incorporated. The difference in scope being:
A method, comprising ([Abstract] “A method and system for generating a privilege based segmented instruction prompt”):
sending the tokenized data to a language model to perform inferencing (The Examiner interprets the sending the tokenized data as encompassing the exact same step as the send step in Claim 1.).
Regarding Claim 16, the rejection of claim 1 is incorporated. The difference in scope being:
A system, comprising: one or more processors to send secret data to a language model trained on special tokens associated with special token IDs by, at least in part, synchronizing generated random text between a prompt template and a tokenizer ([0023] system, processor
[0040] “The following example python code snippet can be used to generate and insert the program segment boundary tags and the data segment boundary tags into instruction prompt template”
On [0040] Ali discloses a code snippet (reproduced below) for randomly generating a program segment boundary tag and inserting the generated tag into an instruction prompt template.
PNG
media_image2.png
743
959
media_image2.png
Greyscale
The program tag is generated by the function random_tag_openai() which uses a tokenizer to decode token IDs generated from a special token ID into boundary segment tags comprised of random strings (see [0037]). Therefore, the program tag (comprised of random strings) inserted in the instruction prompt template is the same program tag generated with the tokenizer (and therefore the random strings between the prompt template and tokenizer are synchronized).
The Examiner interprets the send secret data step as encompassing the exact same step as the send step in Claim 1.),
… and tokenize data from the prompt template, including the random text, to send the corresponding special token IDs tokenized from the random text to the language model as the secret data (The Examiner interprets the “tokenize data … to send …” step as encompassing the exact same “tokenize” “and “send” steps in Claim 1.).
Regarding Claim 18, the combined method of Ali/Storhaug teaches:
The system of claim 16, wherein the one or more processors are further to: train the language model using the special tokens (Storhaug discloses at (P. 1, Abstract) “we fine-tune an LLM to include vulnerability labels when generating code”
(P. 6, Sec. III-B2, ¶1-2) “To perform vulnerability-tuning of the model on the labeled vulnerability dataset, we first add the vulnerability labels as special tokens to the tokenizer.”).
Regarding Claim 20, the combined method of Ali/Storhaug teaches:
The system of claim 16, wherein the system is comprised at least one of: a system for performing simulation operations; a system for performing simulation operations to test or validate autonomous machine applications; a system for rendering graphical output;
a system for performing deep learning operations ([0016] “program instructions enable the generative LLM to implement one or more tasks with respect to data instructions received from an end-user”);
a system implemented using an edge device; a system for generating or presenting virtual reality (VR) content; a system for generating or presenting augmented reality (AR) content; a system for generating or presenting mixed reality (MR) content;
a system incorporating one or more Virtual Machines (VMs) ([0068] “The application platform 518 manages the creation and storage of the applications into one or more database objects and the execution of the applications in one or more virtual machines in the process space of the system 516.”);
a system implemented at least partially in a data center; a system for performing hardware testing using simulation; a system for synthetic data generation; a collaborative content creation platform for 3D assets;
or a system implemented at least partially using cloud computing resources ([0088] “system 700 may be in the form of a computer system within which a set of instructions, for causing the machine to perform any one or more of the methodologies discussed … computer system 700 may represent, for example, elements of the cloud-based computing platform or any other elements of FIG. 1”).
The following are the references relied upon in the rejections below:
Clement et al. (US 20240386103 A1)
Claims 5-6, 9, 14-15, 17 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Ali in view of Storhaug, further in view of Clement.
Regarding Claims 5, 14 and 19, the combined method of Ali/Storhaug teaches:
The processor of claim 1, wherein the one or more circuits are further to: reference, using the LLM, the one or more special tokens based on the special token IDs included in the tokenized data ([0036] “Secure program segment boundary tag and data segment boundary tag creation typically take into account the vocabulary or token set utilized by each individual generative LLM 104. … The vocabulary has a size of 50257 with a single special token <|endoftext|> of id 50256.”
An LLM learns a special token with an id, therefore the LLM learns to map a special token to an id (and therefore reference using the LLM, the one or more special tokens based on the special token IDs).);
However, the combination does not teach generating a sequence of output token IDs and converting output tokens to character strings which is taught by Clement:
cause a sequence of output token IDs to be generated based at least in part on the one or more special tokens referenced by the LLM (The Examiner interprets “special token” according to its broadest reasonable interpretation in view of the applicant’s specification as encompassing a secret as disclosed by Clement. This interpretation is consistent with the descriptions in the Applicant’s specification at [0051] and [0057], (see excerpts below).
Applicant’s written description at [0051] “Special tokens may provide a clear separation between intended instructions and all other information that is sent to a language model from a prompt template. The clear separation provided by special tokens may prevent a language model from performing unintended behavior that would occur due to confusion about instructions. Special tokens may therefore provide isolation of instructions”
Applicant’s written description at [0057] “The special tokens may be encoded with information relevant to the language model, including but not limited to instructions, personas, and controls.”
[0019] “A secret is composed of a few random short-length natural language words separated by spaces. Natural language words are used since the model's input consists of token embeddings where a token represents a portion of natural language text of the prompt”
[0036] “the large language model is given initial instructions 202 to perform the target task. System prompts (<system> . . . </system>) are created by the developer of the conversational interactions with the model to constrain the model to acting in prescribed ways consistent with the intent and policies of the service hosting the large language model. For example, as shown in FIG. 2, the initial instructions 202 indicate, in part, that “you will always begin every response by repeating the SECRET in the user prompt.” This is the original goal (i.e., intent/policy) of the large language model. When the model generates a response that does not repeat the SECRET then it is assumed that a prompt injection attack has occurred.”
[Abstract] “A technique to prevent a prompt injection attack utilizes a security agent to sign a large language model prompt with a secret that is isolated from the user application or device that generates a user prompt.”
A secret is comprised of random natural language words that are represented as tokens when inputted into an LLM. The LLM is instructed to repeat the secret in a user prompt when generating its response, and if the secret is not repeated, then a prompt injection attack is assumed to have occurred. Therefore, the secret differentiates a prompt injection from a user prompt (and therefore a secret is a special token that separates (differentiates) instructions and prevents a language model from performing unintended behavior).
A secret is comprised of random natural language words (represented as tokens), therefore these words are used to identify the secret (and therefore are token IDs). An LLM uses a secret (‘special token’) in its generated response, therefore the secret in a generated response is a sequence of output token IDs.);
and parse the output sequence to convert the output tokens to character strings and remove the one or more random strings from the character strings ([0022] “When the secret is embedded in the response 112, then the security agent 104 knows that the model is still aligned to the original goal, scope, or instructions. In this case, the security agent 104 extracts the secret from the response and returns the remaining portion of the response back to the user application 116.”
To extract a secret from a generated response (‘output sequence’), the secret has to be parsed. Therefore, a secret comprised of random natural language words (random strings) is converted to character strings. When the secret is extracted (deleted) from the response, the secret’s random words (‘random strings’) are removed.).
Clement teaches deleting tokens from model-generated responses is a known method in the art. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to modify the combined method of Ali/Storhaug with the token deletion technique disclosed by Clement to delete tokens from model responses. By deleting tokens from model responses, users are not able to know which tokens classify prompts as malicious, thereby making it more difficult for users to learn which tokens bypass content filters.
Regarding Claims 6 and 15, the combined method of Ali/Storhaug/Clement teaches:
The processor of claim 5, wherein the text data includes input from a user and the one or more circuits are further to: (Ali discloses “The data instructions are often received from untrusted end-users.” [0002].)
provide the parsed output sequence to the user (Clement discloses “The security agent checks each response for the secret before passing the response back to the user application. If the secret is repeated, then the security agent extracts the secret out of the response and returns the remaining portion to the user application. If the secret is not repeated in the response generated by the large language model, then the security agent assumes that the there was a prompt injection attack.” [0017]. See [0019] describing a secret is composed of random tokens.
When a secret is extracted (deleted) from a response, the remaining portions of a response (‘parsed output sequence’) is returned to a user.).
Clement teaches deleting tokens from model-generated responses is a known method in the art. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to modify the combined method of Ali/Storhaug/Clement with the token deletion technique disclosed by Clement to delete tokens from responses sent to users. By deleting tokens from responses sent to users, users are not able to know which tokens classify prompts as malicious, thereby making it more difficult for users to learn which tokens bypass content filters.
Regarding Claim 9, the combined method of Ali/Storhaug teaches: The processor of claim 1, wherein the one or more circuits, however the combination does not teach updating and replacing random strings in a prompt template which is taught by Clement:
cause one or more updated random strings to be generated ([0019] “A secret is composed of a few random short-length natural language words separated by spaces. Natural language words are used since the model's input consists of token embeddings where a token represents a portion of natural language text of the prompt”
[0008] “the secret may be associated with a turn count which is a limit on how often a same secret is used in a conversation which may span several user prompts and responses within a single session. Alternatively, the turn count may limit how often a same secret is used for a particular user identifier. When the turn count limit is exceeded, a new secret is used in the conversation.”);
insert the one or more updated random strings in the prompt template to replace the one or more random strings (The Examiner interprets “prompt template” according to its broadest reasonable interpretation in view of the applicant’s specification as encompassing a LLM prompt as disclosed by Clement. This interpretation is consistent with the descriptions in the Applicant’s specification at [0027] and [0045], (see excerpts below).
Applicant’s written description at [0027] “The specifications may also include descriptions for information within the prompt template 108 which may include one or more markers or identified section. In one example, a user input location may be marked off in as a specific location in the prompt template 108 and identified to include "user data."”
Applicant’s written description at [0045] “the prompt template 202 may include demarcations for "instruction" start, "instruction" end, "user input" start, and "user input" end.”
[0038] “The user-prompt 202 (<user-prompt> . . . </user-prompt>) includes the question, “How can you help me?” The LLM-prompt 206 (<-prompt> . . . </LLM-prompt>) generated by the security agent includes the SECRET … and the user-prompt”
[0006] “A security agent is used sign a user prompt destined to a large language model from a user with a secret in order to prevent a prompt injection attack. … The secret is tailored for a specific user identifier and session identifier associated with the user prompt. The large language model is instructed to repeat the secret in each response. The security agent retrieves the response from the large language model and checks for the secret. When the secret is not part of the response, an error message is forwarded to the user application instead of the response.”
Clement discloses Figure 2 (reproduced below) depicting a LLM prompt that includes a secret and a user-prompt. The LLM prompt is enclosed by <LLM-prompt> tags and includes a secret and user-prompt. The LLM prompt distinguishes the secret from the user-prompt by assigning the secret to a variable SECRET, therefore the LLM prompt clearly defines where the LLM prompt, secret, and user-prompt (user input) begin and end (and therefore an LLM prompt is a prompt template).
PNG
media_image3.png
113
454
media_image3.png
Greyscale
Since an LLM is required to repeat a secret in each response, when the secret is updated with new random natural language words, the LLM uses the new secret in the LLM prompt (and therefore inserting updated random strings in the prompt template to replace the one or more random strings).);
and assign, within the tokenizer, the one or more updated random strings to the corresponding special token IDs (The Examiner interprets “special token” according to its broadest reasonable interpretation in view of the applicant’s specification as encompassing a secret as disclosed by Clement. This interpretation is consistent with the descriptions in the Applicant’s specification at [0051] and [0057].
A secret is comprised of random natural language words that are represented as tokens when inputted into an LLM (see [0019]). The LLM is instructed to repeat the secret in a user prompt when generating its response, and if the secret is not repeated, then a prompt injection attack is assumed to have occurred (see [0006]). Therefore, the secret differentiates a prompt injection from a user prompt (and therefore a secret’s tokens are special tokens that separate (differentiate) instructions and prevents a language model from performing unintended behavior).
A secret’s words are represented as tokens, therefore these tokens are used to identify the secret (and therefore random strings are assigned to special token IDs). A tokenizer that assigns natural language words to tokens is implied since a tokenizer is required to map natural language to token embeddings.).
Clement teaches updating random strings and their corresponding tokens in LLM responses is a known method in the art. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to modify the combined method of Ali/Storhaug with the technique disclosed by Clement to update tokens used in LLM responses. By updating tokens used in LLM responses, tokens used to authenticate user prompts are always changing, thereby making it more difficult for adversarial users to guess tokens and decreasing the likelihood of users injecting malicious instructions into prompts.
Regarding Claim 17, the combined method of Ali/Storhaug teaches The system of claim 16, however, the combination does not teach preventing random text from being publicly exposed while in the prompt template and assigned within the tokenizer, which is taught by Clement:
wherein the random text are prevented from being publicly exposed while in the prompt template and assigned within the tokenizer (The Examiner interprets “prompt template” according to its broadest reasonable interpretation in view of the applicant’s specification as encompassing a LLM prompt as disclosed by Clement. This interpretation is consistent with the descriptions in the Applicant’s specification at [0027] and [0045].
[0019] “A secret is composed of a few random short-length natural language words separated by spaces. Natural language words are used since the model's input consists of token embeddings where a token represents a portion of natural language text of the prompt”
[0038] “The LLM-prompt 206 (<-prompt> . . . </LLM-prompt>) generated by the security agent includes the SECRET … and the user-prompt”
[0017] “The security agent checks each response for the secret before passing the response back to the user application. If the secret is repeated, then the security agent extracts the secret out of the response and returns the remaining portion to the user application. If the secret is not repeated in the response generated by the large language model, then the security agent assumes that the there was a prompt injection attack.”
Clement discloses Figure 2 (reproduced above) depicting a LLM prompt that includes a secret and a user-prompt. The LLM prompt is enclosed by <LLM-prompt> tags and includes a secret and user-prompt. The LLM prompt distinguishes the secret from the user-prompt by assigning the secret to a variable SECRET, therefore the LLM prompt clearly defines where the LLM prompt, secret, and user-prompt (user input) begin and end (and therefore an LLM prompt is a prompt template).
A secret is comprised of random natural language words (random text) that are mapped to tokens. A tokenizer is implied by assigning token embeddings to natural language. When a response (LLM prompt) containing a secret is returned to a user, the secret is first extracted (deleted) from the response, therefore preventing the random words of the secret from being exposed to the user while in a prompt template and during token assignment.).
Clement teaches deleting tokens from model-generated responses is a known method in the art. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to modify the combined method of Ali/Storhaug with the token deletion technique disclosed by Clement to delete tokens from responses sent to users. By deleting tokens from responses sent to users, users are not able to know which tokens classify prompts as malicious, thereby making it more difficult for users to learn which tokens bypass content filters.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Cefalu et al. (US 12118471 B2) teaches mitigating prompt injection attacks by converting user prompt instructions into tokens and attaching a tag (that indicates the type of instruction) to each token.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PEDRO J MORALES whose telephone number is (571)272-6106. The examiner can normally be reached 8:30 AM - 6:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, MIRANDA M HUANG can be reached at (571)270-7092. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PEDRO J MORALES/Examiner, Art Unit 2124
/VINCENT GONZALES/Primary Examiner, Art Unit 2124
Prosecution Insights