DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-6, 8-13, 15-19 and 21-23 have been examined.
Response to Arguments
Applicant's arguments filed on 11/19/25 have been fully considered but they are not persuasive.
Applicant mainly argues that none of the references disclose generate embeddings of sentences as feature vectors according to amended claims and because none of the references classify or evaluate individual sentences for maliciousness when detecting prompt injection attacks. Applicant asserts that this sentence level analysis facilitates identifying a source of malicious instructions within an input sequence as opposed to generally labeling an input sequence as malicious. However, the examiner disagrees.
In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
The combination of references are directed toward the same field of endeavor as instant application, which is to classify generative AI prompts are being malicious or benign by tokenizing sentences to generate feature vectors. Specifically, Jackson and Rideout both teach or at least suggest tokenizing input to generate embeddings for natural language processing (Jackson: [0021]-[0025]: natural language processing algorithms…tokenization and embeddings allow the model to convert discrete tokens into a format that can be processed by the neural network; Rideout: col. 9 ll. 6-18: transform input into sentence embeddings before input into prompt injection classifier). Therefore, applicant’s argument is not persuasive in light of above explanation.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-6, 8-13 and 15-19 are rejected under 35 U.S.C. 103 as being unpatentable over Jackson U.S. 2025/0103715 (hereinafter Jackson) in view of Vaknin et al. U.S. 2025/0209208 (hereinafter Vaknin) further in view of Rideout et al. U.S. 12,248,883 (hereinafter Rideout).
As per claim 1, Jackson discloses a method comprising:
intercepting an input sequence comprising task instructions for a language model, where the input sequence comprises potentially compromised data (Jackson: [0032]-[0034]: indirect prompt injection attack using instructions comprising potentially compromised data; [0075]-[0076]: pass the received prompt to pre-filter);
generating feature vectors for the input sequence (Jackson: [0023]-[0025]: embeddings are continuous vector representations of words or token; [0065]: process input data via sentence parsing, word segmentation, stemming and lemmatization and tokenization for phrases, sentences, paragraphs and passages);
inputting the feature vectors into a machine learning model to obtain confidence scores indicating confidence that the input sequence comprise malicious task instructions for the language model, wherein the machine learning model was trained on feature vectors of sentences comprising known malicious or benign task instructions (Jackson: [0034]: pre-filtering instructions include using direct string matching, natural language processing, machine learning, or other anomaly detection to identify characteristics common to known malicious prompts, to evaluate whether prompt is safe or malicious); and
determining whether to allow the input sequence to be passed to the language model or block the input sequence based, at least in part, on the confidence scores (Jackson: [0077]-[0079]: reject if prompt content is not safe, or allow if prompt is safe).
Jackson does not explicitly disclose analyzing sentences in a prompt to determine prompt injection attack. However, Vaknin discloses detection of prompt injection attacks against LLM based on sentence incoherence of the prompt using feature vector values (Vaknin: [0060]-[0062]; [0075]). It would have been obvious to one having ordinary skill in the art to analyze sentences in a prompt that is suspicious or malicious based on trained data because Vaknin and Jackson are analogous art involving detection of prompt injection attack. The motivation to combine would be to identify specific anomaly associated with prompt.
Jackson does not explicitly disclose wherein generating the feature vector comprises tokenizing the sentence and generating a natural language processing embedding of the tokenized sentence, wherein the natural language processing embedding comprises one of the feature vectors corresponding to the sentence, and output confidence scores that sentences comprise malicious task instructions. However, Rideout discloses generating feature vectors from LLM input by parsing sentence and transform into sentence embedding for natural language processing, and output confidence score to classify prompt as being benign or malicious (Rideout: col. 1 ll. 36-67: vectorize prompt to generate embeddings to determine whether prompt is benign or malicious; col. 9 ll. 6-18: after the prompt injection classifier has been trained, the analysis engine can preprocess incoming prompts…original prompt is transformed into sentence embeddings and then input into the prompt injection classifier to determine confidence score). It would have been obvious to one having ordinary skill in the art to generate confidence score based on feature vectors or tokenization of input prompt to determine prompt is benign or malicious because Jackson and Rideout both disclose detection of prompt injection attack on LLMs. The motivation to combine would be to classify and determine likelihood of malicious/unwanted behaviors based on context of the prompt input derived from sentence.
As per claim 2, Jackson as modified discloses the method of claim 1. Jackson as modified further discloses based on one or more of the confidence scores for the input sequence exceeding a threshold score, blocking the input sequence; and indicating one or more sentences in the input sequence corresponding to the one or more of the confidence scores as comprising malicious task instructions (Jackson: [0009]-[0011]: flagging and sending the prompt in response to determination that the prompt is not safe; Rideout: col. 7 ll. 1-13: analysis engine take remediation actions based on instruction and scores; Vaknin: [0038]-[0040]). It would have been obvious to one having ordinary skill in the art to correspond confidence score with input sentences as comprising malicious task instruction because they are analogous art. The motivation to combine would be to fine-tune the machine learning model that identifies and detect malicious prompts.
As per claim 3, Jackson as modified discloses the method of claim 2. Jackson as modified further discloses further comprising indicating a sentence of the one or more sentences with a highest confidence score in the confidence scores as a source of the malicious task instructions in the input sequence (Rideout: col. 9 ll. 6-18: confidence score to determine if prompt is benign or malicious). Same rationale applies here as above in rejecting claim 1.
As per claim 4, Jackson as modified discloses the method of claim 1. Jackson as modified further discloses wherein the potentially compromised data comprises data stored in a knowledge base for augmenting input sequences for the language model, wherein sources of the potentially compromised data are exposed to malicious attackers (Jackson: [0033]: inject malicious text into web pages and direct the model to interact with the compromised pages).
As per claim 5, Jackson as modified discloses the method of claim 4. Jackson as modified further discloses wherein the input sequence comprises an input sequence augmented by data stored in the knowledge base (Jackson: [0031]-[0033]).
As per claim 6, Jackson as modified discloses the method of claim 1. Jackson as modified further discloses wherein the malicious task instructions comprise task instructions to the language model to ignore a conversational history for the language model (Jackson: [0032]: “ignore previous instructions” and to reveal what is at the “beginning of the document above”).
As per claim 8, Jackson as modified discloses the method of claim 1. Jackson further discloses wherein the machine learning model comprises at least one of a Bidirectional Encoder Representations from Transformers model and a one- dimensional convolutional neural network (Jackson: [0023]: BERT model) .
As per claim 9 and 15, Jackson discloses a non-transitory machine-readable medium/apparatus comprising processor and a machine-readable medium having program code stored thereon, the program code comprising instructions to:
intercept input sequences comprising task instructions for a language model, where the input sequences comprise input sequences augmented with potentially compromised data (Jackson: [0032]-[0034]: indirect prompt injection attack using instructions comprising potentially compromised data; [0075]-[0076]: pass the received prompt to pre-filter);
filter, from the input sequences, input sequences comprising malicious task instructions, wherein the instructions to filter, from the input sequences, input sequences comprising malicious task instructions comprise instructions to, for each input sequence, generate feature vectors for the input sequence (Jackson: [0034]: pre-filtering instructions include using direct string matching, natural language processing, machine learning, or other anomaly detection to identify characteristics common to known malicious prompts, to evaluate whether prompt is safe or malicious);
determine whether the input sequence is malicious based on classifications by a machine learning model on the feature vectors (Jackson: [0023]-[0025]: embeddings are continuous vector representations of words or token; [0065]: process input data via sentence parsing, word segmentation, stemming and lemmatization and tokenization for phrases, sentences, paragraphs and passages; [0034]: techniques to evaluate prompt as being malicious); and
based on a determination that the input sequence is malicious, filter the input sequence (Jackson: [0007]-[0011]).
Jackson does not explicitly disclose analyzing sentences in a prompt to determine prompt injection attack. However, Vaknin discloses detection of prompt injection attacks against LLM based on sentence incoherence of the prompt using feature vector values (Vaknin: [0060]-[0062]; [0075]-[0076]). It would have been obvious to one having ordinary skill in the art to analyze sentences in a prompt that is suspicious or malicious based on trained data because Vaknin and Jackson are analogous art involving detection of prompt injection attack. The motivation to combine would be to identify specific anomaly associated with prompt.
Jackson does not explicitly disclose output confidence scores that sentences comprise malicious task instructions. However, Rideout discloses generating feature vectors from LLM input and output confidence score to classify prompt as being benign or malicious (Rideout: col. 1 ll. 36-67: vectorize prompt to determine whether prompt is benign or malicious; col. 8 l. 36 – col. 9 l. 4-18: prompt is transformed into sentence embeddings and then input prompt injection classifier to determine confidence score). It would have been obvious to one having ordinary skill in the art to generate confidence score based on feature vectors or tokenization of input prompt to determine prompt is benign or malicious because Jackson and Rideout both disclose detection of prompt injection attack on LLMs. The motivation to combine would be to classify and determine likelihood of malicious/unwanted behaviors based on trained data.
As per claim 10 and 16, Jackson as modified discloses the limitations of claims 9 and 15 respectively. Jackson as modified does not explicitly disclose wherein the instructions to, for each input sequence, determine whether the input sequence is malicious based on classifications by the machine learning model on the feature vectors comprise instructions to: input each of the feature vectors into the machine learning model to obtain confidence scores for corresponding sentences in the input sequence as output; and determine that the confidence scores satisfy a criterion for maliciousness. However, Rideout discloses generating feature vectors from LLM input and output confidence score to classify prompt as being benign or malicious (Rideout: col. 1 ll. 36-67: vectorize prompt to determine whether prompt is benign or malicious; col. 8 l. 36 – col. 9 l. 4-18: prompt is transformed into sentence embeddings and then input prompt injection classifier to determine confidence score). It would have been obvious to one having ordinary skill in the art to generate confidence score based on feature vectors or tokenization of input prompt to determine prompt is benign or malicious because Jackson and Rideout both disclose detection of prompt injection attack on LLMs. The motivation to combine would be to classify and determine likelihood of malicious/unwanted behaviors based on trained data.
As per claim 11 and 17, Jackson as modified discloses the limitations of claims 10 and 16 respectively. Jackson as modified further discloses wherein the criterion for maliciousness comprises that one or more of the confidence scores exceed a threshold confidence score (Jackson: [0009]-[0011]: flagging and sending the prompt in response to determination that the prompt is not safe; Rideout: col. 7 ll. 1-13: analysis engine take remediation actions based on instruction and scores; Vaknin: [0038]-[0040]). It would have been obvious to one having ordinary skill in the art to correspond confidence score with input sentences as comprising malicious task instruction because they are analogous art. The motivation to combine would be to fine-tune the machine learning model that identifies and detect malicious prompts.
As per claim 12 and 18, Jackson as modified discloses the limitations of claims 11 and 17 respectively. Jackson as modified further discloses wherein the program code further comprises instructions to generate an alert indicating one or more of the sentences in the input sequence corresponding to the one or more of the confidence scores as comprising malicious task instructions (Jackson: [0007]-[0011]).
As per claim 13 and 19, Jackson as modified discloses the limitations of claims 9 and 15 respectively. Jackson as modified further disclose wherein the program code further comprises instructions to, for each input sequence, based on a determination by the machine learning model that the input sequence is benign, communicate the input sequence to the language model (Jackson: [0006]-[0011]: allow prompt to be processed by LLM if determined to be safe).
Claims 21-23 are rejected under 35 U.S.C. 103 as being unpatentable over Jackson U.S. 2025/0103715 (hereinafter Jackson) in view of Vaknin et al. U.S. 2025/0209208 (hereinafter Vaknin) further in view of Rideout et al. U.S. 12,248,883 (hereinafter Rideout) and further in view of Mayande et al. U.S. 2025/0252320 (hereinafter Mayande).
Regarding claims 21-23, Jackson as modified discloses the limitations of claims 1, 9 and 15 respectively. Jackson as modified does not explicitly disclose wherein the instruction to generate the natural language processing embedding comprise instructions to apply sentence2vec to the tokenized sentence. However, Mayande discloses process generative AI input by tokenizing and convert into sentence embedding using sentence2vec (Mayande: [0043]). It would have been obvious to one having ordinary skill in the art to convert input prompt into various formats, including sentence embeddings, using sentence2vec because the references are in the same field of endeavor related to processing generative AI input. The motivation to combine would be that sentence2vec is well know in the art for converting data to determine semantic similarities.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHIN HON (ERIC) CHEN whose telephone number is (571)272-3789. The examiner can normally be reached Monday to Thursday 9am- 7pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Lynn Feild can be reached at 571-272-2092. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHIN-HON (ERIC) CHEN/Primary Examiner, Art Unit 2431