Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
The office action sent in response to Applicant’s communication received on 6/18/2024 for the application number 18746775. The office hereby acknowledges receipt of the following placed of record in the file: Specification, Abstract, Oath/Declaration and claims.
Status of the claims
Claims 1-20 are presented for examination.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 101
Claim 1 recites:
A system, comprising: a memory that stores computer executable components; and a processor, operably coupled to the memory, that executes the computer executable components stored in the memory, wherein the computer executable components comprise: an obtaining component that intercepts a semantic source from being submitted to a retrieval augmented generation (RAG) architecture; and a transforming component that transforms the semantic source into a transformed source by identifying and converting prompt-misleading text of the semantic source into prompt-non-misleading text.
Claim 1 can be broken to the following steps
an obtaining component that intercepts a semantic source from being submitted to a retrieval augmented generation (RAG) architecture
and a transforming component that transforms the semantic source into a transformed source by identifying and converting prompt-misleading text of the semantic source into prompt-non-misleading text.
Step a can be performed by the human mind as a person can look at the prompt/input which is written and intercept it
Step b can be performed as human can take a look at the prompt decide whether the prompt is malicious or not.
Step 1: This part of the eligibility analysis evaluates whether the claim falls within any statutory category. See MPEP 2106.03. The claim recites a system, hence a machine , which is one of the statutory categories of invention. (Step 1: YES).
Step 2A, Prong One: This part of the eligibility analysis evaluates whether the claim recites a judicial exception. As explained in MPEP 2106.04, subsection II, a claim “recites” a judicial exception when the judicial exception is “set forth” or “described” in the claim. As discussed above, the broadest reasonable interpretation of steps a) and b) recites mental process for e.g. for e.g. human can take a look at the input received and decides whether the input is misleading and change it to make it non misleading , hence the claim encompasses mental processes practically performed in the human mind by observation, evaluation, judgment, and opinion. See MPEP 2106.04(a)(2), subsection III. (Step 2A, Prong One: YES).
Step 2A, Prong Two: This part of the eligibility analysis evaluates whether the claim as a whole integrates the recited judicial exception into a practical application of the exception or whether the claim is “directed to” the judicial exception. This evaluation is performed by (1) identifying whether there are any additional elements recited in the claim beyond the judicial exception, and (2) evaluating those additional elements individually and in combination to determine whether the claim as a whole integrates the exception into a practical application. See MPEP 2106.04(d). The claim recites RAG architecture, A system, comprising: a memory that stores computer executable components; and a processor, operably coupled to the memory, that executes the computer executable components stored in the memory, obtaining and transforming component. However, RAG architecture is a mere computer component which is not performing any function. The other computer components are generic component performing a function of human mind for e.g. obtaining and transforming and hence these mere instructions to apply the exception using a generic computer component. The other additional element like processor, memory and/or non-transitory computer readable medium is recited at a high level of generality and these are mere generic computer component. Accordingly, this additional element does not integrate the abstract idea of incepting an input and changing it into a practical application because it does not impose any meaningful limits on practicing the abstract idea.(Step 2A, Prong Two: NO), and the claim is directed to the judicial exception. (Step 2A: YES).
Step 2B. As discussed with respect to Step 2A Prong Two, the additional element in the claim amounts to no more than mere instructions to apply the exception using a generic computer component. The same analysis applies here in 2B, i.e., mere instructions to apply an exception using a generic computer component cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B. The claim is ineligible.
Regarding claims 11 and 16 , analysis analogous to claim 1, are applicable.
Regarding claim 2 and 3, it cites the RAG architecture where the prompt would be submitted however as described above RAG architecture is itself is a known computer component, same analysis as in claim 1, under prong 2, step and prong 2b are applicable.
Regarding claim 4 and 5, the claims describes how the prompt is being received, these are the mere data gathering activity and hence under step 2a, prong 2 and step 2b, does not integrate the abstract idea to a practical applicable.
Regarding claim 6-10, the claims merely recites the computer component performing the function of human mind.
Regarding claim 12-15 and 17-20, analysis analogous to claims 2-10, are applicable.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
And
KSR, 550 U.S. at 418, 82 USPQ2d at 1396. Exemplary rationales that may support a conclusion of obviousness include:
(A) Combining prior art elements according to known methods to yield predictable results;
(B) Simple substitution of one known element for another to obtain predictable results;
(C) Use of known technique to improve similar devices (methods, or products) in the same way;
(D) Applying a known technique to a known device (method, or product) ready for improvement to yield predictable results;
(E) "Obvious to try" – choosing from a finite number of identified, predictable solutions, with a reasonable expectation of success;
(F) Known work in one field of endeavor may prompt variations of it for use in either the same field or a different one based on design incentives or other market forces if the variations are predictable to one of ordinary skill in the art;
(G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention.
See MPEP § 2143 for a discussion of the rationales listed above along with examples illustrating how the cited rationales may be used to support a finding of obviousness. See also MPEP § 2144 - § 2144.09 for additional guidance regarding support for obviousness determination.
Claims 1-2, 4, 8, 11-13 and 16-18 are rejected under 35 U.S.C. 103 as being unpatentable over Cappel ( US 11995180 ) in view of Jain (US 12106205 )
Regarding claim 1, Crabtree teaches a system, comprising: a memory that stores computer executable components; and a processor, operably coupled to the memory, that executes the computer executable components stored in the memory, wherein the computer executable components comprise: an obtaining component that intercepts a semantic source from being submitted to a retrieval augmented generation (RAG) architecture ( The proxy 150 can, in some variations, relay received queries to the monitoring environment 160 prior to ingestion by the MLA 130, Col 5, line 50-55) ; and a transforming component that transforms ( remediation engine, Fig 2-4) the semantic source into a transformed source by identifying and converting prompt-misleading text of the semantic source into prompt-non-misleading text ( The analysis engine 170 can analyze the relayed queries and/or information in order to make an assessment or other determination as to whether the queries are indicative of being malicious. In some cases, a remediation engine 180 which can form part of the monitoring environment 160 (or be external such as illustrated in FIG. 2-4) can take one or more remediation actions in response to a determination of a query as being malicious…..In some cases, the remediation engine 180 can cause data to be transmitted to the proxy 150 which causes the query to be modified in order to be non-malicious, to remove sensitive information, and the like. Such queries, after modification, can be ingested by the MLA 130 and the output provided to the requesting client device 110. Alternatively, the output of the MLA 130 (after query modification) can be subject to further analysis by the analysis engine 170, Col 5, line 60-67, Col 6, line 1-10)
Cappel does not explicitly teach a model is a retrieval augmented generation model
However, Jain teaches incepting prompt to a retrieval augmented generation model ( fig 6- prompt/input validation interception before being inputted to the LLM, wherein the LLM the model (e.g., LLM) includes augmented or modified LLMs, such as retrieval-augmented generation (RAG) algorithms. A RAG algorithm can include a document retriever (and/or another type of retriever). For example, the retriever can determine, based on the prompt, one or more relevant documents (or other suitable text records, as in a textual database). For example, the document retriever can encode the query within a vector space, as well as the documents, and determine relevant documents based on their distance from the query within the vector space. The LLM can generate an output based on the query and the retrieved documents, thereby improving the accuracy and relevance of the generated outputs., Col 19, line 60-67; Col 20, line 1-10; sanitizing prompt, Col 45, line 45-60)
It would have been obvious having the teachings of Cappel to further include the concept of Jain before effective filing date of the claimed invention because by doing so, the data generation platform enables modular, flexible, and configurable prompt evaluation in an automated manner. Based on the results of the prompt validation model, the data generation platform can modify the prompt such that the prompt satisfies any associated validation criteria (e.g., through the redaction of sensitive data or other details) thereby mitigating the effect of potential security breaches, inaccuracies, or adversarial manipulation associated with the user's prompt ( Col 4, line 1-10, Jain)
Regarding claim 2, Cappel modified by Jain as above in claim 1, wherein the semantic source is a semantic query having been submitted to the RAG architecture ( generative model, Fig 8, Cappel; RAG, Col 19, line 59-67, Jain)
Regarding claim 4, Cappel above in claim 1, teaches wherein the prompt-misleading text originated in connection with an origination of the semantic source ( receiving a prompt for ingestion model, Fig 11, Cappel; users prompt, Col 4, line 5-10, Jain)
Regarding claim 8, Jain as above in claim 1, teaches a directing component that directs use of the transformed source as an input to the RAG architecture ( RAG) ( input to the LLM ( where LLM is built on RAG) , Fig 4)
Regarding claim 11, arguments analogous to claim 1 are applicable.
Regarding claim 12, arguments analogous to claim 2, are applicable.
Regarding claim 13, arguments analogous to claim 4, are applicable.
Regarding claim 16, arguments analogous to claim 1 are applicable.
Regarding claim 17, arguments analogous to claim 2, are applicable.
Regarding claim 18, arguments analogous to claim 4, are applicable.
Claims 3, 5, 7, 14 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Cappel ( US 11995180 ) in view of Jain (US 12106205 ) and further in view of Crabtree ( US 20250259075)
Regarding claim 3, Jain as above in claim 1, wherein the semantic source is a retrieved source having been retrieved by the RAG architecture in a process of providing a prompt
However, Crabtree teaches wherein the semantic source is a retrieved source having been retrieved by the RAG architecture in a process of providing a prompt (The platform may use retrieval augmented generation (RAG) to retrieve relevant information from external knowledge sources and condition the model's prompts and/or output on retrieved data and/or facts, Para 0070)
It would have been obvious having the concept of Cappel and Jain to further include the concept of Crabtree before effective filing date to ensures that the generated text is more grounded in reality and less likely to contain hallucinated information ( Para 0070, Crabtree)
Regarding claim 5, Cappel modified by Jain as above in claim 1, does not explicitly teach wherein the prompt-misleading text was caused by an adversarial attack corresponding to the semantic source
However, Crabtree teaches wherein the prompt-misleading text was caused by an adversarial attack corresponding to the semantic source ( adversarial model for misleading text, Fig 33-35; he platform can generate random prompts to identify problematic outputs and maintain session state for users or groups to develop a “user/group RAG.” Model training sandbox system 174 may use techniques like adversarial training and input perturbation to test the robustness of models against poisoning attacks and other adversarial examples, Para 0111)
It would have been obvious having the teachings of Cappel and Jain to further include the concept of Crabtree before effective filing date to make the model more robust ( Para 0111, Crabtree)
Regarding claim 7, Cappel modified by Jain as above in claim 1, does not explicitly teach , further comprising: a training component that submits the transformed source to a language model to be tuned and to a known adversarial attack code and tunes the language model based on an output of the adversarial attack code
However, Crabtree teaches a training component that submits the transformed source to a language model to be tuned and to a known adversarial attack code and tunes the language model based on an output of the adversarial attack code ( adversarial model for misleading text, Fig 33-35; he platform can generate random prompts to identify problematic outputs and maintain session state for users or groups to develop a “user/group RAG.” Model training sandbox system 174 may use techniques like adversarial training and input perturbation to test the robustness of models against poisoning attacks and other adversarial examples, Para 0111)
It would have been obvious having the teachings of Cappel and Jain to further include the concept of Crabtree before effective filing date to make the model more robust ( Para 0111, Crabtree)
Regarding claim 14, arguments analogous to claim 7, are applicable.
Regarding claim 19, arguments analogous to claim 7, are applicable.
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Cappel ( US 11995180 ) view of Jain (US 12106205 ) and further in view of Varerkar (US 20250342360)
Regarding claim 6, Cappel modified by Jain as above in claim 1, teaches an evaluating component that analyzes the transformed source based on user entity feedback ( the output of the MLA 130 (after query modification) can be subject to further analysis by the analysis engine 170, Col 6, line 10-15;Fig 13’ feedback for the LLM responses, Jain)
Cappel modified Jain does not explicitly teach analyzes the transformed source using ground truth, recall-oriented understudy for gisting evaluation (ROUGE) scoring, or user entity feedback
However, Varerkar teaches analyzes the transformed source using ground truth, recall-oriented understudy for gisting evaluation (ROUGE) scoring, ( analysis the input and output prompts, Table 2 and 3 using rouge and ground truth)
It would have been obvious having the teachings of Cappel and Jain to further include the concept of Varerkar before effective filing date so to judge the performance of the model
Claims 9, 15 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Cappel ( US 11995180 ) in view of Jain (US 12106205 ) and further in view of Salim ( US 20250278574 )
Regarding claim 9, Jain as above in claim 1, teaches an iterating component that directs the transforming component to perform one or more additional iterations of transforming of the same semantic source ( prompt augmentation, Fig 6, Col 16, line 60-67) ; and a directing component that directs use of a set of transformed sources resulting therefrom as different inputs of execution of the RAG architecture ( LLM generated response is flexible, Col 60-67)
Although Jain mention generated response is flexible, it does not explicitly teach transformed sources resulting therefrom as different inputs to different instances of execution of the RAG architecture
However transformed sources resulting therefrom as different inputs to different instances of execution of the RAG architecture ( The system can execute a large language model using each of the modified queries to generate a response for each of the queries. The system can compare each of the modified queries (e.g., compare text content, parts of speech) to identify one or more queries that include content that is substantially the same or that is substantially different, according, for example to a percentage of common characters, text, tokenized text, list elements, or any combination thereof, Para 0022)
It would have been obvious having the teachings of Cappel and Jain to further include the concept of Salim before effective filing date since it can provide computational efficiency in identifying responses with veracity, by imputing veracity to convergent responses that can be generated at a rate and at a level of verifiability beyond the capability of manual processes ( Para 0022, Salim)
Regarding claim 10, Salim as above in claim 9, teaches a reporting component that generates a report comparing different outputs of the different instances of executions of the RAG architecture ( The system can generate a convergence metric indicative of the similarity between any of the responses, and can compare the convergence metric to a convergence threshold (e.g., the convergence threshold can be indicative of 80%, 90% or 95% same content), Para 0022)
Regarding claim 15, arguments analogous to claim 10, are applicable.
Regarding claim 20, arguments analogous to claim 10, are applicable.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
BadRAG: Identifying Vulnerabilities in Retrieval Augmented Generation of Large Language Models teaches ( Our defense exploits the strong, unique link between trigger words and the adversarial passage: removing the trigger from the query prevents retrieval of the adversarial passage, while a clean query considers overall semantic similarity, Fig 3- modifying the query before being used to retrieve from corpus
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Richa Sonifrank whose telephone number is (571)272-5357. The examiner can normally be reached M-T 7AM - 5:30PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Phan Hai can be reached at (571)272-6338. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Richa Sonifrank/Primary Examiner, Art Unit 2654