DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
In response to communications filed on 19 March 2026, claims 1-27 are presently pending in the application, of which, claims 1, 10, and 19 are presented in independent form. The Examiner acknowledges amended claims 1, 10, and 19. No claims were newly added or cancelled.
Priority
The Examiner acknowledges the instant application claims priority to U.S. Provisional 63/624,416, filed on 23 January 2025, and has been accorded the earliest effective file date.
Response to Remarks/Arguments
All objections and/or rejections issued in the previous Office Action, mailed 23 September 2025, have been withdrawn, unless otherwise noted in this Office Action.
Applicant’s arguments, see page 9, filed 19 March 2026, with respect to the rejections of claims 1-27 under 35 U.S.C. 102 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new grounds of rejection is made in view of Belgi, Amir, et al (U.S. 2025/0156567, filed 10 November 2023, and known hereinafter as Belgi) in view of a non-patent literature titled “A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions,” by Huang, Lei, et al, ACM Transactions on Information Systems, Vol 1, Issue 1, Article 1, January 2024 (known hereinafter as Huang).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 6-13, 15-22, and 24-27 are rejected under 35 U.S.C. 103 as being unpatentable by Belgi, Amir, et al (U.S. 2025/0156567, filed 10 November 2023, and known hereinafter as Belgi)(newly presented) in view of a non-patent literature titled “A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions,” by Huang, Lei, et al, ACM Transactions on Information Systems, Vol 1, Issue 1, Article 1, January 2024 (known hereinafter as Huang)(newly presented).
As per claim 1, Belgi teaches a computer-implemented method comprising:
accessing text data, wherein the text data includes one or more machine-generated responses that generated using generic machine-learning model in response to a prompt (e.g. Belgi, see paragraphs [0053-0054], which discloses the description may be generated using the video file as input together with additional data from the video file itself, where the additional data may be termed a prompt, where prompts may include labelled speech segments which are generated, any text data which has been extracted, or any metadata which has been extracted from the video file itself.), and wherein the one or more machine-generated responses are supplemented by outputs generated by a retrieval-augmentation generation (RAG) system (e.g. Belgi, see paragraphs [0054-0055], which discloses the prompts may also be selected from examples in the description database which may include training data used to train the model used by the description module, where the prompts may be selected using any suitable method such as a retrieval augmentation generation, which typically augments (e.g. supplements) the prompts with additional relevant context.);
applying one or more hallucination-detection models to the text data to generate a set of classification labels (e.g. Belgi, see paragraphs [0055-0058], which discloses when vectors are based on tokenized pieces of text, the generated text-based summary may be tokenized in a similar manner and RAG may be used with the relevant vectors based on tokenized text to prevent hallucinations. See further paragraphs [0072-0075]), wherein a classification label indicates whether a corresponding machine-generated response contradicts at least part of a knowledge base accessed by the RAG system (e.g. Belgi, see paragraphs [0055-0058], which discloses the prompts may be selected using RAG for example by checking the cosine similarity of the current text scription to text descriptions in the security database, where the text descriptions are both expressed in vectors, which then allow for the classification of the data as a second fine-tuned generative AI or LLM classifier. Additionally, see further paragraphs [0080-0082], which discloses once the model has been trained, the model can then be used to create a classification label for new input video files.), and wherein the one or more hallucination-detection models were trained using a training dataset that includes previous machine-generated responses annotated with the set of classification labels (e.g. Belgi, see paragraphs [0079-0086], which discloses the LLM classifier is then trained using the training data and then once the model has been trained, the model can then be used to create a classification, where for example newly received text descriptions is processed using the trained model together with generated prompts (e.g. RAG system), where the classification label and optionally reasons for the classifications are output. The output classification label together with the input text description for which the classification was generated may be stored in the security database.);
generating annotated text data that includes the one or more machine-generated responses annotated with corresponding classification labels of the set of classification labels (e.g. Belgi, see paragraphs [0074], which discloses for training LLM classifier, the input of each example is primarily the text description which is generated. The input text description is annotated (e.g. labelled) with an output in the form of a classification label. The input may also be annotated with an output which gives reasons for the classification labels which has been assigned. These annotations may be considered to be extensions of the annotations in the training dataset used in training the description module.).
Although Belgi discloses processing the annotated text data, it does not explicitly disclose processing the annotated text data to identify one or more hallucinations in the outputs generated by the RAG system, thereby improving performance of the generative machine-learning model in generating factually accurate responses.
Huang teaches processing the annotated text data to identify one or more hallucinations in the outputs generated by the RAG system, thereby improving performance of the generative machine-learning model in generating factually accurate responses (e.g. Huang, see section 4, pages 12-13, which discloses factuality hallucination detection which involves assessing whether the output LLM aligns with real-world facts, where the factuality of the generated response is verified against trusted knowledge sources, where the fact is extracted and then verified.).
Belgi is directed to classifying and controlling transmission of a file. Huang is directed to mitigating LLM hallucinations. Both are analogous art because they are directed to validating data based on generative artificial intelligence data and therefore it would have been obvious to one of ordinary skilled in the art at the time the invention was filed to modify the teachings of Belgi with the teachings of Huang to include the claimed features with the motivation to improve classification of factually correct data.
As per claim 10, Al-Qurishi teaches a system comprising:
one or more processors (e.g. Al-Qurishi, see paragraphs [0210-0212], which discloses one or more processors coupled to memory.); and
memory storing thereon instructions that, as a result of being executed by the one or more processors (e.g. Al-Qurishi, see paragraphs [0210-0212], which discloses one or more processors coupled to memory.), cause the system to perform operations comprising:
accessing text data, wherein the text data includes one or more machine-generated responses that generated using generic machine-learning model in response to a prompt (e.g. Belgi, see paragraphs [0053-0054], which discloses the description may be generated using the video file as input together with additional data from the video file itself, where the additional data may be termed a prompt, where prompts may include labelled speech segments which are generated, any text data which has been extracted, or any metadata which has been extracted from the video file itself.), and wherein the one or more machine-generated responses are supplemented by outputs generated by a retrieval-augmentation generation (RAG) system (e.g. Belgi, see paragraphs [0054-0055], which discloses the prompts may also be selected from examples in the description database which may include training data used to train the model used by the description module, where the prompts may be selected using any suitable method such as a retrieval augmentation generation, which typically augments (e.g. supplements) the prompts with additional relevant context.);
applying one or more hallucination-detection models to the text data to generate a set of classification labels (e.g. Belgi, see paragraphs [0055-0058], which discloses when vectors are based on tokenized pieces of text, the generated text-based summary may be tokenized in a similar manner and RAG may be used with the relevant vectors based on tokenized text to prevent hallucinations. See further paragraphs [0072-0075]), wherein a classification label indicates whether a corresponding machine-generated response contradicts at least part of a knowledge base accessed by the RAG system (e.g. Belgi, see paragraphs [0055-0058], which discloses the prompts may be selected using RAG for example by checking the cosine similarity of the current text scription to text descriptions in the security database, where the text descriptions are both expressed in vectors, which then allow for the classification of the data as a second fine-tuned generative AI or LLM classifier. Additionally, see further paragraphs [0080-0082], which discloses once the model has been trained, the model can then be used to create a classification label for new input video files.), and wherein the one or more hallucination-detection models were trained using a training dataset that includes previous machine-generated responses annotated with the set of classification labels (e.g. Belgi, see paragraphs [0079-0086], which discloses the LLM classifier is then trained using the training data and then once the model has been trained, the model can then be used to create a classification, where for example newly received text descriptions is processed using the trained model together with generated prompts (e.g. RAG system), where the classification label and optionally reasons for the classifications are output. The output classification label together with the input text description for which the classification was generated may be stored in the security database.);
generating annotated text data that includes the one or more machine-generated responses annotated with corresponding classification labels of the set of classification labels (e.g. Belgi, see paragraphs [0074], which discloses for training LLM classifier, the input of each example is primarily the text description which is generated. The input text description is annotated (e.g. labelled) with an output in the form of a classification label. The input may also be annotated with an output which gives reasons for the classification labels which has been assigned. These annotations may be considered to be extensions of the annotations in the training dataset used in training the description module.).
Although Belgi discloses processing the annotated text data, it does not explicitly disclose processing the annotated text data to identify one or more hallucinations in the outputs generated by the RAG system, thereby improving performance of the generative machine-learning model in generating factually accurate responses.
Huang teaches processing the annotated text data to identify one or more hallucinations in the outputs generated by the RAG system, thereby improving performance of the generative machine-learning model in generating factually accurate responses (e.g. Huang, see section 4, pages 12-13, which discloses factuality hallucination detection which involves assessing whether the output LLM aligns with real-world facts, where the factuality of the generated response is verified against trusted knowledge sources, where the fact is extracted and then verified.).
Belgi is directed to classifying and controlling transmission of a file. Huang is directed to mitigating LLM hallucinations. Both are analogous art because they are directed to validating data based on generative artificial intelligence data and therefore it would have been obvious to one of ordinary skilled in the art at the time the invention was filed to modify the teachings of Belgi with the teachings of Huang to include the claimed features with the motivation to improve classification of factually correct data.
As per claim 19, Al-Qurishi teaches a non-transitory, computer-readable storage medium storing thereon executable instructions that, as a result of being executed by one or more processors of a computer system, cause the computer system to perform operations comprising:
accessing text data, wherein the text data includes one or more machine-generated responses that generated using generic machine-learning model in response to a prompt (e.g. Belgi, see paragraphs [0053-0054], which discloses the description may be generated using the video file as input together with additional data from the video file itself, where the additional data may be termed a prompt, where prompts may include labelled speech segments which are generated, any text data which has been extracted, or any metadata which has been extracted from the video file itself.), and wherein the one or more machine-generated responses are supplemented by outputs generated by a retrieval-augmentation generation (RAG) system (e.g. Belgi, see paragraphs [0054-0055], which discloses the prompts may also be selected from examples in the description database which may include training data used to train the model used by the description module, where the prompts may be selected using any suitable method such as a retrieval augmentation generation, which typically augments (e.g. supplements) the prompts with additional relevant context.);
applying one or more hallucination-detection models to the text data to generate a set of classification labels (e.g. Belgi, see paragraphs [0055-0058], which discloses when vectors are based on tokenized pieces of text, the generated text-based summary may be tokenized in a similar manner and RAG may be used with the relevant vectors based on tokenized text to prevent hallucinations. See further paragraphs [0072-0075]), wherein a classification label indicates whether a corresponding machine-generated response contradicts at least part of a knowledge base accessed by the RAG system (e.g. Belgi, see paragraphs [0055-0058], which discloses the prompts may be selected using RAG for example by checking the cosine similarity of the current text scription to text descriptions in the security database, where the text descriptions are both expressed in vectors, which then allow for the classification of the data as a second fine-tuned generative AI or LLM classifier. Additionally, see further paragraphs [0080-0082], which discloses once the model has been trained, the model can then be used to create a classification label for new input video files.), and wherein the one or more hallucination-detection models were trained using a training dataset that includes previous machine-generated responses annotated with the set of classification labels (e.g. Belgi, see paragraphs [0079-0086], which discloses the LLM classifier is then trained using the training data and then once the model has been trained, the model can then be used to create a classification, where for example newly received text descriptions is processed using the trained model together with generated prompts (e.g. RAG system), where the classification label and optionally reasons for the classifications are output. The output classification label together with the input text description for which the classification was generated may be stored in the security database.);
generating annotated text data that includes the one or more machine-generated responses annotated with corresponding classification labels of the set of classification labels (e.g. Belgi, see paragraphs [0074], which discloses for training LLM classifier, the input of each example is primarily the text description which is generated. The input text description is annotated (e.g. labelled) with an output in the form of a classification label. The input may also be annotated with an output which gives reasons for the classification labels which has been assigned. These annotations may be considered to be extensions of the annotations in the training dataset used in training the description module.).
Although Belgi discloses processing the annotated text data, it does not explicitly disclose processing the annotated text data to identify one or more hallucinations in the outputs generated by the RAG system, thereby improving performance of the generative machine-learning model in generating factually accurate responses.
Huang teaches processing the annotated text data to identify one or more hallucinations in the outputs generated by the RAG system, thereby improving performance of the generative machine-learning model in generating factually accurate responses (e.g. Huang, see section 4, pages 12-13, which discloses factuality hallucination detection which involves assessing whether the output LLM aligns with real-world facts, where the factuality of the generated response is verified against trusted knowledge sources, where the fact is extracted and then verified.).
Belgi is directed to classifying and controlling transmission of a file. Huang is directed to mitigating LLM hallucinations. Both are analogous art because they are directed to validating data based on generative artificial intelligence data and therefore it would have been obvious to one of ordinary skilled in the art at the time the invention was filed to modify the teachings of Belgi with the teachings of Huang to include the claimed features with the motivation to improve classification of factually correct data.
As per claims 2, 11, and 20, the modified teachings of Belgi and Huang teaches the computer-implemented method of claim 1, the system of claim 10, and the non-transitory computer-readable storage medium of claim 19, respectively, wherein generating the outputs includes:
encoding the prompt into one or more embeddings, wherein when the one or more embeddings are entered in a database, the RAG system uses prompt results outputted from the database to supplement the one or more machine-generated responses (e.g. Belgi, see paragraphs [0055-0058], which discloses the prompts may be selected using RAG for example by checking the cosine similarity of the current text scription to text descriptions in the security database, where the text descriptions are both expressed in vectors, which then allow for the classification of the data as a second fine-tuned generative AI or LLM classifier. Additionally, see further paragraphs [0080-0082], which discloses once the model has been trained, the model can then be used to create a classification label for new input video files.).
As per claims 3, 12, and 21, the modified teachings of Belgi and Huang teaches the computer-implemented method of claim 1, the system of claim 10, and the non-transitory computer-readable storage medium of claim 19, respectively, wherein the knowledge base includes domain-specific information, wherein the domain-specific information is associated with a particular domain (e.g. Belgi, see paragraphs [0053-0054], which discloses the description may be generated using the video file as input together with additional data from the video file itself, where the additional data may be termed a prompt, where prompts may include labelled speech segments which are generated, any text data which has been extracted, or any metadata which has been extracted from the video file itself.).
As per claims 4, 13, and 22, the modified teachings of Belgi and Huang teaches the computer-implemented method of claim 1, the system of claim 10, and the non-transitory computer-readable storage medium of claim 19, respectively, wherein the set of classification labels includes a no-info classification label indicating that the corresponding machine-generated response includes non-verifiable information, a supported classification label indicating that the corresponding machine-generated response is supported by the knowledge base, and an unsupported classification label indicating that the corresponding machine- generated response contradicts the at least part of the knowledge base (e.g. Belgi, see paragraphs [0074], which discloses for training LLM classifier, the input of each example is primarily the text description which is generated. The input text description is annotated (e.g. labelled) with an output in the form of a classification label. The input may also be annotated with an output which gives reasons for the classification labels which has been assigned. These annotations may be considered to be extensions of the annotations in the training dataset used in training the description module.).
As per claims 6, 15, and 24, the modified teachings of Belgi and Huang teaches the computer-implemented method of claim 1, the system of claim 10, and the non-transitory computer-readable storage medium of claim 19, respectively, wherein the one or more hallucination-detection models include a pretrained large-language model (LLM) (e.g. Belgi, see paragraph [0055], which discloses pre-trained LLM classifier.).
As per claims 7, 16, and 25, the modified teachings of Belgi and Huang teaches the computer-implemented method of claim 1, the system of claim 10, and the non-transitory computer-readable storage medium of claim 19, respectively, wherein applying the one or more hallucination-detection models to the text data includes:
applying a first hallucination-detection model of the one or more hallucination- detection models to the text data to generate a verifiable classification label indicating that the corresponding machine-generated response includes information verifiable from the knowledge base (e.g. Belgi, see paragraphs [0055-0058], which discloses when vectors are based on tokenized pieces of text, the generated text-based summary may be tokenized in a similar manner and RAG may be used with the relevant vectors based on tokenized text to prevent hallucinations. See further paragraphs [0072-0075]); and
applying a second hallucination-detection model of the one or more hallucination- detection models to the corresponding machine-generated response to generate the classification label indicating whether the corresponding machine-generated response contradicts at least part of the knowledge base (e.g. Belgi, see paragraphs [0055-0058], which discloses when vectors are based on tokenized pieces of text, the generated text-based summary may be tokenized in a similar manner and RAG may be used with the relevant vectors based on tokenized text to prevent hallucinations. See further paragraphs [0072-0075]).
As per claims 8, 17, and 26, the modified teachings of Belgi and Huang teaches the computer-implemented method of claim 1, the system of claim 10, and the non-transitory computer-readable storage medium of claim 19, respectively, wherein applying the one or more hallucination-detection models to the text data includes:
applying a first hallucination-detection model of the one or more hallucination- detection models to the text data to generate a no-info classification label indicating that the corresponding machine-generated response includes non-verifiable information (e.g. Belgi, see paragraphs [0055-0058], which discloses when vectors are based on tokenized pieces of text, the generated text-based summary may be tokenized in a similar manner and RAG may be used with the relevant vectors based on tokenized text to prevent hallucinations. See further paragraphs [0072-0075])).
As per claims 9, 18, and 27, the modified teachings of Belgi and Huang teaches the computer-implemented method of claim 1, the system of claim 10, and the non-transitory computer-readable storage medium of claim 19, respectively, wherein outputting the annotated text data includes displaying in real-time the annotated text data on a graphical user interface (e.g. Belgi, see paragraphs [0074], which discloses for training LLM classifier, the input of each example is primarily the text description which is generated. The input text description is annotated (e.g. labelled) with an output in the form of a classification label. The input may also be annotated with an output which gives reasons for the classification labels which has been assigned. These annotations may be considered to be extensions of the annotations in the training dataset used in training the description module.), as messages are exchanged between the user and an agent during an instant-chat session (e.g. Belgi, see paragraphs [0074], which discloses for training LLM classifier, the input of each example is primarily the text description which is generated. The input text description is annotated (e.g. labelled) with an output in the form of a classification label. The input may also be annotated with an output which gives reasons for the classification labels which has been assigned. These annotations may be considered to be extensions of the annotations in the training dataset used in training the description module.).
Claims 5, 14, and 23 are rejected under 35 U.S.C. 103 as being unpatentable by Belgi, Amir, et al (U.S. 2025/0156567, filed 10 November 2023, and known hereinafter as Belgi)(newly presented) in view of a non-patent literature titled “A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions,” by Huang, Lei, et al, ACM Transactions on Information Systems, Vol 1, Issue 1, Article 1, January 2024 (known hereinafter as Huang)(newly presented) and in further view of Al-Qurishi, Muhammed Saleh Saeed et al (U.S. 2024/0330993, filed 17 October 2023, claiming priority to provisional application no 63/455,445, filed 29 March 2023, and known hereinafter as Al-Qurishi).
As per claims 5, 14, and 23, the modified teachings of Belgi and Huang teaches the computer-implemented method of claim 1, the system of claim 10, and the non-transitory computer-readable storage medium of claim 19, respectively, however it does not explicitly disclose wherein the one or more hallucination-detection models include a pretrained Decoding-enhanced Bidirectional Encoder Representations from Transformers with Disentangled attention (DeBERTa) model.
Al-Qurishi teaches wherein the one or more hallucination-detection models include a pretrained Decoding-enhanced Bidirectional Encoder Representations from Transformers with Disentangled attention (DeBERTa) model (e.g. Al-Qurishi, see paragraphs [0117-0119], which discloses the system utilizes a shared LLM backbone which can be DEBERTA etc with task-specific layers to learn and generalize from this large dataset effectively.).
Belgi is directed to classifying and controlling transmission of a file. Huang is directed to mitigating LLM hallucinations. Al-Qurishi is directed to realtime measuring of product reputation. All are analogous art because they are directed to validating data based on generative artificial intelligence data and therefore it would have been obvious to one of ordinary skilled in the art at the time the invention was filed to modify the teachings of Belgi with the teachings of Huang and with the further teachings of Al-Qurishi to include the claimed features with the motivation to improve classification of factually correct data.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure. See attached PTO-892 that includes additional prior art of record describing the general state of the art in which the invention is directed to.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to FARHAN M SYED whose telephone number is (571)272-7191. The examiner can normally be reached M-F 8:30AM-5:30PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Apu Mofiz can be reached at 571-272-4080. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/FARHAN M SYED/Primary Examiner, Art Unit 2161 April 23, 2026
Prosecution Insights