CTNF 18/535,069 CTNF 78157 DETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Claim Rejections - 35 USC § 103 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-21-aia AIA Claim s 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Perez et al. (US 2023/0237826 A1), hereinafter “ Perez ”, and in view of Horesh et al. (US 11,972,333 B1), hereinafter “ Horesh ” . As per claim 1 , Perez teaches a computing platform comprising: at least one processor; a communication interface communicatively coupled to the at least one processor; and memory storing computer-readable instructions that, when executed by the at least one processor, cause the computing platform to: “generate, using a test case generation model, a plurality of large language model (LLM) test cases” at [0031]-[0032], [0037] and Fig.1; (Perez teaches the pre-deployment evaluation system 100 uses the test case generation neural network 110 (i.e., “test case generation model”) to automatically generate multiple test inputs 112 (i.e., “test cases”) which are subsequently processed by the target neural network 120) “input, into an LLM, the plurality of LLM test cases, wherein inputting the plurality of LLM test cases into the LLM produces a plurality of unverified LLM test results” at [0033], [0040]; (Perez teaches the generated test inputs 112 is inputted to the target neural network 120 (i.e., “LLM”) in accordance with the target network parameters to generate test output 122 (i.e., “test results”)) “compare, using a falsified output evaluation model, the plurality of unverified LLM test results with the corresponding plurality of validated LLM test results, wherein the comparison produces an LLM compliance score for the LLM; compare the LLM compliance score to a compliance threshold” at [0032]-[0033], [0040]-[0042]; (Perez teaches the pre-deployment evaluation system 100 evaluates these test outputs 122 by evaluating them against one or more criteria 131 (i.e., “validated LLM test results”) to determine whether the target neural network 120 is suitable for deployment in the production environment 140. If the target neural network 120 fails one or more of the criteria during pre-deployment evaluation, then the pre-deployment evaluation system 100 can adjust the network accordingly to improve its suitability with respect to the one or more criteria; if the target neural network 120 passes all the criteria , then the pre-deployment evaluation system 100 can provide the data specifying the target neural network 120 to the production environment 140 in order to allow the production environment 140 to deploy the target neural network 120 for use to perform inference for the machine learning task) “based on identifying that the LLM compliance score meets or exceeds the compliance threshold, automatically deploy the LLM for use in an enterprise environment” at [0033]. (Perez teaches if the target neural network 120 passes all the criteria., then the pre-deployment evaluation system 100 can provide the data specifying the target neural network 120 to the production environment 140 to deploy the target neural network 120 for use to perform inference for the machine learning task) Perez does not teach “input, into a validation model, the plurality of LLM test cases, wherein inputting the plurality of LLM test cases into the validation model produces a plurality of validated LLM test results” as claimed. However, Horesh teaches at Col. 13 line 3 to Col. 15 line 53 and Fig. 2 a method for managing a generative artificial intelligence model including the steps of inputting model input 202 to a first generative AI model 210 (i.e., “an LLM”) to generate first output 212 and a second generative AI model 220 (i.e., “a validation model”) to generate second output 222. The classification model 230 (i.e., “a falsified output evaluation model”) receives the first output 212 and the second output 222 and compares the outputs to generate similarity indicator 252 (i.e., “compliance score”) which is compared with the threshold 254 to make a binary decision as to whether the first output 212 is similar to the second output 222 and generates identification 262. The identification 262 is used by the policy engine 270 to instruct the system 100 to output the first output 212 for use (i.e., “deploy the LLM for use”) in response to identifying that the first output 212 is similar to the second output 222. Thus, it would have been obvious to one of ordinary skill in the art to combine Horesh with Perez’s teaching in order to provide an automated method for reviewing the generative AI outputs to ensure that the outputs are relevant by comparing the outputs with outputs of another generative AI model, instead of manual review and adjustment, which takes a long time, as suggested by Horesh at Col. 13 line 3 to Col. 15 line 53. As per claim 2 , Perez and Horesh teach the platform of claim 1 discussed above. Perez also teaches “wherein a first subset of the plurality of LLM test cases comprises toxic data test cases and a second subset of the plurality of LLM test case comprise unknown data test cases” at [0037]-[0042] and Fig. 2. As per claim 3 , Perez and Horesh teach the platform of claim 2 discussed above. Perez also teaches wherein “the toxic data test cases comprise test cases prompting the LLM to output a false output” at [0037]-[0042] and Fig. 2. As per claim 4 , Perez and Horesh teach the platform of claim 2 discussed above. Perez also teaches wherein “the unknown data test cases comprise test cases prompting the LLM to provide an output for an unknown topic” at [0037]-[0042] and Fig. 2. As per claim 5 , Perez and Horesh teach the platform of claim 1 discussed above. Horesh also teaches wherein “the plurality of LLM test cases comprise prompts for input to the LLM” at Col. 4 lines 55-65. As per claim 6 , Perez and Horesh teach the platform of claim 1 discussed above. Perez also teaches: wherein “the LLM is hosted in a sandbox environment” at Fig. 1. As per claim 7 , Perez and Horesh teach the platform of claim 1 discussed above. Horesh also teaches “based on identifying that the LLM compliance score does not meet or exceed the compliance threshold, identify a significance of the failure to meet or exceed the compliance threshold, wherein the significance comprises one of material, significant, or inconsequential” at Col. 17 line 1 to Col. 20 line 40. As per claim 8 , Perez and Horesh teach the platform of claim 7 discussed above. Horesh also teaches send, to an enterprise computing device of the enterprise environment, a notification of the significance and one or more commands directing the enterprise computing device to display the notification, wherein sending the one or more commands directing the enterprise computing device to display the notification cause the enterprises computing device to display the notification” at Col. 17 line 1 to Col. 20 line 40. As per claim 9 , Perez and Horesh teach the platform of claim 1 discussed above. Horesh also teaches “wherein the compliance threshold is specific to an industry associated with the enterprises environment” Col. 13 line 3 to Col. 15 line 53. As per claim 10 , Perez and Horesh teach the platform of claim 7 discussed above. Horesh also teaches “train, using historical LLM compliance scores and deviation significance information, the falsified output evaluation model, wherein training the falsified output evaluation model configures the falsified output evaluation model to output the LLM compliance score; and update, via a dynamic feedback loop and using the LLM compliance score and a result of the comparison of the LLM compliance score to the compliance threshold, the falsified output evaluation model” at Col. 13 line 48 to Col. 15 line 50 Claims 11-20 recite similar limitations as in claims 1-10 and are therefore rejected by the same reasons. Conclusion Examiner's Note: Examiner has cited particular columns and line numbers in the references applied to the claims above for the convenience of the applicant. Although the specified citations are representative of the teachings of the art and are applied to specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested from the applicant in preparing responses, to fully consider the references in entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the Examiner. In the case of amending the Claimed invention, Applicant is respectfully requested to indicate the portion(s) of the specification which dictate(s) the structure relied on for proper interpretation and also to verify and ascertain the metes and bounds of the claimed invention. Any inquiry concerning this communication or earlier communications from the examiner should be directed to KHANH B PHAM whose telephone number is (571)272-4116. The examiner can normally be reached Monday - Friday, 8am to 4pm. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sanjiv Shah can be reached at (571)272-4098. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /KHANH B PHAM/Primary Examiner, Art Unit 2166 May 20, 2026 Application/Control Number: 18/535,069 Page 2 Art Unit: 2166 Application/Control Number: 18/535,069 Page 3 Art Unit: 2166 Application/Control Number: 18/535,069 Page 4 Art Unit: 2166 Application/Control Number: 18/535,069 Page 5 Art Unit: 2166 Application/Control Number: 18/535,069 Page 6 Art Unit: 2166 Application/Control Number: 18/535,069 Page 7 Art Unit: 2166 Application/Control Number: 18/535,069 Page 8 Art Unit: 2166 Application/Control Number: 18/535,069 Page 9 Art Unit: 2166
Prosecution Insights