CTNF 18/956,504 CTNF 98332 DETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Claims 1-18 are pending and claims 1, 12 and 13 are independent claims. Information Disclosure Statement The information disclosure statements (IDS) submitted on 11/22/2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner. Claim Rejections - 35 USC § 101 07-04-01 AIA 07-04 35 U.S.C. 101 reads as follows: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. Claim 14 is drawn to a " software " per se “Computer program product” and as such is non-statutory subject matter. See MPEP § 2106.1V.B.1 .a. Computer program product not claimed as embodied in computer readable media are descriptive material per se and are not statutory because they are not capable of causing functional change in the computer. See, e.g., Warmerdam, 33 F.3d at 1361, 31 USPQ2d at 1760 (claim to a data structure per se held nonstatutory). Such claimed computer program product do not define any structural and functional interrelationships between the data structure and other claimed aspects of the invention, which permit the data structure's functionality to be realized. In contrast, a claimed computer readable medium encoded with a data structure defines structural and functional interrelationships between the data structure and the computer software and hardware components which permit the data structure's functionality to be realized, and is thus statutory. Similarly, computer programs claimed as computer listings per se, i.e., the descriptions or expressions of the programs are not physical "things." They are neither computer components nonstatutory processes, as they are not "acts" being performed. Such claimed computer programs do not define any structural and functional interrelationships between the computer program and other claimed elements of a computer, which permit the computer program's functionality to be realized. Claims 1-18 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The independent claims 1, 12 and 13 recite “ obtaining …; accessing a corpus of data; providing … identify …; applying …; determining …; and providing …” as drafted cover an abstract idea of data analysis/retrieval and mental steps. More specifically, the “obtaining a prompt for providing the data element; accessing a corpus of data; providing a machine-learned function configured to identify data elements in corpora of data based on prompts; applying the machine-learned function to the corpus of data to identify at least one data element in the corpus of data corresponding to the prompt; determining a confidence measure for the identified at least one data element using a verification function, the verification function being independent from the machine-learned function; and providing the identified at least one data element as the data element based on the confidence measure” which requires just data analysis / retrieval step and mental process. For instance, one can obtain a prompt for providing data element and access a corpus of data from papers, articles, books, websites, etc. since many people can do this using a pen and pencil. Moreover, their claim indicated providing a machine-learned function configured to identify data elements in corpora of data based on prompts. However, the “machine-learned function” in the claim is just a kind of a generalized machine learned function or a generalized LLM (Spec. para 0004) which is basically an additional element. Applying this additional element “machine-learned function” to the corpus of data to identify the data element in the corpus of data can be implemented using a generalized computer. The step of determining a confidence measure for the identified at least one data element requires just a human judgement. The last step of “providing the identified at least one data element as the data element based on the confidence measure” is just an after-solution activity. The claimed invention is, therefore, directed to an abstract idea and a mental process without significantly more and thus, claims 1, 12 and 13 are rejected under 35 U.S.C. 101. Similarly, the dependent claims 2-11 and 14-20 recite similar claim language as in claims 1, 12 and 13. Claim 2 recites “the corpus of data comprises unstructured natural language text, and the machine-learned function is configured to identify data elements in natural language text,” which requires just a mental step of checking at unstructured natural language text and the use of the machine-learned function as an additional element for identifying the data element is something that can be implemented with any generalized machine learning algorithm or any generalized LLM. No other additional limitations are present. Thus, claim 2 is directed to an abstract idea. Claim 3 which recites “obtaining a predetermined data structure with a plurality of data types, and defining the prompt as a prompt directed to identify data elements corresponding to one or more of the plurality of data types, wherein the confidence measure is indicative of a correspondence between the identified at least one data element and the corresponding data type,” which also requires just a mental step and data retrieval/analysis. One can obtain a predetermined data structure with a plurality of data types using any generalized computer and use any generalized LLM to identify the data elements. For these steps, one can apply a mental step of human judgement . No other additional limitations are present. Thus, claim 3 is directed to an abstract idea. Claim 4 recites “the data structure at least one of is based on an ontology of data types or comprises a tree structure of data types,” which also requires just a mental step of checking that the data structure at least one of is based on an ontology of data types or comprises a tree structure of data types. Thus, claim 4 is directed to an abstract idea. Claim 5 recites “the machine-learned function is configured to provide a source in the corpus of data from which source the identified at least one data element was obtained, the verification function is configured to provide confidence measures for identified data elements based on corresponding sources, and the determining the confidence measure comprises inputting the source in the verification function,” which requires just the additional element of generalized machine learning function or any generalized LLM to provide a source in the corpus of data from which source the identified at least one data element was obtained. Once the relation or correspondence between the identified data element and the source of the corpus of data from which the data element source is identified is established, one can apply a human judgement to determine the confidence measure. No other additional limitations are present. Thus, claim 5 is directed to an abstract idea. Claim 6 recites “the machine-learned function is configured to provide a source in the corpus of data from which source the identified at least one data element was obtained, the verification function is configured to derive detailed source information indicating portions within corresponding sources from which data elements have been obtained based on identified data elements and corresponding sources, the determining the confidence measure comprises deriving a detailed source information indicating from which portion within the source the identified at least one data element has been obtained, and the providing comprises providing the detailed source information,” which requires just applying the additional element of machine-learned function provide a source in the corpus of data. Once the relation or correspondence between the identified data element and the source of the corpus of data from which the data element source is identified is established, one can apply a human judgement to determine the confidence measure. , and the providing of the detailed source information would be just an after solution activity. No other additional limitations are present. Thus, claim 6 is directed to an abstract idea. Claim 7 recites “the verification function comprises a second machine-learned function different from the machine-learned function,” which requires just an abstract idea implementing another generalized machine learning algorithm or some other generalized LLM. No other additional limitations are present. Thus, claim 7 is directed to an abstract idea. Claim 8 recites “comparing the confidence measure to a predetermined criterion, if the confidence measure does not fulfill the predetermined criterion providing the identified at least one data element to a user via a user interface, receiving a user input directed to rejecting, accepting, or correcting the identified at least one data element, and providing the identified data element based on the user input,” which requires just a mental step of comparing the confidence measure to a predetermined criterion, and if the confidence measure does not fulfill the predetermined criterion providing the identified data element, it can receive a user input directed to rejecting, accepting, or correcting the identified at least one data element through a generalized computer, and providing the identified data element based on the user input is just an after solution activity. No other additional limitations are present Thus, claim 8 is directed to an abstract idea. Claim 9 recites “receiving a natural language query from a user via a user interface, comparing the confidence measure to a predetermined criterion, if the confidence measure fulfills the predetermined criterion generating a natural language answer to the query by applying a natural language generation function to the identified at least one data element, and providing the answer to the user via the user interface,” which requires just a mental step of receiving a natural language query from a user via a paper and pencil, etc., just apply a mathematical procedure or a mental step of comparing the confidence measure to a predetermined criterion, and if the confidence measure fulfills the predetermined criterion generating mentally a natural language answer to the query to the identified data element, and the last step of providing the answer to the user is basically an after solution activity. No other additional limitations are present Thus, claim 9 is directed to an abstract idea. Claim 10 recites “the corpus of data relates to an electronic medical health record of a patient, and the data element relates to a medical finding,” which requires just a mental step of determining that the corpus of data to be checked out needs to be an electronic medical health record of a patient, and the data element to be looking for should be related to a medical finding. Thus, claim 10 is directed to an abstract idea. Claim 11 recites “the data structure is based on a medical ontology, or the data structure is based on a communication standard in healthcare,” which requires just a mental step of determining that the data structure is based on a medical ontology, or the data structure is based on a communication standard in healthcare. Thus, claim 11 is directed to an abstract idea. Claim 14 recites “program elements that, when executed by a computing unit of a system, cause the system to perform the method of claim 1,” which requires just additional generalized “program elements” that, when executed by a computing unit of a system, cause the system to perform the method of claim 1. Thus, claim 14 is directed to an abstract idea. Claim 15 recites “non-transitory computer-readable medium comprising program elements that, when executed by a computing unit of a system, cause the system to perform the method of claim 1,” which requires just an additional “non-transitory computer-readable medium” that, when executed by a computing unit of a system, cause the system to perform the method of claim 1. Thus, claim 15 is directed to an abstract idea. Claim 16 recites “the data structure is at least one of SNOMED, RADLEX, FHIR or DICOM,” which requires just a mental step of determining that the data structure is at least one of SNOMED, RADLEX, FHIR or DICOM. Thus, claim 16 is directed to an abstract idea. Claim 17 recites “the machine-learned function is configured to provide a source in the corpus of data from which source the identified at least one data element was obtained, the verification function is configured to provide confidence measures for identified data elements based on corresponding sources, and the determining the confidence measure comprises inputting the source in the verification function,” which requires just a step of employing any generalized machine learning algorithm or any generalized LLM, which is just an additional element, to provide a source in the corpus of data from which source the identified data element was obtained. Once the relation or correspondence between the identified data element and the source of the corpus of data from which the data element source is identified is established, one can apply a human judgement to determine the confidence measure. No other additional limitations are present. Thus, claim 17 is directed to an abstract idea. Claim 18 recites “the machine-learned function is configured to provide a source in the corpus of data from which source the identified at least one data element was obtained, the verification function is configured to derive detailed source information indicating portions within corresponding sources from which data elements have been obtained based on identified data elements and corresponding sources, the determining the confidence measure comprises deriving a detailed source information indicating from which portion within the source the identified at least one data element has been obtained, and the providing comprises providing the detailed source information,” which requires just an implementation of the generalized machine learning algorithm or the generalized LLM, which is considered to be just an additional element, to provide a source in the corpus of data from which source the identified data element was obtained. Once the relation or correspondence between the identified data element and the source of the corpus of data from which the data element source is identified is established, one can apply a human judgement to determine the confidence measure., and then finally the after solution activity of providing output comprises of providing the detailed source information. No other additional limitations are present. Thus, claim 18 is directed to an abstract idea. Thus, claims 1-18 as drafted cover a mental process, mathematical steps and abstract idea of data gathering/retrieval and analysis/processing steps, and they are mental processes directed to an abstract idea of implementing mathematical procedure for data processing and data analysis using a conventional/generic (general-purpose) computer or generalized ML function or generalized LLM and thus, all the claims are directed to an abstract idea. This judicial exception is not integrated into a practical application. In particular, claims 1-2, 5-7, 12-13 and 17-18 recite an additional element of “machine-learned function,” and claim 15 recites an additional element of “non-transitory computer- readable medium” as per the independent and dependent claims. Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea. Thus, taken alone, the additional elements do not amount to significantly more than the above-identified judicial exception (the abstract idea). Looking at the limitations as an ordered combination adds nothing that is not already present when looking at the elements taken individually. There is no indication that the combination of elements improves the functioning of a computer or improves any other technology. Their collective functions merely provide conventional general purpose computer implementation. Claims 1-18, are therefore not drawn to patent eligible subject matter as they are directed to an abstract idea without significantly more. Thus, the claimed invention is directed to an abstract idea and a mental process without significantly more and thus, claims 1-18 are rejected under 35 U.S.C. 101. The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element of using a computer is noted as a general computer as noted. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept (Spec., para 0144). Further, the additional limitation in the claims noted above are directed towards insignificant solution activity. The claims are not patent eligible. Dependent claims 2-11 and 14-18 are also directed toward an abstract idea and do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements when considered both individually and as an ordered combination do not amount to significantly more than the abstract idea. Therefore, claims 1-18 do not contain patent eligible subject matter that has been identified by the courts. Claim Rejections - 35 USC § 103 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-21-aia AIA Claim s 1-2, 4, 7, and 10-15 are rejected under 35 U.S.C. 103 as being unpatentable over Kai He, Rui Mao, Qika Lin, Yucheng Ruan, Xiang Lan, Mengling Feng, Erik Cambria, “ A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics ,” arXiv:2310.05694v1 , Mon, 9 Oct 2023 (He) in view of Yixuan Weng, Minjun Zhu, Fei Xia, Bin Li, Shizhu He, Shengping Liu, Bin Sun, Kang Liu, Jun Zhao, “ Large Language Models are Better Reasoners with Self-Verification, ” In Artificial Intelligence (cs.AI); Computation and Language, arXiv:2212.09561, or arXiv:2212.09561v5, Thu, 19 Oct 2023 (Weng) . Regarding Claim 1. He discloses a computer-implemented method for providing a data element (He, 6 th page, 3 rd para, VisualGPT utilizes linguistic knowledge from large language models and adapts it to …multimodal data. To balance the visual input and prior linguistic knowledge, VisualGPT employs a novel self-resurrecting encoder-decoder attention mechanism that enables the pretrained language model to quickly adapt to a small amount of in-domain image-text data. ChatCAD introduced LLMs into medical-image Computer Aided Diagnosis (CAD) networks; [“Computer Aided…” as “computer-implemented”]), the method comprising: obtaining a prompt for providing the data element (He, 11 th page, 4 th para, the model uses a prompt designer module to pre-process the user’s input); accessing a corpus of data (He, 30 th page, 2 nd para, CNNs trained on publicly accessible chest X-ray datasets); providing a machine-learned function configured to identify data elements in corpora of data based on prompts (He, Figure 11, 24 th page, 3 rd para, Our aim is to assist those interested in training or fine-tuning Healthcare LLMs in easily identifying the appropriate datasets; [i.e., LLM as “a machine-learned function”, Spec. para 0004; Figure 11 shows prompts]); applying the machine-learned function to the corpus of data to identify at least one data element in the corpus of data corresponding to the prompt (He, Fig. 11, 25 th page, left col., 3 rd page – right col., 1 st para, To illustrate, an instruction instance is presented in Figure 11. In this example, the LLM is tasked with identifying chemical-disease relations and understanding that its response should align with the given instruction, rather than predicting the next word. By leveraging a sufficient amount of instruction data for fine-tuning, an LLM can appropriately generate the desired output, as demonstrated in Figure 11); He does not specifically disclose determining a confidence measure for the identified at least one data element using a verification function, the verification function being independent from the machine-learned function, and providing the identified at least one data element as the data element based on the confidence measure. However, Weng, in the same field of endeavor, discloses: determining a confidence measure for the identified at least one data element using a verification function, the verification function being independent from the machine-learned function (Weng, 8 th page, left col, 2 nd para, Masked conditions can guide the LLMs to reason more effectively. As shown in Figure 6, we compared the results of using CMV (Conditional Masked Verification) and TFV (Token Form Verification) for self-verification. We found that the performance of CMV is generally better than TFV; Weng, 5 th page, left col, 2 nd -6 th para, We design a chain of thought prompt, like forward reasoning, to guide LLM in generating a solving process. We input the newly constructed sentences into LLM. For TFV, we can directly count the number of answers that are True as the score, and for CMV, we will match its final result with the masked condition. Due to the limited performance of LLM itself, if the condition is verified only once in the backward verification step, it is easy to have the same score, resulting in a lack of differentiation. To address this, we repeat the sampling decoding process P times, so that the verification score can more accurately reflect the model’s confidence for a given conclusion. The verification score is calculated as follows: PNG media_image1.png 20 277 media_image1.png Greyscale This is an indicator function. Finally, we select the one with the highest verification score from the K candidate answers generated as a result. PNG media_image2.png 43 197 media_image2.png Greyscale For example, for CMV, in Figure 2.3) Verification, we match the results generated by the self-verification of LLM with the masked conditions; [i.e., LLM as “the machine-learned function”, Spec. para 0004; CMV, TFV as “verification function”; “the LLM” as “the machine-learned function”, as per the Spec para 0004; “model’s confidence ” as “confidence measure”; “verification score” and “indication function” as “verification function”]); and providing the identified at least one data element as the data element based on the confidence measure (Weng, 9 th page, left col, 2 nd para – right col, 1 st para, The results in Table 2 provide further evidence that the proposed self-verification technique can effectively improve the accuracy of commonsense reasoning models. Across all 6 datasets, the verification accuracy is consistently and considerably higher than both the random guessing baseline and the standalone CoT model accuracy. For example, on the challenging GSM8K dataset, the verification stage obtains 58.9% accuracy, substantially outper forming the 53.4% CoT accuracy and 35.7% ran domguess accuracy. The largest accuracy gains are witnessed on the MultiArith and SingleEq datasets, where the verification stage lifts the accuracy by 17.8% and 23.5% respectively over the CoT model. This indicates that the self-verification technique is particularly adept at rectifying errors made by the CoT model on arithmetic and symbolic equation problems… In this study, we show that large language models have a strong ability to self-verification, allowing them to assess the conclusions they generate accurately; [Table 2 shows the various datasets against the corresponding verification accuracy values; “verification accuracy” as “confidence measure”, since the “verification score can more accurately reflect the model’s confidence”]). Therefore, it would have been obvious for one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the method of Weng in the method of He because this would enable the proposed method of verification for large language models (LLMs) with the chain of thought (CoT) prompting to improve the reasoning performance on various datasets (Weng, Abstract). Regarding Claim 2. He in view of Weng disclose the method of claim 1, wherein: the corpus of data comprises unstructured natural language text (He, 10 th page, right col, 3 rd para, The digitization of Healthcare has allowed for the sharing and reuse of medical data… Compared to existing medical text data deidentification methods, DeID-GPT demonstrated the highest accuracy and remarkable reliability in masking private information from unstructured medical text while preserving the original structure and meaning of the text. This study is among the first to utilize ChatGPT and GPT-4 for medical text data … the use of LLMs such as ChatGPT/GPT-4 in Healthcare; “unstructured medical text” as “unstructured natural language text”), and the machine-learned function is configured to identify data elements in natural language text (He, 10 th page, right col, 3 rd para, In the study [184], the authors developed a novel de-identification framework called DeID-GPT, which utilizes GPT-4 to automatically identify…This study is among the first to utilize ChatGPT and GPT-4 for medical text data processing … for further research and solution development on the use of LLMs such as ChatGPT/GPT-4 in Healthcare; [i.e., GPT-4 as “the machine-learned function”; “medical text data processing … on the use of LLMs such as ChatGPT/GPT-4 in Healthcare” as “natural language text”]). Regarding Claim 4. He in view of Weng disclose the method of claim 3, wherein the data structure at least one of is based on an ontology of data types or comprises a tree structure of data types (He, 5 th page, left col, 3 rd para- right col, 1 st para, To maintain good results without overburdening physicians, automated dialogue systems are a promising technology for Healthcare. In the early stages, the study of [113] proposed an ontology based dialogue system that supports electronic referrals for breast cancer. This system can handle the informative responses of users based on the medical domain ontology. Another study KR-DS [114] is an end-to-end knowledge routed relational dialogue system that seamlessly incorporates a rich medical knowledge graph into topic transitions in dialogue management; [“ontology of dialogue data system” for “electronic referrals for breast cancer …to handle the informative responses of users based on the medical domain ontology”]). Regarding Claim 7. He in view of Weng disclose the method of claim 1. Furthermore, Weng teaches: wherein the verification function comprises a second machine-learned function different from the machine-learned function (Weng, 5 th page, left col, 2 nd para – right col, 2 nd para, This backward verification chain of thought is similar to solving an equation. We design a chain of thought prompt, like forward reasoning, to guide LLM in generating a solving process. We input the newly constructed sentences into LLM...We conducted experiments to evaluate the original GPT-3 (Chen et al., 2021) (code-davinci-001) model and the Instruct-GPT model (Ouyang et al., 2022) (code-davinci-002). Additionally, we con ducted analysis experiments with public GPT-3 (Brown et al., 2020). All prediction results of different reasoning tasks and datasets are obtained by OpenAI’s API 1. Appendix A.3 shows the reproducibility statement; [i.e., verification test conducted via at least three LLMs (three different “machine-learned functions”), i.e. original GPT-3 model, Instruct-GPT model, and the public GPT-3]). Regarding Claim 10. He in view of Weng disclose the method of claim 3, wherein: the corpus of data relates to an electronic medical health record of a patient (He, 6 th page, 5 th para, recent related developments present a multimodal trend, providing significant support to the data of EHRs, med ical images, and medical sequence signals.), and the data element relates to a medical finding (5 th page, right col, 3 rd para- 6 th page, 1 st para, Medical reports are of significant clinical value to radiologists and specialists, but the process of writing them can be tedious and time-consuming for experienced radiologists, and error-prone for inexperienced ones. Therefore, the automatic generation of medical reports has emerged as a promising research direction in the field of Healthcare combined with AI… This capability can assist radiologists in clinical decision making and reduce the burden of report writing by automat ically drafting reports that describe both abnormalities and relevant normal findings). Regarding Claim 11. He in view of Weng disclose the method of claim 10, wherein at least one of the data structure is based on a medical ontology (He, 5 th page, 4 th para, the study of [113] proposed an ontology based dialogue system that supports electronic referrals for breast cancer. This system can handle the informative responses of users based on the medical domain ontology; [i.e., “dialogue system … for breast cancer” as “data structure”]), or the data structure is based on a communication standard in healthcare (He, 24 th page, left col, 4 th para – right col, 1 st para, the most common sources of data for Healthcare LLMs include EHR, scientific literature, web data, and public knowledge bases. When considering the data structure, QA and dialogue data are the most frequently encountered… The Medical Information Mart for Intensive Care III dataset (MIMIC III) is widely recognized as one of the most widely used EHR datasets… MIMIC III provides valuable and extensive information for research and analysis in the field of Healthcare, which facilitates many PLMs and LLMs developments, such as MIMIC BERT [131], GatorTron [181], and MedAGI [192]; [i.e., MIMIC III EHR data for Healthcare LLMs is well organized in a communication standard for healthcare]). Regarding Claim 12. A computer-implemented method for providing a mapping of an unstructured corpus of data onto a predetermined data structure, the method comprising: providing the predetermined data structure with a plurality of different data types (He, 24 th page, right col, 3 rd -4 th para, health-themed forums on Reddit to form COMETA corpus as LLMs train ing data. Tweets are also usually employed to collect data, and COVID-twitter-BERT [140], Twitter BERT [305], and TwHIN-BERT [306] are trained with these data. Public Knowledge Bases. There exist many Healthcare related knowledge bases, such as UMLS [307], CMeKG [308], BioModels [309], and DrugBank [310]. Among them, UMLS is one of the most popular, which is a repository of biomed ical vocabularies developed by the US National Library of Medicine); obtaining the corpus of data (He, 24 th page, right col, 3 rd para, health-themed forums on Reddit to form COMETA corpus as LLMs training data. Tweets are also usually employed to collect data); providing a machine-learned function configured to map input data to data types (He, Figure 11, 24 th page, 3 rd para, Our aim is to assist those interested in training or fine-tuning Healthcare LLMs in easily identifying the appropriate datasets; [i.e., LLM as “a machine-learned function”, Spec. para 0004; Figure 11 shows prompts]); applying the machine-learned function to the corpus of data to generate a mapping for one or more data elements in the corpus of data to one or more of the plurality of data types (He, Fig. 11, 25 th page, left col., 3 rd page – right col., 1 st para, To illustrate, an instruction instance is presented in Figure 11. In this example, the LLM is tasked with identifying chemical-disease relations and understanding that its response should align with the given instruction, rather than predicting the next word. By leveraging a sufficient amount of instruction data for fine-tuning, an LLM can appropriately generate the desired output, as demonstrated in Figure 11); He does not specifically disclose determining a confidence measure using a verification function for each mapping, the verification function being independent from the machine-learned function, and providing the mapping based on the confidence measure. However, Weng, in the same field of endeavor, discloses: determining a confidence measure using a verification function for each mapping, the verification function being independent from the machine-learned function (Weng, 8 th page, left col, 2 nd para, Masked conditions can guide the LLMs to reason more effectively. As shown in Figure 6, we compared the results of using CMV (Conditional Masked Verification) and TFV (Token Form Verification) for self-verification. We found that the performance of CMV is generally better than TFV; Weng, 5 th page, left col, 2 nd -6 th para, We design a chain of thought prompt, like forward reasoning, to guide LLM in generating a solving process. We input the newly constructed sentences into LLM. For TFV, we can directly count the number of answers that are True as the score, and for CMV, we will match its final result with the masked condition. Due to the limited performance of LLM itself, if the condition is verified only once in the backward verification step, it is easy to have the same score, resulting in a lack of differentiation. To address this, we repeat the sampling decoding process P times, so that the verification score can more accurately reflect the model’s confidence for a given conclusion. The verification score is calculated as follows: PNG media_image1.png 20 277 media_image1.png Greyscale This is an indicator function. Finally, we select the one with the highest verification score from the K candidate answers generated as a result. PNG media_image2.png 43 197 media_image2.png Greyscale For example, for CMV, in Figure 2.3) Verification, we match the results generated by the self-verification of LLM with the masked conditions; [i.e., LLM as “the machine-learned function”, Spec. para 0004; “comparison” as a “confidence measure”; CMV, TFV as “verification function”; “the LLM” as “the machine-learned function”, as per the Spec para 0004; “model’s confidence ” as “confidence measure”; “verification score” and “indication function” as “verification function”]); and providing the mapping based on the confidence measure (Weng, 9 th page, left col, 2 nd para – right col, 1 st para, The results in Table 2 provide further evidence that the proposed self-verification technique can effectively improve the accuracy of commonsense reasoning models. Across all 6 datasets, the verification accuracy is consistently and considerably higher than both the random guessing baseline and the standalone CoT model accuracy. For example, on the challenging GSM8K dataset, the verification stage obtains 58.9% accuracy, substantially outper forming the 53.4% CoT accuracy and 35.7% ran domguess accuracy. The largest accuracy gains are witnessed on the MultiArith and SingleEq datasets, where the verification stage lifts the accuracy by 17.8% and 23.5% respectively over the CoT model. This indicates that the self-verification technique is particularly adept at rectifying errors made by the CoT model on arithmetic and symbolic equation problems… In this study, we show that large language models have a strong ability to self-verification, allowing them to assess the conclusions they generate accurately. We propose a novel method that uses self-verification to generate interpretable scores for ranking results in few-shot tasks. Our approach demonstrates the potential of using self- verification to improve the accuracy and reliability of large language models in reasoning tasks.; [i.e., Table 2 shows the various datasets against the corresponding verification accuracy values; “verification accuracy” as “confidence measure”, since the “verification score can more accurately reflect the model’s confidence”; The self-verification also generates interpretable scores for ranking results means that mapping can be provided based on the self-verification as confidence]). Therefore, it would have been obvious for one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the method of Weng in the method of He because this would enable the proposed method of verification for large language models (LLMs) with the chain of thought (CoT) prompting to improve the reasoning performance on various datasets (Weng, Abstract). Regarding Claim 13. He discloses a system for providing a data element (He, 6 th page, right col, 5 th para, recent related developments present a multimodal trend, providing significant support to the data of EHRs, medical images, and medical sequence signals), the system comprising: an interface unit (He, 5 th page, right col, 2 nd para, interactive system based on LLMs); and a computing unit (He, 2 nd page, left col, 3 rd para, This paper considers GPT-3 [17] as a crucial milestone that signifies the start of the transition from PLMs to LLMs. GPT-3 is the first renowned LLM that has over 100 billion parameters, displays exceptional few-shot learning ability; [“GPT-3” or “LLM” as a computing unit]), wherein the computing unit is configured to, obtain a prompt for providing the data element (He, 11 th page, 4 th para, the model uses a prompt designer module to pre-process the user’s input), access a corpus of data via the interface unit (He, 30 th page, 2 nd para, CNNs trained on publicly accessible chest X-ray datasets), host a machine-learned function configured to identify data elements in corpora of data based on prompts (He, Figure 11, 24 th page, 3 rd para, Our aim is to assist those interested in training or fine-tuning Healthcare LLMs in easily identifying the appropriate datasets; [i.e., LLM as “a machine-learned function”, Spec. para 0004; Figure 11 shows prompts]), apply the machine-learned function to the corpus of data to identify at least one data element in the corpus of data corresponding to the prompt (He, Fig. 11, 25 th page, left col., 3 rd page – right col., 1 st para, To illustrate, an instruction instance is presented in Figure 11. In this example, the LLM is tasked with identifying chemical-disease relations and understanding that its response should align with the given instruction, rather than predicting the next word. By leveraging a sufficient amount of instruction data for fine-tuning, an LLM can appropriately generate the desired output, as demonstrated in Figure 11). He does not specifically disclose determine a confidence measure for the identified at least one data element using a verification function, the verification function being independent from the machine-learned function, and provide the identified at least one data element as the data element based on the confidence measure via the interface unit. However, Weng, in the same field of endeavor, discloses: determine a confidence measure for the identified at least one data element using a verification function, the verification function being independent from the machine-learned function (Weng, 8 th page, left col, 2 nd para, Masked conditions can guide the LLMs to reason more effectively. As shown in Figure 6, we compared the results of using CMV (Conditional Masked Verification) and TFV (Token Form Verification) for self-verification. We found that the performance of CMV is generally better than TFV; Weng, 5 th page, left col, 2 nd -6 th para, We design a chain of thought prompt, like forward reasoning, to guide LLM in generating a solving process. We input the newly constructed sentences into LLM. For TFV, we can directly count the number of answers that are True as the score, and for CMV, we will match its final result with the masked condition. Due to the limited performance of LLM itself, if the condition is verified only once in the backward verification step, it is easy to have the same score, resulting in a lack of differentiation. To address this, we repeat the sampling decoding process P times, so that the verification score can more accurately reflect the model’s confidence for a given conclusion. The verification score is calculated as follows: PNG media_image1.png 20 277 media_image1.png Greyscale This is an indicator function. Finally, we select the one with the highest verification score from the K candidate answers generated as a result. PNG media_image2.png 43 197 media_image2.png Greyscale For example for CMV, in Figure 2.3) Verification, we match the results generated by the self-verification of LLM with the masked conditions; [i.e., LLM as “the machine-learned function”, Spec. para 0004; “comparison” as a “confidence measure”; CMV, TFV as “verification function”; “the LLM” as “the machine-learned function”, as per the Spec para 0004; “model’s confidence ” as “confidence measure”; “verification score” and “indication function” as “verification function”]), and provide the identified at least one data element as the data element based on the confidence measure via the interface unit (Weng, 9 th page, left col, 2 nd para – right col, 1 st para, The results in Table 2 provide further evidence that the proposed self-verification technique can effectively improve the accuracy of commonsense reasoning models. Across all 6 datasets, the verification accuracy is consistently and considerably higher than both the random guessing baseline and the standalone CoT model accuracy. For example, on the challenging GSM8K dataset, the verification stage obtains 58.9% accuracy, substantially outper forming the 53.4% CoT accuracy and 35.7% ran domguess accuracy. The largest accuracy gains are witnessed on the MultiArith and SingleEq datasets, where the verification stage lifts the accuracy by 17.8% and 23.5% respectively over the CoT model. This indicates that the self-verification technique is particularly adept at rectifying errors made by the CoT model on arithmetic and symbolic equation problems… In this study, we show that large language models have a strong ability to self-verification, allowing them to assess the conclusions they generate accurately; [Table 2 shows the various datasets against the corresponding verification accuracy values; “verification accuracy” as “confidence measure”, since the “verification score can more accurately reflect the model’s confidence”]). Therefore, it would have been obvious for one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the method of Weng in the method of He because this would enable the proposed method of verification for large language models (LLMs) with the chain of thought (CoT) prompting to improve the reasoning performance on various datasets (Weng, Abstract). Regarding Claim 14. Computer program product comprising program elements that, when executed by a computing unit of a system, cause the system to perform the method of claim 1 (He, 1 st page, left col., 1 st para, Besides the discussion about Healthcare concerns, we support the computer science community by compiling a collection of open source resources, such as accessible datasets, the latest methodologies, code implementations; [i.e., “code implementation” inherently implies “Computer program product”]; He, 29 th page, left col., 2 nd para, The current evaluation practices predominantly concentrate on assessing the performance of LLMs on one specific medical task… Consequently, there is a clear need for a multitask evaluation system that can comprehensively evaluate the performance of LLMs across various medical tasks). Regarding Claim 15. A non-transitory computer-readable medium comprising program elements that, when executed by a computing unit of a system, cause the system to perform the method of claim 1 (He, 1 st page, left col., 1 st para, Besides the discussion about Healthcare concerns, we support the computer science community by compiling a collection of open source resources, such as accessible datasets, the latest methodologies, code implementations; [i.e., “code implementation” inherently requires “non-transitory computer-readable medium”]; He, 28 th page, left col., 3 rd para, Several studies evaluate the comprehensive capability of LLMs. For example, the study [333] provides a comprehensive evaluation of ChatGPT’s zero-shot performance on various benchmark biomedical tasks, i.e., relation extraction, docu ment classification, question answering, and summarization. Zero-shot ChatGPT achieves comparable performance to fine-tuned generative transformers such as BioGPT and BioBART. Additionally, when evaluated on datasets with limited training data, zero-shot ChatGPT outperforms these fine-tuned models) . 07-21-aia AIA Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over He in view of Weng, further in view of Muras et al. Pat App No. US 20220035728 A1 (Muras), and further in view of Wnek Pat App No. US 20030215137 A1 (Wnek) . Regarding Claim 3. He in view of Weng disclose the method of claim 1, wherein the obtaining the prompt further comprises: obtaining a predetermined data structure with a plurality of data types (24 th page, left col, 4 th para, When considering the data structure, QA and dialogue data are the most frequently encountered. Additionally, apart from the conventional text data used in LLMs, it is crucial to acknowledge the significance of multimodal data; [i.e., “QA and dialogue data” as just two of the most common/frequent “data structures”]), and defining the prompt as a prompt directed to identify data elements corresponding to one or more of the plurality of data types (He, 24 th page, left col, 4 th -5 th para, Our aim is to assist those interested in training or fine-tuning Healthcare LLMs in easily identifying the appropriate datasets. In general, the most common sources of data for Healthcare LLMs include EHR, scientific literature, web data, and public knowledge bases. When considering the data structure, QA and dialogue data are the most frequently encountered; He, 9 th page, Table V, Designed CoT Prompts for Healthcare QA; He, 10 th page, left col, 4 th para, Med-PaLM [99] is a variant of PaLM [40] by employing instruction prompt tuning. Instruction prompt tuning is a parameter-efficient approach for aligning LLMs to new domains using a few exemplars proposed in the study [99]. Instead of using a hard prompt that is specific to each medical dataset, instruction prompt tuning used in this study employs a soft prompt as an initial prefix that is shared across multiple datasets. The soft prompt is then followed by a task-specific human-engineered prompt that includes instructions and/or few-shot exemplars, which may include CoT examples, along with the actual question and/or context; [i.e., “designing prompts” as “defining prompts”]). He in view of Weng do not specifically disclose wherein the confidence measure is indicative of a correspondence between the identified at least one data element and the corresponding data type. However, Wnek, in the same field of endeavor, discloses wherein the confidence measure is indicative of a correspondence between the identified at least one data element and the corresponding data type (Wnek, para 0106, Data element verification involves testing if the element's content matches the general description of that data element acquired from the training document and from the user, or inferred from separate or combined inputs, and stored in the document, record, and data element models. The general description may include data types and scope of valid values. The data element is assigned a confidence based on the degree of match; [i.e., confidence measure is based on matching data elements and data types] ). Therefore, it would have been obvious for one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the method of Wnek in the method of He in view of Weng because this would have enable an effective verification of data elements after their extraction from the document (Wnek, para 0106) . 07-21-aia AIA Claim s 5 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over He in view of Weng, further in view of Muras et al. Pat App No. US 20220035728 A1 (Muras), further in view of Fozdar Pat App No. US 20250077538 A1 (Fozdar), and further in view of Wnek . Regarding Claim 5. He in view of Weng disclose the method of claim 1. Furthermore, Weng teaches: the determining the confidence measure comprises inputting the source in the verification function (Weng, 5 th page, left col, 2 nd -6 th para, This backward verification chain of thought is sim ilar to solving an equation. We design a chain of thought prompt, like forward reasoning, to guide LLM in generating a solving process. We input the newly constructed sentences into LLM… The verification score is calculated as follows: PNG media_image1.png 20 277 media_image1.png Greyscale [i.e., LLM p is input for verification score determination]). He in view of Weng do not specifically disclose wherein the machine-learned function is configured to provide a source in the corpus of data from which source the identified at least one data element was obtained. However, Fozdar, in the same field of endeavor discloses wherein the machine-learned function is configured to provide a source in the corpus of data from which source the identified at least one data element was obtained ( Fozdar, para 0023, the present techniques utilize an LLM to identify the specific data sources that are relevant to the natural language query ), Therefore, it would have been obvious for one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the method of Fozdar in the method of He in view of Weng because this would enable utilizing an LLM to perform this process of identification which provides a significant advantage over conventional data integration techniques because of the complexity of the task (Fozdar, para 0023). He in view of Weng and Fozdar do not specifically disclose the verification function is configured to provide confidence measures for identified data elements based on corresponding sources. However, Wnek, in the same field of endeavor, discloses the verification function is configured to provide confidence measures for identified data elements based on corresponding sources (Wnek, para 0106, Data element verification involves testing if the element's content matches the general description of that data element acquired from the training document and from the user, or inferred from separate or combined inputs, and stored in the document, record, and data element models. The general description may include data types and scope of valid values. The data element is assigned a confidence based on the degree of match; [i.e., confidence measure is based on matching data elements and data types] ). Therefore, it would have been obvious for one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the method of Wnek in the method of He in view of Weng and Fozdar because this would have enable an effective verification of data elements after their extraction from the document (Wnek, para 0106). Regarding Claim 17. The method of claim 2, wherein Furthermore, Weng teaches: the determining the confidence measure comprises inputting the source in the verification function (Weng, 5 th page, left col, 2 nd -6 th para, This backward verification chain of thought is sim ilar to solving an equation. We design a chain of thought prompt, like forward reasoning, to guide LLM in generating a solving process. We input the newly constructed sentences into LLM… The verification score is calculated as follows: PNG media_image1.png 20 277 media_image1.png Greyscale [i.e., LLM p is input for verification score determination]). He in view of Weng do not specifically disclose wherein the machine-learned function is configured to provide a source in the corpus of data from which source the identified at least one data element was obtained. However, Fozdar, in the same field of endeavor discloses wherein the machine-learned function is configured to provide a source in the corpus of data from which source the identified at least one data element was obtained ( Fozdar, para 0023, the present techniques utilize an LLM to identify the specific data sources that are relevant to the natural language query ), Therefore, it would have been obvious for one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the method of Fozdar in the method of He in view of Weng because this would enable utilizing an LLM to perform this process of identification which provides a significant advantage over conventional data integration techniques because of the complexity of the task (Fozdar, para 0023). He in view of Weng and Fozdar do not specifically disclose the verification function is configured to provide confidence measures for identified data elements based on corresponding sources. However, Wnek, in the same field of endeavor, discloses the verification function is configured to provide confidence measures for identified data elements based on corresponding sources (Wnek, para 0106, Data element verification involves testing if the element's content matches the general description of that data element acquired from the training document and from the user, or inferred from separate or combined inputs, and stored in the document, record, and data element models. The general description may include data types and scope of valid values. The data element is assigned a confidence based on the degree of match; [i.e., confidence measure is based on matching data elements and data types] ). Therefore, it would have been obvious for one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the method of Wnek in the method of He in view of Weng and Fozdar because this would have enable an effective verification of data elements after their extraction from the document (Wnek, para 0106) . 07-21-aia AIA Claim s 6 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over He in view of Weng, and further in view of Muras et al. Pat App No. US 20220035728 A1 (Muras) . Regarding Claim 6. He in view of Weng disclose the method of claim 1, wherein: the machine-learned function is configured to provide a source in the corpus of data from which source the identified at least one data element was obtained (He, 4 th page, left col, 4 th para, The aim of Text Classification (TC) is to assign labels to text of different lengths, such as phrases, sentences, paragraphs, or documents. In Healthcare research, a large amount of patient data is collected in the electronic format… A research study [76] proposed several methods, based on hybrid Long Short-Term Memory (LSTM) and bidirectional gated recurrent units (Bi-GRU) to achieve medical TC. These methods were demonstrated effective in the Hallmarks dataset and AIM dataset [77] (Both these two datasets were sourced from biomedical publication abstracts). Another research study [78] used text classification to identify prescription medication mentioned in tweets and achieved good results using models like BERT, RoBERTa, XLNet, ALBERT, and DistillBERT with four proposed information fusion methods; He, 9 th page, right col, 3 rd para – 10 th page, left col, 3 rd para: The study revealed that the general LLM can significantly outperform fine-tuned BERT baselines for Healthcare QA tasks. … Galactica: Aiming to solve the problem of information overload in the scientific field, Galactica was proposed to store, combine, and reason about scientific knowledge, including Healthcare. Galactica [38] was trained on a large corpus of papers, reference material, and knowledge bases to potentially discover hidden connections between different research and bring insights to the surface. Unlike other PLMs and LLMs, which rely on an un-curated crawl-based paradigm, Galactica’s training data consists of 106 billion tokens from high-quality sources, such as papers, reference material, and encyclopedias… Galactica emphasizes the importance of dataset design for LLMs. In response to this, the study curated a high-quality dataset and engineered an interface to interact with the body of knowledge. As a result, Galactica performs exceptionally well in knowledge-intensive scientific tasks, achieving promising results on PubMedQA and MedMCQA; [i.e., The Galactica is a healthcare LLM engineered with an interface to interact with the body of knowledge and designed to identify the sources of the datasets]), the providing comprises providing the detailed source information (He, 4 th page, 5 th para, Drugbank is a free and comprehensive online database that provides information on drugs and drug targets; [“Drugbank… comprehensive online database” as the source information]). He in view of Weng do not specifically disclose the verification function is configured to derive detailed source information indicating portions within corresponding sources from which data elements have been obtained based on identified data elements and corresponding sources, and the determining the confidence measure comprises deriving a detailed source information indicating from which portion within the source the identified at least one data element has been obtained. However, Muras, in the same field of endeavor, discloses: the verification function is configured to derive detailed source information indicating portions within corresponding sources from which data elements have been obtained based on identified data elements and corresponding sources (Muras, para 0007, the system and accompanying methods may intelligently and efficiently identify concepts, relationships, and groupings between data elements within a static or dynamic data source through a variety of ways. For example, the system and accompanying methods may identify such concepts, relationships, and/or groupings by intelligently reasoning about one or more organizational and/or geometrical input sources and one or more language input sources to generate concept, relationship, and grouping hypotheses, using supervised learning techniques to train reasoners utilized by the system and methods, verifying generated hypotheses using an active hypothesis tester, continually improving algorithmic or learned confidence thresholds), the determining the confidence measure comprises deriving a detailed source information indicating from which portion within the source the identified at least one data element has been obtained (Muras, para 0010, information to generate one or more suggested constraints for the field, the order of operations, and/or the transitions of the software application. In certain embodiments, the suggested constraints may include a confidence value, which may be based on a variety of factors. In certain embodiments, the confidence value may be based on the strength of the correlation or association between the processed information and the field, order of operations, and/or transitions of the software application, the quality of the natural language processing of the textual and contextual information, the source of the textual information (e.g. the system may trust one source (e.g. an internal API document) over another source (an online source or internet document) and thus having a higher confidence value for the API document source), the number of reinforcing and/or conflicting sources of constraint information, the complexity of the constraint, a history of a constraint, metadata associated with a constraint (e.g. metadata describing how often the constraint fails or passes, how often the constraint is used, confidence levels/scores for the constraint). Therefore, it would have been obvious for one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the method of Muras in the method of He in view of Weng because this provides enhanced functionality and features that may effectively decrease the effort required to determine concepts, relationship, and/or groupings associated with data elements present in various sources of information (Muras, para 0006). Regarding Claim 18. The method of claim 17, wherein: the machine-learned function is configured to provide a source in the corpus of data from which source the identified at least one data element was obtained (He, 4 th page, left col, 4 th para, The aim of Text Classification (TC) is to assign labels to text of different lengths, such as phrases, sentences, paragraphs, or documents. In Healthcare research, a large amount of patient data is collected in the electronic format… A research study [76] proposed several methods, based on hybrid Long Short-Term Memory (LSTM) and bidirectional gated recurrent units (Bi-GRU) to achieve medical TC. These methods were demonstrated effective in the Hallmarks dataset and AIM dataset [77] (Both these two datasets were sourced from biomedical publication abstracts). Another research study [78] used text classification to identify prescription medication mentioned in tweets and achieved good results using models like BERT, RoBERTa, XLNet, ALBERT, and DistillBERT with four proposed information fusion methods; He, 9 th page, right col, 3 rd para – 10 th page, left col, 3 rd para: The study revealed that the general LLM can significantly outperform fine-tuned BERT baselines for Healthcare QA tasks. … Galactica: Aiming to solve the problem of information overload in the scientific field, Galactica was proposed to store, combine, and reason about scientific knowledge, including Healthcare. Galactica [38] was trained on a large corpus of papers, reference material, and knowledge bases to potentially discover hidden connections between different research and bring insights to the surface. Unlike other PLMs and LLMs, which rely on an un-curated crawl-based paradigm, Galactica’s training data consists of 106 billion tokens from high-quality sources, such as papers, reference material, and encyclopedias… Galactica emphasizes the importance of dataset design for LLMs. In response to this, the study curated a high-quality dataset and engineered an interface to interact with the body of knowledge. As a result, Galactica performs exceptionally well in knowledge-intensive scientific tasks, achieving promising results on PubMedQA and MedMCQA; [i.e., The Galactica is a healthcare LLM engineered with an interface to interact with the body of knowledge and designed to identify the sources of the datasets]), the providing comprises providing the detailed source information (He, 4 th page, 5 th para, Drugbank is a free and comprehensive online database that provides information on drugs and drug targets; [“Drugbank… comprehensive online database” as the source information]). He in view of Weng do not specifically disclose the verification function is configured to derive detailed source information indicating portions within corresponding sources from which data elements have been obtained based on identified data elements and corresponding sources, and the determining the confidence measure comprises deriving a detailed source information indicating from which portion within the source the identified at least one data element has been obtained. However, Muras, in the same field of endeavor, discloses: the verification function is configured to derive detailed source information indicating portions within corresponding sources from which data elements have been obtained based on identified data elements and corresponding sources (Muras, para 0007, the system and accompanying methods may intelligently and efficiently identify concepts, relationships, and groupings between data elements within a static or dynamic data source through a variety of ways. For example, the system and accompanying methods may identify such concepts, relationships, and/or groupings by intelligently reasoning about one or more organizational and/or geometrical input sources and one or more language input sources to generate concept, relationship, and grouping hypotheses, using supervised learning techniques to train reasoners utilized by the system and methods, verifying generated hypotheses using an active hypothesis tester, continually improving algorithmic or learned confidence thresholds), the determining the confidence measure comprises deriving a detailed source information indicating from which portion within the source the identified at least one data element has been obtained(Muras, para 0010, information to generate one or more suggested constraints for the field, the order of operations, and/or the transitions of the software application. In certain embodiments, the suggested constraints may include a confidence value, which may be based on a variety of factors. In certain embodiments, the confidence value may be based on the strength of the correlation or association between the processed information and the field, order of operations, and/or transitions of the software application, the quality of the natural language processing of the textual and contextual information, the source of the textual information (e.g. the system may trust one source (e.g. an internal API document) over another source (an online source or internet document) and thus having a higher confidence value for the API document source), the number of reinforcing and/or conflicting sources of constraint information, the complexity of the constraint, a history of a constraint, metadata associated with a constraint (e.g. metadata describing how often the constraint fails or passes, how often the constraint is used, confidence levels/scores for the constraint). Therefore, it would have been obvious for one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the method of Muras in the method of He in view of Weng because this provides enhanced functionality and features that may effectively decrease the effort required to determine concepts, relationship, and/or groupings associated with data elements present in various sources of information (Muras, para 0006) . 07-21-aia AIA Claim s 8 and 9 are rejected under 35 U.S.C. 103 as being unpatentable over He in view of Weng, and further in view of Wills Pat No. US 20200380032 A1 (Wills) . Regarding Claim 8. He in view of Weng disclose the method of claim 1, wherein the providing comprises: Furthermore, Weng teaches: comparing the confidence measure to a predetermined criterion (Weng, 5 th page, left col, 6 th para, we select the one with the highest verification score from the K candidate answers generated as a result. PNG media_image2.png 43 197 media_image2.png Greyscale ), He in view of Weng do not specifically disclose if the confidence measure does not fulfill the predetermined criterion providing the identified at least one data element to a user via a user interface, receiving a user input directed to rejecting, accepting, or correcting the identified at least one data element, and providing the identified data element based on the user input. However, Wills, in the same field of endeavor, discloses: if the confidence measure does not fulfill the predetermined criterion (Wills, para 0122, In circumstances where the query search result probability is determined to be smaller than the query search result threshold; [i.e., “probability” as “confidence measure”; “threshold” as “predetermined criterion”]), providing the identified at least one data element to a user via a user interface (Wills, para 0122, retrieving additional group-based communication data objects to be added to the first group-based communication data object subset for rendering in a second search results interface; [i.e., “data objects …rendering in a second search results interface” as “providing .. data element to a user via a user interface”]), receiving a user input directed to rejecting, accepting, or correcting the identified at least one data element (Wills, para 0065, A query may be an “action query,” which asks for operations (such as insertion, deletion, and/or updating) on the data stored in a network database or a database… user requesting for operating a search function on the database 109 a to retrieve a subset of a group-based communication data corpus stored in the database 109 a ), and providing the identified data element based on the user input (Wills, para 0114, he search query classification circuitry 310 may send and/or receive data from client devices and offline search index management system 109 . In some implementations, the sent and/or received data may be search queries received from client devices and to be transmitted to the offline search index management system 109 for retrieving group-based communication data objects stored in the database of the offline search index management system 109 via tiered indices search operations. The search query classification circuitry 310 is configured to determine whether to query the high retrieval probability corpus and the low retrieval probability corpus in parallel or not. The search query classification circuitry 310 is further configured to retrieve a subset of a group-based communication data corpus, for rendering a search results interface, by querying the high retrieval probability corpus, the low retrieval probability corpus, or both; [“send data…based on queries” as “provide the identified data element based on the user input”]). Therefore, it would have been obvious for one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the method of Wills in the method of He in view of Weng because would enable a search function utilizing the multi-tiered search index fields to be operated in parallel (i.e., conduct the search function for every tier of search index in parallel), if the query search result probability is below a threshold value (Wills, para 0083). Regarding Claim 9. He in view of Weng disclose the method of claim 1, further comprising: receiving a natural language query from a user via a user interface (He, 27 th page, right col, 3 rd para, To explore the accuracy and completeness of ChatGPT for medical queries, the study [324] collected 284 medical questions from 33 physicians across 17 specialties). Furthermore, Weng teaches: comparing the confidence measure to a predetermined criterion (Weng, 5 th page, left col, 6 th para, we select the one with the highest verification score from the K candidate answers generated as a result PNG media_image2.png 43 197 media_image2.png Greyscale ), He in view of Weng do not specifically disclose if the confidence measure fulfills the predetermined criterion, generating a natural language answer to the query by applying a natural language generation function to the identified at least one data element, and providing the answer to the user via the user interface. However, Wills, in the same field of endeavor, discloses: if the confidence measure fulfills the predetermined criterion (Wills, para 0123, In circumstances where the query search result probability is determined to be larger than the query search result threshold) generating a natural language answer to the query by applying a natural language generation function to the identified at least one data element (Wills, para 0123, generate a group-based communication data object subset for rendering within a combined search results interface]), and providing the answer to the user via the user interface (Wills, para 0103, provide output to the user and, in some embodiments, to receive an indication of a user input. The input/output circuitry 206 may comprise a user interface and may include a display and may comprise a web user interface). Therefore, it would have been obvious for one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the method of Wills in the method of He in view of Weng because would enable a search function utilizing the multi-tiered search index fields to be operated in parallel (i.e., conduct the search function for every tier of search index in parallel), if the query search result probability is below a threshold value (Wills, para 0083) . 07-21-aia AIA Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over He in view of Weng, and further in view of Rajan Pat No. US 11195600 B2 (Rajan) . Regarding Claim 16. He in view of Weng disclose the method of claim 11. He in view of Weng do not specifically disclose wherein the data structure is at least one of SNOMED, RADLEX, FHIR or DICOM. However, Rajan, in the same field of endeavor, discloses wherein the data structure is at least one of SNOMED, RADLEX, FHIR or DICOM (Rajan, col 5, ln 22-27, To extract evidence of aortic stenosis from echocardiogram reports, a large knowledge graph is generated of over 5.6 million concept terms by combining over 70 reference vocabularies such as SNOMED CT, ICD9, ICD10, RadLex… where its concept nodes are used as vocabulary phrases; [“SNOMED, RADLEX” are mapped here]). Therefore, it would have been obvious for one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the method of Rajan in the method of He in view of Weng because this provides an extraction of evidence of a disease condition from clinical reports, with vocabularies such as SNOMED CT, ICD9, ICD10, RadLex, RxNorm, and LOINC, one of disease condition extracted is aortic stenosis and the clinical report is an echocardiogram report, in addition to the general advantage of providing the automatic discrepancy detection in medical data (Rajan, col 5, ln 17-27, and Abstract) Conclusion Any inquiry concerning this communication or earlier communications from the examiner should be directed to MULUGETA T. DUGDA whose telephone number is (703)756-1106. The examiner can normally be reached Mon - Fri, 4:30am - 7:00pm. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Paras D. Shah can be reached at 571-270-1650. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /MULUGETA TUJI DUGDA/Examiner, Art Unit 2653 /Paras D Shah/Supervisory Patent Examiner, Art Unit 2653 06/13/2026 Application/Control Number: 18/956,504 Page 2 Art Unit: 2653 Application/Control Number: 18/956,504 Page 3 Art Unit: 2653 Application/Control Number: 18/956,504 Page 4 Art Unit: 2653 Application/Control Number: 18/956,504 Page 5 Art Unit: 2653 Application/Control Number: 18/956,504 Page 6 Art Unit: 2653 Application/Control Number: 18/956,504 Page 7 Art Unit: 2653 Application/Control Number: 18/956,504 Page 8 Art Unit: 2653 Application/Control Number: 18/956,504 Page 9 Art Unit: 2653 Application/Control Number: 18/956,504 Page 10 Art Unit: 2653 Application/Control Number: 18/956,504 Page 11 Art Unit: 2653 Application/Control Number: 18/956,504 Page 12 Art Unit: 2653 Application/Control Number: 18/956,504 Page 13 Art Unit: 2653 Application/Control Number: 18/956,504 Page 14 Art Unit: 2653 Application/Control Number: 18/956,504 Page 15 Art Unit: 2653 Application/Control Number: 18/956,504 Page 16 Art Unit: 2653 Application/Control Number: 18/956,504 Page 17 Art Unit: 2653 Application/Control Number: 18/956,504 Page 18 Art Unit: 2653 Application/Control Number: 18/956,504 Page 19 Art Unit: 2653 Application/Control Number: 18/956,504 Page 20 Art Unit: 2653 Application/Control Number: 18/956,504 Page 21 Art Unit: 2653 Application/Control Number: 18/956,504 Page 22 Art Unit: 2653 Application/Control Number: 18/956,504 Page 23 Art Unit: 2653 Application/Control Number: 18/956,504 Page 24 Art Unit: 2653 Application/Control Number: 18/956,504 Page 25 Art Unit: 2653 Application/Control Number: 18/956,504 Page 26 Art Unit: 2653 Application/Control Number: 18/956,504 Page 27 Art Unit: 2653 Application/Control Number: 18/956,504 Page 28 Art Unit: 2653 Application/Control Number: 18/956,504 Page 30 Art Unit: 2653 Application/Control Number: 18/956,504 Page 31 Art Unit: 2653 Application/Control Number: 18/956,504 Page 32 Art Unit: 2653 Application/Control Number: 18/956,504 Page 33 Art Unit: 2653 Application/Control Number: 18/956,504 Page 34 Art Unit: 2653 Application/Control Number: 18/956,504 Page 35 Art Unit: 2653 Application/Control Number: 18/956,504 Page 36 Art Unit: 2653 Application/Control Number: 18/956,504 Page 37 Art Unit: 2653 Application/Control Number: 18/956,504 Page 38 Art Unit: 2653 Application/Control Number: 18/956,504 Page 39 Art Unit: 2653 Application/Control Number: 18/956,504 Page 40 Art Unit: 2653 Application/Control Number: 18/956,504 Page 41 Art Unit: 2653 Application/Control Number: 18/956,504 Page 42 Art Unit: 2653 Application/Control Number: 18/956,504 Page 44 Art Unit: 2653 Application/Control Number: 18/956,504 Page 45 Art Unit: 2653 Application/Control Number: 18/956,504 Page 46 Art Unit: 2653 Application/Control Number: 18/956,504 Page 47 Art Unit: 2653 Application/Control Number: 18/956,504 Page 48 Art Unit: 2653 Application/Control Number: 18/956,504 Page 49 Art Unit: 2653
Prosecution Insights