8Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103 is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Priority
Acknowledgment is made of applicant's claim for domestic priority based US provisional application 63/544022 filed on 10/13/2023.
Claim Objection
Claim 15 is objected to for being dependent on claim 1 instead of claim 8. In particular, Claim 15 recites “comparing the conformed response with the classical response”. While claim 8 provided the antecedent basis for conformed response, Claim 1 does not. For the purpose of examination, claim 15 is understood as being dependent on claim 8 rather than claim 1.
Claim Rejections - 35 USC § 103
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 103 that form the basis for the rejections under this section made in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-14 and 16 are rejected under 35 USC 103(a) as being unpatentable over Siracusano et al. (US 2024/0411994 A1) in view of Bolcer et al. (US 2025/0103818 A1).
Regarding Claim 1, Siracusano discloses a method of training a large language model (¶147, LLM zero-shot prompting and in context learning), the method comprising:
obtaining a plurality of terms (¶67, user interface has specified some terms indicating a topic of interest that can be used to download additional information from the internet) and unstructured text associated with a medical topic (¶66, receive unstructured reports; e.g., ¶44, medical records);
generating, using a natural language processing (NLP) (¶71, LLM agent 310 can form / generate a query / prompt for LLM 312; ¶87, LLM agent 310 includes processes to evaluate information received from data acquisition module 308 to generate a prompt template for LLM 312), a plurality of prompts based on the plurality of terms and the unstructured text (¶68, parses content from the report and the download; ¶¶71-72, form a prompt / query for LLM 312 by selecting a portion of the parsed information; ¶73, provide several queries to the LLM 312), the plurality of prompts being configured to train a large language model (LLM) (¶147, LLM zero-shot prompting and in context learning); and
providing each of the plurality of prompts to the LLM (¶73 provide several queries to the LLM 312 to preprocess and extract information), thereby training the LLM on the plurality of terms and the unstructured text (¶147, LLM zero-shot prompting and in context learning).
Siracusano does not disclose that the NLP / LLM agent is an NLP model.
Bolcer discloses using a natural language processing / embedding model to convert unstructured text into embeddings (¶24) in order to generate prompt for LLM (¶25).
It would’ve been obvious to one ordinarily skilled in the art before the effective filing date of the invention to implement the LLM agent with an NLP model to generate the prompts (Siracusano, ¶73) by converting unstructured text (Siracusano, ¶68 and ¶71, parsed unstructured reports) into embeddings representing word or phrase closeness to other words or phrases to enable prompt engineering leveraging the embeddings (Bolcer, ¶25; compare Siracusano, ¶208, embedding is the encoding of a sentence generated by a language model).
Regarding Claim 2, Siracusano discloses wherein the plurality of terms include a plurality of terms from a given domain (¶67, user interface 306 has specified some terms indicating a topic of interest).
Regarding Claim 3, Siracusano discloses wherein the plurality of prompts include a definition of at least one of the plurality of terms (¶195, definition for an entity to be extracted can be provided in an instance of instructing the LLM to use only information that is exclusively contained in the provided input; ¶202, each provided prompt can contain definition for an entity to be extracted can be provided).
Regarding Claim 4, Siracusano discloses providing, to the LLM, a medical record of an individual (¶74, further prompt requests LLM 312 to extract entities from provided input pertaining to a medical record including patients and treatments per ¶44).
Regarding Claim 5, Siracusano discloses wherein obtaining the plurality of terms associated with the medical topic further comprises receiving the plurality of terms from a database over a network (¶67, when user interface specified terms indicating topic of interest, data acquisition module 308 can download additional information from internet 314).
Regarding Claim 6, Siracusano discloses pre-processing the unstructured text (Fig. 5, preprocessing 504; see ¶73 and ¶94).
Regarding Claim 7, Siracusano discloses wherein the plurality of prompts includes the plurality of terms as structured text (¶74, map the extracted information to the desired data model 512 to generate graph representation 510 in accordance to the specified format in the further prompt).
Regarding Claim 16, Siracusano discloses wherein the large language model is trained by:
obtaining a plurality of terms and unstructured text associated with a medical topic (¶68, data acquisition module 308 parses text of report 304; ¶69, unstructured reports; in view of ¶44, medical records);
generating, using a natural language processing (NLP) model, a plurality of prompts based on the plurality of terms and the unstructured text (¶¶72-73, LLM agent generates several queries to LLM 312 based on the parsed text; as modified by Bolcer, ¶¶24-25, using NLP embedding model to convert parsed text into embedding that enables prompt engineering to leverage the embedding to generate LLM prompts); and
providing each of the plurality of prompts to the LLM, thereby training the LLM on the plurality of terms and the unstructured text (¶73, provide several queries to LLM 312 to preprocess and extract information; ¶147, LLM zero-shot prompting and in context learning).
Regarding Claim 8, Siracusano discloses a method of providing medical treatment decision support (¶44 and ¶89, use LLM for question answering by inputting text data including medical records including patients, doctors, treatments, hospitals, and drugs), the method comprising:
receiving a user query (¶¶73-74, perform several queries to the LLM 312 to preprocess and extract information includes receiving further prompt to LLM 312 requesting LLM 312 to extract entities from provided input text data in a specified format and answer a specific question);
providing the user query to a large language model (LLM) (¶74, the further prompt requests LLM 312 to extract entities from provided input text data and answer a specific question; ¶110, using prompt templates to query LLM to obtain different pieces of information required), the LLM trained based on a plurality of terms (¶89, LLM trained on natural language data, uses deep learning to learn underlying patterns and structure of language, and fine-tuned to perform specific NLP tasks such as question answering; e.g., ¶206 and ¶208, LLM trained in such a way that sentences with similar meanings have embeddings that are close based on cosine similarity to confirm that extracted entity / relation is present in the original text);
receiving from the LLM an unstructured response to the user query (¶75, receive information output from LLM 312; ¶115, provide extracted information to the formatting component 508);
processing the unstructured response using an NLP model (¶74, a desired data model including entities and relations to be identified; Fig. 5, data model 512) to conform the unstructured response with a structured data set (¶115 and Fig. 5, formatting component 508 maps information obtained to the ontology required; e.g., map the extracted information to the desired data model 512 to generate graph representation 510 in accordance to the specified format in the further prompt per ¶74); and
outputting the conformed response to a user (¶83, display received information to the user; e.g., ¶89, question answering).
Siracusano does not disclose that the LLM was trained based on a plurality of terms provided by an NLP model.
Bolcer discloses fine-tuning / training a LLM based on ad hoc fine-tuning training set of prompts (¶10) using a natural language processing / embedding model to convert unstructured text into embeddings (¶24) in order to generate the prompts (¶25).
It would’ve been obvious to one ordinarily skilled in the art before the effective filing date of the invention to implement an NLP / language model to convert terms into embeddings and generate prompt based fine-tuning dataset to train LLM (Bolcer, ¶10; compare Siracusano, ¶208, embedding is the encoding of a sentence generated by a language model).
Regarding Claim 9, Siracusano discloses wherein the user query includes a request to provide at least one treatment steps for an individual with a condition (¶74, further prompt request LLM 412 to answer a specific question pertaining to entities and relations in medical records including patients and treatments).
Regarding Claim 10, Siracusano discloses wherein the unstructured response includes information regarding treatment of a condition (¶74, further prompt requests LLM 312 to extract entities from provided input pertaining to a medical record including patients and treatments per ¶44; ¶110, query the LLM to obtain the different pieces of information required).
Regarding Claim 11, Siracusano discloses wherein the structured data set includes at least one definition of one of the plurality of terms (¶207, a task can be about identifying behaviors described in the text and associating them to a definition of a behavior; per ¶126, applicable to medical reports / medical related documents; i.e., tasking the LLM to retrieve definition of text in medical record or health care report).
Regarding Claim 12, Siracusano discloses wherein the unstructured response includes one of the plurality of terms, and conforming the unstructured response includes applying the at least one definition to one of the plurality of terms (¶115, formatting component 508 can map the information obtained from the LLM to the ontology required; per Fig. 6 and ¶151, ontology describes all the entity and relation type comprising conceptual entities and defined relations between these entities).
Regarding Claim 13, Siracusano discloses pre-processing the user query with the NLP model (¶¶73-74, provide several queries to LLM 312 to preprocess and extract information; e.g., provide a desired data model (Fig. 5, data model 512) including entities and relations to be identified).
Regarding Claim 14, Siracusano discloses wherein the plurality of terms includes structured text (Fig. 5, graph representation 510; ¶115, map the information obtained from the LLM to the ontology / desired data model 512 required to generate graph representation 510; ¶116, graph form 510 includes all the extracted entities that are concrete instantiations of the types of entities requested by the user).
Claim 15 are rejected under 35 USC 103(a) as being unpatentable over Siracusano et al. (US 2024/0411994 A1) and Bolcer et al. (US 2025/0103818 A1) as applied to claim 8, in view of Magary et al. (US 2025/0124230 A1).
Regarding Claim 15, Siracusano does not disclose providing the user query to one or more NLP model; receiving from the NLP model a classical response to the user query; and comparing the conformed response with the classical response, thereby detecting hallucination by the large language model.
Magary discloses verifying LLM outputs (¶18) by querying a first LLM to obtain a first response (Fig. 3, steps 302-304 and ¶128, parse output into verifiable statements), providing query to one or more NLP model (¶130, search module 232 executes a second trained ML model and using a search query to query information repository); receiving from the NLP model a classical response to the user query (¶130, search results obtained by the search module 232 executing the second trained ML model); and comparing the conformed response with the classical response (¶130, using the verifiable statement and the search results to output an indication of a degree to which the search results matched the verifiable statements), thereby detecting hallucination by the large language model (¶74, model output verification software application determines that the LLM hallucinated the output).
It would’ve been obvious to one ordinarily skilled in the art before the effective filing date of the invention to providing the user query to a second machine learning model / NLP model, receiving from the second machine learning model / NLP model a classical response to the user query, and comparing the conformed response with the classical response in order to verify the veracity of machine learning outputs (Magary, Abstract).
Conclusion
Prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
US 2024/0330603 A1 discloses training a language model by tuning a prompt.
US 2024/0346254 A1 discloses using large language model to augment a small language model in executing natural language tasks by retrieving external information using the large language model to generate an augmentation input to provide context and a language framework to the small language model to enhance overall outputs.
US 2025/0021767 A1 discloses prompting a plurality of LLMs, receiving responses from the plurality of LLMs, using user feedback indications on the LLM responses to generate training data for training a LLM.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to examiner Richard Z. Zhu whose telephone number is 571-270-1587 or examiner’s supervisor King Poon whose telephone number is 571-272-7440. Examiner Richard Zhu can normally be reached on M-Th, 0730:1700.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RICHARD Z ZHU/Primary Examiner, Art Unit 2654 06/26/2026
Prosecution Insights