DETAILED ACTION
This office action is in response to Applicant’s submission filed on 4/14/2025. Claims 1-20 are pending in the application of which Claims 1, 10, and 16 are independent and have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Applicant’s claim for the benefit of a prior-filed application under 35 U.S.C. 119(e) or under 35 U.S.C. 120, 121, or 365 is acknowledged. The prior-filed application (Provisional application No. 63/633898 Filed on 4/15/2024) is acknowledged.
Information Disclosure Statement
The information disclosure statement(s)(IDS) submitted on 7/31/2025 has been considered by the examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 5, 10 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Zhou et al. ("A Survey of Large Language Models in Medicine: Progress, Application, and Challenge", 2023, arXiv.org)(herein " Zhou "), and in further view of Priestas et al. (US20190005012A1)(herein "Priestas").
Regarding claims 1, 10 and 16 Zhou teaches receiving a plurality of raw reports; (Zhou, Section 3.1:” The typical input for discriminative tasks can be medical questions, clinical notes, medical documents, research papers, and patient EHRs.”)
extracting raw data from the plurality of raw reports; (Zhou, Section 4.2:” Clinical coding, such as the International Classification of Diseases (ICD)142, medication coding, and procedure coding, plays a crucial role in healthcare by standardizing diagnostic, procedural, and treatment information. These codes are essential for tracking health metrics, treatment outcomes, billing, and reimbursement processes. However, the manual entry of these codes is time-consuming and prone to errors. Large language models (LLMs) have shown promise in automating the clinical coding process by extracting relevant medical terms from clinical notes and assigning corresponding codes, including ICD codes159,161,162,163, medication codes (e.g., National Drug Code195 or RxNorm196), and procedure codes (e.g., Current Procedural Terminology197). By leveraging the vast medical knowledge and natural language understanding capabilities of LLMs, healthcare professionals can benefit from reduced workload and improved accuracy in clinical coding.”) Note: raw data is implied from the plurality of raw reports.
providing the extracted raw data to a large language model (LLM); (Zhou, Section 4.2:” By leveraging the vast medical knowledge and natural language understanding capabilities of LLMs, healthcare professionals can benefit from reduced workload and improved accuracy in clinical coding. … The development of LLMs for ICD coding has the potential to reduce the manual effort required from healthcare professionals, improve the accuracy and consistency of coded data, and facilitate more efficient billing and reimbursement processes.”)
providing a prompt to the LLM; (Zhou, Section 4.2:” ChatICD162 and LLM-codex163 both utilize the ChatGPT model with prompts for ICD coding, with LLM-codex163 taking a step further by training an LSTM model on top of the ChatGPT responses, demonstrating its strong performance.”)
receiving a response from the LLM, the response including data labels derived from the extracted raw data; (Zhou, Section 3.1:” The output can be labels, categories, extracted entities, relationships, or answers to specific questions, which are often structured and categorized information derived from the input text.”, and Section 4.2:” developing LLM-based applications for ICD coding, several notable examples can serve as guidance and inspiration, especially in automating the process of ICD coding. PLM-ICD159 is an example of an LLM-based approach that builds upon the RoBERTa model39, fine-tuning it specifically for ICD coding.”)
validating the received response against the plurality of raw reports; and (Zhou, Section 4.6:” LLMs can generate plausible-sounding but factually incorrect information, which can mislead students and healthcare professionals if relied upon without proper verification. This can lead to the propagation of misconceptions, inappropriate treatment strategies, or misdiagnosis 218. To mitigate these risks, it is essential to establish rigorous fact-checking and validation processes and emphasize the importance of critical thinking, evidence-based practice, and the verification of information from multiple reliable sources in medical education.”)
training a machine learning model using the received response. (Zhou, Section 4.2:” ChatICD162 and LLM-codex163 both utilize the ChatGPT model with prompts for ICD coding, with LLM-codex163 taking a step further by training an LSTM model on top of the ChatGPT responses, demonstrating its strong performance.”)
Zhou does not teach, however, Priestas teaches [a computer-implemented method for report parsing using a large language model, the method comprising: - claim 1], [a system for report parsing using a large language model, the system comprising: a data storage device storing instructions for report parsing using a large language model in an electronic storage medium; and a processor configured to execute the instructions to perform operations comprising: - claim 10], and [a non-transitory machine-readable medium storing instructions that, when executed by a computing system, causes the computing system to perform operations for report parsing using a large language model, the operations comprising: -claim 16] (Priestas, Par. 0101:” … a computer readable storage medium 1106. …The computer readable storage medium 1106 may be any suitable medium which participates in providing instructions to the processor(s) 1102 for execution. For example, the computer readable storage medium 1106 may be non-transitory … The instructions or modules stored on the computer readable medium 1106 may include machine readable instructions 1164 executed by the processor(s) 1102 to perform the methods and …”).
filtering the plurality of raw reports; (Priestas, Par. 0048:” … In an example, the classifier 262 can be trained with examples of relevant and non-relevant pages from various EHRs. Non-relevant pages are thus removed from further processing by a document editor 264. For example, a cover page for a medical record, such as a lab report, may be filtered out as irrelevant by the page filter 206.”)
Priestas is considered to be analogous to the claimed invention because it is in the same field of endeavor. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhou further in view of Priestas to filter the plurality of raw reports. Motivation to do so would increase data relevancy and reduced cognitive overload, such as eliminating noise, duplicate entries, and irrelevant data points.
Regarding claim 5, Zhou, as modified above, teaches the method of claim 1.
Zhou, as modified above, further teaches wherein the plurality of raw reports includes one or more of: genomic assays, surgical reports, diagnostic information, histochemical stainings, or second opinions. (Zhou, Section 2.2:” … learning to identify abnormalities and generate diagnostic reports. LLaVA-Med’s102 two-stage IFT process involves aligning medical concepts across visual and textual modalities, followed by fine-tuning the model on diverse medical instructions. Med-Gemini’s108 IFT utilizes a curated dataset of medical instructions and multimodal data, enabling it to comprehend complex medical concepts, procedures, and diagnostic reasoning. Meanwhile, MAIRA-1104 and RadFM106 are two multimodal LLMs specifically designed for radiology applications. MAIRA-1104 employs IFT on a dataset of radiology instructions and corresponding medical images, enabling it to analyze radiological images and generate accurate diagnostic reports. “)
Claims 2,11 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Zhou, and Priestas, and in further view of Doshi et al. (US 20250278643 A1)(herein " Doshi ").
Regarding claims 2, 11 and 17, Zhou, as modified above, teaches the method, the system, and the medium of claim 1, 10, and 16 respectively.
Zhou, as modified above, does not teach, however, Doshi teaches upon determining that one or more reports among the plurality of raw reports failed validation: modifying the prompt or one or more reports among the plurality of raw reports; and providing the modified prompt or the modified one or more reports to the LLM. (Doshi, Par. 0027:” … It is known that GenAI models can “hallucinate” [failed validation] and thereby provide factually incorrect or nonsensical output. Embodiments described herein beneficially engineer input prompts [prompt modification] to include explicit instructions that overcome this technical problem. Such instructions may, for example, provide exemplary input and output examples to guide the GenAI model in generating a meaningful GenAI explanation based on an application's output.”)
Doshi is considered to be analogous to the claimed invention because it is in the same field of endeavor. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhou, as modified above, further in view of Doshi to modify the prompt or one or more reports among the plurality of raw reports; and providing the modified prompt or the modified one or more reports to the LLM. Motivation to do so would allow the system to self-correct and recover automatically without human intervention.
Claims 3, 12 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Zhou, and Priestas, and in further view of Wellmann et al. (US20220335518A1)(herein " Wellmann").
Regarding claims 3, 12 and 18, Zhou, as modified above, teaches the method, the system, and the medium of claim 1, 10, and 16 respectively.
Zhou, as modified above, does not teach, however, Wellmann teaches wherein determining that one or more reports among the plurality of raw reports failed validation further comprises comparing a number of the one or more reports that failed validation against a predetermined threshold. (Wellmann, Par. 0040:” … a threshold number or percentage of training documents are accurately classified, … “).
Wellmann is considered to be analogous to the claimed invention because it is in the same field of endeavor. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhou, as modified above, further in view of Wellmann to compare a number of the one or more reports that failed validation against a predetermined threshold. Motivation to do so would allow the system to differentiate between acceptable background noise and systemic failures.
Claims 4, 13 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Zhou, and Priestas, and in further view of Baldua et al. (US20220335518A1)(herein " Baldua ").
Regarding claims 4, 13 and 19, Zhou, as modified above, teaches the method, the system, and the medium of claim 1, 10, and 16 respectively.
Zhou, as modified above, does not teach, however, Baldua teaches upon determining that a number of reports among the plurality of raw reports that failed validation has increased: modifying the prompt to remove any modifications, or modifying the prompt to match an intermediate version of the prompt, or request manual intervention by a user. (Baldua, Par. 0030:” … In artificial intelligence, a hallucination is often defined as model output, e.g., generated content, that diverges from the model input, e.g., is nonsensical, incorrect, or unrelated to the provided input. If the model input is not clearly defined or is repetitive, the risk of AI hallucination increases. “, and Par. 0032:” … the disclosed technologies can dynamically modify the prompt to, for example, increase or decrease the maximum number of facets specified in the constraint, in accordance with the subsequent context data. As another example, embodiments can dynamically select or omit certain prompt portions (e.g., include or omit certain instructions), or dynamically change the prompt type (e.g., switch from a few-shot prompt to a zero-shot prompt or from a chain-of-thought prompt to a few-shot prompt) in response to changes in performance metrics associated with large language model, availability of data resources, and/or other context data.”)
Baldua is considered to be analogous to the claimed invention because it is in the same field of endeavor. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhou, as modified above, further in view of Baldua to modify the prompt to remove any modifications, or modifying the prompt to match an intermediate version of the prompt, or request manual intervention by a user. Motivation to do so would prevent the system from getting stuck in infinite retry loops.
Claims 6, 14 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Zhou, and Priestas, and in further view of Kulshreshtha et al. (US20220335518A1)(herein " Kulshreshtha ").
Regarding claims 6, 14 and 20, Zhou, as modified above, teaches the method, the system, and the medium of claim 1, 10, and 16 respectively.
Zhou, as modified above, does not teach, however, Kulshreshtha teaches wherein the prompt includes instructions to cite a portion of an underlying raw report among the plurality of raw reports for each data label derived from the extracted raw data. (Kulshreshtha, Par. 0068:” An indication of a data set comprising a plurality of records to be used for instruction fine-tuning the pre-trained LLM may be obtained at the analytics service (element 906) in the depicted embodiment. Individual records of the data set may comprise (a) an instruction prompt requesting preparation of an annotated evidence-providing summary of a dialog transcript (b) the transcript (c) an annotated summary whose annotated sentences indicate corresponding sentences in the transcript that comprise evidence for the annotated sentences in the depicted embodiment.”)
Kulshreshtha is considered to be analogous to the claimed invention because it is in the same field of endeavor. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhou, as modified above, further in view of Kulshreshtha to modify the prompt to wherein the prompt includes instructions to cite a portion of an underlying raw report among the plurality of raw reports for each data label derived from the extracted raw data. Motivation to do so would constrain the model's output to the provided data.
Claims 7 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Zhou, Priestas, and Kulshreshtha and in further view of Wadhawan et al. (US20220335518A1)(herein " Wadhawan").
Regarding claims 7 and 15, Zhou, as modified above, teaches the method, and the system of claim 6 and 14 respectively.
Zhou, as modified above, does not teach, however, Wadhawan teaches wherein validating the received response against the plurality of raw reports comprises comparing each derived data label against the cited portion of the underlying raw report. (Wadhawan, Par. 0061:” Element 310 may include a hallucination validation service. The hallucination validation service may remove or edit arrays indicated in some way to indicate timestamped PII, but which are, in fact, incorrect or misleading (e.g., hallucinations). For example, they may disclose text not constituting PII, text containing PII but also containing other information that is not, or text which does not exist in the original transcript. The hallucination validation service may use generative AI to check for hallucinations. It may involve comparing received arrays (e.g., from element 309) to the received transcript (e.g., from element 305), to ensure the received arrays' reference text is actually present in the transcript. The hallucination validation service may output arrays indicating timestamped PII, with a higher degree of accuracy than provided to it (e.g., with hallucinations removed).”)
Wadhawan is considered to be analogous to the claimed invention because it is in the same field of endeavor. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhou, as modified above, further in view of Wadhawan to modify the prompt to compare each derived data label against the cited portion of the underlying raw report. Motivation to do so would ensure accuracy, traceability, and contextual integrity in data analysis.
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Zhou, and Priestas, and in further view of Nayar et al. (US 20250209280 A1)(herein " Nayar ").
Regarding claim 8, Zhou, as modified above, teaches the method of claim 1.
Zhou, as modified above, does not teach, however Nayar teaches wherein the prompt includes instructions for determining data labels. (Nayar, Par. 0010:” The computer-implemented method may also include where providing the text data and the tag data to the tag evaluation LLM includes generating, by the one or more processors, a first prompt identifying (i) the text data and (ii) the tag data, where the first prompt is provided to the tag evaluation LLM.”)
Nayar is considered to be analogous to the claimed invention because it is in the same field of endeavor. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhou, as modified above, further in view of Nayar to wherein the prompt includes instructions for determining data labels. Motivation to do so would improve classification accuracy and consistency by eliminating ambiguity for the AI model.
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Zhou, and Priestas, and in further view of Mondlock et al. (US 12039263 B1)(herein " Mondlock ").
Regarding claim 9, Zhou, as modified above, teaches the method of claim 1.
Zhou, as modified above, does not teach, however Mondlock teaches wherein filtering the plurality of raw reports further comprises detecting keywords in each raw report among the plurality of raw reports, or parsing each raw report among the plurality of raw reports using a second large language model (second LLM). (Mondlock, Col. 8, ll. 23-36:” The relevant information identification module 130 may perform topic modeling to identify the relevant documents, assets, and experts. The topic modeling may include (1) performing topic modeling on the user query to identify one or more topic keywords; and (2) searching the document collections 144, asset collections 146, expert collections 148, document collections 162, asset collections 164, and/or expert collections 166 with the topic keywords to identify relevant documents, assets, and/or experts. The relevant information identification module 130 may retrieve the relevant documents and/or relevant assets from the document collections 144, asset collections 146, expert collections 148, document collections 162, asset collections 164, and/or expert collections 166.”)
Mondlock is considered to be analogous to the claimed invention because it is in the same field of endeavor. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhou, as modified above, further in view of Mondlock to detect keywords in each raw report among the plurality of raw reports. Motivation to do so would reduce data volume while accelerating relevant information retrieval.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure. Mukherjee et al. (US 20230335243 A1) teaches in Par. 0023:” … medical reports belonging to one or more patients can be stored in a data lake such that data from the medical reports can be cataloged using AI based data cataloging workflows. Subsequently, SAC workflows can be applied to the cataloged medical reports to generate additional insights and intelligence on the medical reports, including extracting sentiments and assigning relevant medical categories comprising type and context categories, to make the medical reports searchable. Medical reports can be filtered, based on assignment of the relevant medical categories, by subdomains comprising radiology, cardiology, pulmonology, surgery, or other medical subdomains, relevant to a use case, to generate a cohort of medical reports belonging to the one or more patients for the use case. Based on the cohort generation, the pipeline can facilitate intelligent selection of AI based NLP workflows that can generate AI based annotations for medical reports in the cohort of medical reports. The AI based annotations can be evaluated by human annotators and human curators to ensure that the medical reports in the cohort of medical reports are fully annotated and the completed annotation can be used to train the AI based NLP workflows for accuracy in subsequent applications.”
Examiner's Note: Examiner has cited particular columns and line numbers and/or paragraph numbers in the references applied to the claims above for the convenience of the applicant. Although the specified citations are representative of the teachings of the art and are applied to specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested from the applicant in preparing responses, to fully consider the references in entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the Examiner.
In the case of amending the Claimed invention, Applicant is respectfully requested to indicate the portion(s) of the specification which dictate(s) the structure relied on for proper interpretation and also to verify and ascertain the metes and bounds of the claimed invention.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DARIOUSH AGAHI whose telephone number is (408)918-7689. The examiner can normally be reached Monday - Thursday and alternate Fridays, 7:30-4:30 PT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Bhavesh Mehta can be reached on 571-272-7453. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
DARIOUSH AGAHI, P.E.
Primary Examiner
/DARIOUSH AGAHI/Primary Examiner, Art Unit 2656
