DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on February 10, 2025 and August 25, 2025 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Objections
Claims 6 and 20 are objected to because of the following informalities:
Claim 6, “wherein the first potential patient condition and the second potential patient condition comprises” should read “wherein the first potential patient condition and the second potential patient condition comprise”
Claim 20, “add the recommended digital therapeutic to a report or an entry of a practitioner worklist in association with the patient associate the patient with a patient care network based upon the potential patient condition” should read “add the recommended digital therapeutic to a report or an entry of a practitioner worklist in association with the patient and associate the patient with a patient care network based upon the potential patient condition.”
Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
Invoked despite absence of “means”
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
“image processing modules” in claims 1, 3 and 16-18
“language processing modules” in claims 1 and 13-15
“a key structure detection module” in claims 3 and 17
“a key image localization module” in claim 3, 8 and 17
“a key structure segmentation module” in claims 3-4, 8 and 17-18
Further it should be noted the claims that depend from the claims above do not cure the 35 USC 112(f) interpretation, as such claims 1-20 are interpreted under 35 USC 112(f)
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Based on the 35 USC 112(f) interpretation above, corresponding 35 USC 112(a) and 35 USC 112(b) rejections were considered. Upon review of the specification, the examiner determined 35 USC 112(a) and 35 USC 112(b) rejections were unnecessary. See the chart below for paragraph citations for the applicants PG Publication.
Module
ID Number
Structure/Algorithm (PGPub Paragraphs referenced)
Image processing modules
Paragraph 0087, "In some examples, the one or more image processing modules comprise one or more of: a key structure detection module configured to receive image input and provide key structure presence indicator output based upon the image input, a key image localization module configured to receive image input and provide key image indicator output or key image output, or a key structure segmentation module configured to receive image input and provide region of interest output." Element ID associated with each of the included modules
Structure: see structure associated with the key structure detection module, key image localization module, and key structure segmentation module
Algorithm: paragraph 0087, "In some examples, the one or more image processing modules comprise one or more of: a key structure detection module configured to receive image input and provide key structure presence indicator output based upon the image input, a key image localization module configured to receive image input and provide key image indicator output or key image output, or a key structure segmentation module configured to receive image input and provide region of interest output. " see algorithms for each module below as well
Language processing modules
Paragraph 0049, "a natural language processing module 224"
Paragraph 0067, "a natural language processing module 270 "
Structure: paragraph 0015, "At least some embodiments of the present disclosure are configured to utilize artificial intelligence (AI) and/or natural language processing (NLP) to opportunistically screen for multiple medical conditions within medical imagery and/or medical imaging reports." paragraph 0028, "In some instances, the various acts disclosed herein are performed using a computer system 100. For instance, code/instructions for configuring the computer system 100 to perform the various acts disclosed herein may be stored as instructions 110 on storage 108, and such instructions 110 may be executable by the processor(s) 102 (and/or other components) to facilitate carrying out of the various acts."
Algorithm: paragraph 0049, "As depicted in FIG. 2B, the imaging note(s) 222 may be utilized as input to a natural language processing module 224 trained to detect indications of potential patient conditions within the imaging note(s) 222. The potential patient condition(s) 220 may thus additionally or alternatively be based upon output of the natural language processing module 224 (e.g., generated by processing the imaging note(s) 222)." paragraph 0067, "input to a natural language processing module 270 to determine whether the potential patient condition(s) 220 is/are unmanaged for the patient (decision block 262 in FIG. 2F). For example, the natural language processing module 270 may search through the electronic medical record(s) 268 and/or other data for the patient for terms, ICD, and/or CPT codes that would indicate that the patient has been diagnosed or is being treated for the potential patient condition(s) 220."
Key structure detection module
Paragraph 0032, "a key structure detection module 204 "
Structure: paragraph 0032, "In some implementations, the key structure detection module 204 comprises one or more AI modules that is/are trained to receive image input and output an indication of whether a key structure is present within the input imagery. "
Algorithm: see paragraph 0032-0034
Key image localization module
Paragraph 0035, "a key image localization module 210"
Structure: paragraph 0037, "The key image localization module 210 may comprise one or more AI modules and may take on various forms, such as a deep neural network (e.g., a regression deep neural network, which may be based on a VGG (visual geometry group) or other CNN architecture) or other forms. "
Algorithm: see paragraphs 0035-0038
Key structure segmentation module
Paragraph 0038, "a key structure segmentation module 214"
Structure: paragraph 0040, "The key structure segmentation module 214 may comprise one or more AI modules (e.g., a deep neural network, such as a fully convolutional network, which may comprise a U-Net or other architecture) and/or may employ various segmentation techniques such as thresholding, clustering or dual clustering, compression or histogram based methods, edge detection, region growing, partial differential equation based methods, and/or others."Algorithm: see paragraphs 0038-0039, paragraph 0073
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-9 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Publication No. 2023/00188333 to Das et al. (hereinafter Das), and further in view of U.S. Publication No. 2020/0321100 to Glottmann et al. (hereinafter Glottmann).
Regarding independent claim 1, Das discloses A system (abstract, “In an embodiment, a method comprises accessing, by a system comprising a processor, multimodal clinical data for a plurality of subjects included in one or more clinical data sources.”), comprising:
one or more processors (paragraph 0041, “ The memory can further be operatively coupled to at least one processor (not shown), such that the components (e.g., the multimodal training data generation module 108, the training module 128, the inferencing module 138, and other components described herein), can be executed by the at least one processor to perform the operations described. ”); and
one or more hardware storage devices that store instructions that are executable by the one or more processors (paragraph 0041, “ The memory can further be operatively coupled to at least one processor (not shown), such that the components (e.g., the multimodal training data generation module 108, the training module 128, the inferencing module 138, and other components described herein), can be executed by the at least one processor to perform the operations described. ”) to configure the system to:
determine a first potential patient condition based upon one or more first patient condition metrics (paragraph 0031, “As used herein, a “medical imaging inferencing model” refers to an image inferencing model that is tailored to perform an image processing/analysis task on one or more medical images. For example, the medical imaging processing/analysis task can include (but is not limited to): disease/condition classification, disease region segmentation, organ segmentation, disease quantification, disease/condition staging, risk prediction, temporal analysis, anomaly detection, anatomical feature characterization, medical image reconstruction, and the like. The terms “medical image inferencing model,” “medical image processing model,” “medical image analysis model,” and the like are used herein interchangeably unless context warrants particular distinction amongst the terms.”), the one or more first patient condition metrics being determined using (i) first image processing output of one or more image processing modules, the first image processing output being generated using a first set of input images comprising one or more first images depicting one or more first bodily structures of a first patient (paragraph 0031, “As used herein, a “medical imaging inferencing model” refers to an image inferencing model that is tailored to perform an image processing/analysis task on one or more medical images. For example, the medical imaging processing/analysis task can include (but is not limited to): disease/condition classification, disease region segmentation, organ segmentation, disease quantification, disease/condition staging, risk prediction, temporal analysis, anomaly detection, anatomical feature characterization, medical image reconstruction, and the like. The terms “medical image inferencing model,” “medical image processing model,” “medical image analysis model,” and the like are used herein interchangeably unless context warrants particular distinction amongst the terms.”) and (ii) first natural language processing output of one or more natural language processing modules, the first natural language processing output being generated using one or more first medical imaging reports associated with the first patient or electronic medical record data associated with the first patient (paragraph 0072, “The feature extraction component 106 can employ a variety of text-based and image-based feature detection/extraction algorithms to identify and extract features from image and/or text data included in the initial multimodal datasets 106. For example, the feature extraction component 302 can employ various natural language processing (NLP) to identify and extract key terms and values from text data included in the multimodal datasets 106, such as text data included in clinical reports (e.g., laboratory reports, radiologist reports, clinical notes, etc.), EMRs, metadata associated with medical imaging studies and the like.”);
Das fails to explicitly disclose as further recited. However, Glottman discloses generate a first entry for the first patient at a practitioner worklist based upon the first potential patient condition (paragraph 0069, “The clinical web client 141 may be configured to present a worklist of unreported findings for a certain user and/or reading group. The clinical web client 141 may include a graphical user interface that allows the visualization of different findings.”);
determine a second potential patient condition based upon one or more second patient condition metrics (paragraph 0044, “Additionally, the computer vision (CV) servers 125 may include classification convolutional neural networks that are configured to classify images on a global and/or local level. Examples of global level classifications may include determining whether an image is a CT image, MRI image and the like. Examples of local level classifications may include a classification as to whether a region of interest includes a benign and/or cancerous lesions.”), the one or more second patient condition metrics being determined using (i) second image processing output of the one or more image processing modules, the second image processing output being generated using a second set of input images comprising one or more second images depicting one or more second bodily structures of a second patient (paragraph 0044, “ In some embodiments, the classification convolutional neural networks may be configured to receive an image with a pre-defined size and output a vector with a field corresponding to each possible classification. Each value in the vector may correspond to a score for the given class. Higher values may be associated with a predicted class.”) and (ii) second natural language processing output of the one or more natural language processing modules, the second natural language processing output being generated using one or more second medical imaging reports associated with the second patient (paragraph 0053, “ In some embodiments, the text processing unit 123 may be configured to communicate with one or more natural language processing (NLP) servers 127 configured to apply the natural language processing algorithms to the received textual data in order to generate structured text data.” Paragraph 0055, “In some embodiments, the NLP servers 127 may include classification recurrent neural networks (RNNs). The classification RNNs may be configured to classify text into specific classes on a global and/or local level”); and
generate a second entry for the second patient at the practitioner worklist based upon the second potential patient condition (paragraph 0069, “The clinical web client 141 may be configured to present a worklist of unreported findings for a certain user and/or reading group. The clinical web client 141 may include a graphical user interface that allows the visualization of different findings. The clinical web client 141 may allow for the filtering and sorting of findings, presenting key images and relevant sections from radiology reports, and allowing the management of various reviewing statuses per each detected finding.”).
Das is directed toward, “generating, by the system, a training data cohort comprising the datasets for training a clinical inferencing model to perform the clinical processing task (abstract).” Glottmann is directed toward, “Systems and methods for the improved analysis and generation of medical imaging reports (abstract).” As can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention, Das and Glottmann are directed toward similar methods of endeavor of medical image analysis methods. Further, one of ordinary skill in the art would be aware there are multiple diagnoses that can be applied to a patient based on various data; said differently, a patient could be diagnosed with either “breast cancer” or “stage II breast cancer”, or a patient could be diagnosed with both “cancer” and “COPD” based on different data and analysis. This additional diagnosis information can provide greater detail of an overall patient picture allowing for better review to be performed by a physician for determining treatment. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Glottmann in order to provide greater detail to a reviewing user to determine the most optimal treatment.
Regarding dependent claim 2, the rejection of claim 1 is incorporated herein. Additionally, Das in the combination further discloses wherein the first set of input images or the second set of input images comprises one or more radiography images, computed tomography images, magnetic resonance imaging images, positron emission tomography images, or ultrasound images (paragraph 0032, “The types of medical images processed/analyzed by the medical image inferencing models described herein can include images captured using various types of image capture modalities. For example, the medical images can include (but are not limited to): radiation therapy (RT) images, X-ray (XR) images, digital radiography (DX) X-ray images, X-ray angiography (XA) images, panoramic X-ray (PX) images, computerized tomography (CT) images, mammography (MG) images (including a tomosynthesis device), a magnetic resonance imaging (MRI) images, ultrasound (US) images, color flow doppler (CD) images, position emission tomography (PET) images, single-photon emissions computed tomography (SPECT) images, nuclear medicine (NM) images, and the like.”).
Regarding dependent claim 3, the rejection of claim 1 is incorporated herein. Additionally, Das in the combination further discloses wherein the one or more image processing modules comprise one or more of:
a key structure detection module configured to receive image input and provide key structure presence indicator output based upon the image input, a key image localization module configured to receive image input and provide key image indicator output or key image output, or a key structure segmentation module configured to receive image input and provide region of interest output (paragraph 0054, “The clinical inferencing tasks can include tasks related to triage, such as classification of the medical condition, segmentation of a disease region associated with the medical condition, segmentation of an organ associated with the medical condition or the like.”).
Regarding dependent claim 4, the rejection of claim 3 is incorporated herein. Additionally, Das in the combination further discloses wherein the first potential patient condition or the second potential patient condition is based upon one or more first patient condition metrics or one or more second patient condition metrics, respectively, determined using region of interest output of the key structure segmentation module (paragraph 0031, “As used herein, a “medical imaging inferencing model” refers to an image inferencing model that is tailored to perform an image processing/analysis task on one or more medical images. For example, the medical imaging processing/analysis task can include (but is not limited to): disease/condition classification, disease region segmentation, organ segmentation, disease quantification, disease/condition staging, risk prediction, temporal analysis, anomaly detection, anatomical feature characterization, medical image reconstruction, and the like.” paragraph 0054, “The clinical inferencing tasks can include tasks related to triage, such as classification of the medical condition, segmentation of a disease region associated with the medical condition, segmentation of an organ associated with the medical condition or the like. For instance, as applied to triage of COVID-19 disease based on chest medical images (e.g., provided in one or more capture modalities), the clinical inferencing models 132/132′ can include a model for classifying the chest images with and without the disease, a model for segmenting the COVID-19 disease region to facilitate further inspection by radiologists, a model for segmenting the entire lung even in the presence of lung consolidation and other abnormalities, and the like. The clinical inferencing tasks can include tasks related to disease quantification, staging and risk prediction. For example, in some implementations, the clinical inferencing model 132/132′ can include a model for computing biomarker metrics such as disease region/total lung region expressed as a ratio in XR images. In another example, the clinical inferencing model 132/132′ can include a model that uses volumetric measures in paired CT and XR image data to build a regression model in XR to obtain volumetric measurements from chest XR images. In another example, clinical inferencing model 132/132′ can include a model that determines whether a patient needs a ventilator or not based on chest XR data combined with other multimodal data inputs using regression analysis when outcomes data is available in addition to the image data for training. In another example, the clinical inferencing model 132/132′ can include a model configured to perform temporal analysis and monitor changes in the disease region over time”).
Regarding dependent claim 5, the rejection of claim 1 is incorporated herein. Additionally, Das in the combination further discloses wherein the first entry or the second entry comprises:
identifying information for the first patient or the second patient, respectively (paragraph 0029, “Some example types of clinical data that may be included in a pool of multimodal clinical data from which a data cohort may be generated includes (but is not limited to): medical images and associated metadata (e.g., acquisition parameters), radiology reports, clinical laboratory data, patient EHR data, patient physiological data, pharmacy information, pathology reports, hospital admission data, discharge and transfer data, discharge summaries, and progress notes.”);
the first potential patient condition or the second potential patient condition (paragraph 0031, “ the medical imaging processing/analysis task can include (but is not limited to): disease/condition classification, disease region segmentation, organ segmentation, disease quantification, disease/condition staging, risk prediction, temporal analysis, anomaly detection, anatomical feature characterization, medical image reconstruction, and the like.”), respectively;
identifying information for one or more first key images of the first set of input images or for one or more second key images of the second set of input images, respectively (paragraph 0046, “The scanner localization data can include information regarding the position, orientation and field of view of the imaging scanner relative to one or more anatomical features/planes of reference. In some implementations, the scanner localization data may also include information identifying the relative size, shape and/or position of anatomical features and/or scan-planes. For example, the scanner localization data can provide the relative position and dimensions of the bounding box used to define the capture region, the relative position and orientation of the prescription scan-plane used to capture the image data (e.g., relative to the bounding box and/or one or more anatomical landmarks).”); and/or
one or more recommended practitioner actions based upon the first potential patient condition or the second potential patient condition, respectively (NOTE: not required based on “and/or”).
Regarding dependent claim 6, the rejection of claim 1 is incorporated herein. Additionally, Das in the combination further discloses wherein the first potential patient condition and the second potential patient condition comprises one or more of:
a neurologic condition, a dental condition, a cardiovascular condition, an endocrine condition, a pulmonary condition (paragraph 0054, “The clinical inferencing tasks can include tasks related to triage, such as classification of the medical condition, segmentation of a disease region associated with the medical condition, segmentation of an organ associated with the medical condition or the like. For instance, as applied to triage of COVID-19 disease based on chest medical images (e.g., provided in one or more capture modalities), the clinical inferencing models 132/132′ can include a model for classifying the chest images with and without the disease, a model for segmenting the COVID-19 disease region to facilitate further inspection by radiologists, a model for segmenting the entire lung even in the presence of lung consolidation and other abnormalities, and the like.”), a mammary condition, a musculoskeletal condition, a bone density condition, a gastrointestinal condition, a genitourinary condition, a liver condition, a biliary condition, a gallbladder condition, a pancreatic condition, a spleen condition, an adrenal condition, a kidney condition, a lymph node condition, a metabolic condition, a cancer condition, or a reproductive condition.
Regarding dependent claim 7, the rejection of claim 1 is incorporated herein. Additionally, Das in the combination further discloses wherein the first potential patient condition is determined to be a first undiagnosed or untreated condition, or wherein the second potential patient condition is determined to be a second undiagnosed or untreated condition (paragraph 0054, “the clinical inferencing model 132/132′ can include a model configured to perform temporal analysis and monitor changes in the disease region over time;” if there is no change in a disease region over time or an increase in the disease region the disease would be considered untreated (i.e. either no treatment applied or treatment isn’t working).
Regarding independent claim 8, Das discloses A system (abstract, “In an embodiment, a method comprises accessing, by a system comprising a processor, multimodal clinical data for a plurality of subjects included in one or more clinical data sources.”), comprising:
one or more processors (paragraph 0041, “ The memory can further be operatively coupled to at least one processor (not shown), such that the components (e.g., the multimodal training data generation module 108, the training module 128, the inferencing module 138, and other components described herein), can be executed by the at least one processor to perform the operations described. ”); and
one or more hardware storage devices that store instructions that are executable by the one or more processors (paragraph 0041, “ The memory can further be operatively coupled to at least one processor (not shown), such that the components (e.g., the multimodal training data generation module 108, the training module 128, the inferencing module 138, and other components described herein), can be executed by the at least one processor to perform the operations described. ”) to configure the system to:
obtain a set of input images, the set of input images comprising one or more images, each of the one or more images depicting one or more bodily structures of a patient (paragraph 0024, “The multimodal clinical data can include medical image data, including medical images captured from different capture modalities and information regarding acquisition or capture parameters. The multimodal clinical data can also include non-imaging data, such as (but is not limited to) radiologists reports, physician notes, laboratory data, physiological parameters, electronic health record (EMR) data, demography data and other non-imaging data.”);
after determining that one or more key structures are represented within the one or more images of the set of input images, determine one or more key images of the one or more images of the set of input images by utilizing at least part of the one or more images as input to a key image localization module, the key image localization module being configured to receive image input and provide key image indicator output or key image output (paragraph 0045, “In addition to the medical images themselves, the imaging data 204 can also include or otherwise be associated with a variety of rich information regarding the acquisition parameters/protocol and scanner localization.” Paragraph 0046, “The scanner localization data can include information regarding the position, orientation and field of view of the imaging scanner relative to one or more anatomical features/planes of reference. In some implementations, the scanner localization data may also include information identifying the relative size, shape and/or position of anatomical features and/or scan-planes. For example, the scanner localization data can provide the relative position and dimensions of the bounding box used to define the capture region, the relative position and orientation of the prescription scan-plane used to capture the image data (e.g., relative to the bounding box and/or one or more anatomical landmarks). In some implementations, the scanner localization data may also include one or more low resolution calibration images (e.g., scout images, localizer images, etc.) captured of the patient's region of interest to be scanned prior to capture of subsequent high-resolution (e.g., isotropic) 3D image data 112. The calibration images are generally used to position/align the scanner relative to a desired scan prescription plane for which the 3D image data is captured.”);
determine key structure segmentation by utilizing the one or more key images as input to a key structure segmentation module (paragraph 0042, “System 100 further includes one or more clinical data sources 102 that can collectively provide a pool of multimodal clinical data for a plurality of different patients/subjects for processing by the multimodal training data cohort generation module 108 and the inferencing module 138. ”), the key structure segmentation module being configured to receive image input and provide region of interest output (paragraph 0031, “For example, the medical imaging processing/analysis task can include (but is not limited to): disease/condition classification, disease region segmentation, organ segmentation, disease quantification, disease/condition staging, risk prediction, temporal analysis, anomaly detection, anatomical feature characterization, medical image reconstruction, and the like.”);
determine one or more patient condition metrics using the key structure segmentation (paragraph 0031, “For example, the medical imaging processing/analysis task can include (but is not limited to): disease/condition classification, disease region segmentation, organ segmentation, disease quantification, disease/condition staging, risk prediction, temporal analysis, anomaly detection, anatomical feature characterization, medical image reconstruction, and the like.”).
Das fails to explicitly disclose as further recited. However, Glottmann discloses
(i) generate a report associated with the patient based upon the one or more patient condition metrics (abstract, “generating improved medical image reports and/or alerts based on the generated enhanced medical image data and the generated structured text data.”), or
(ii) generate an entry at one or more practitioner worklists based upon the one or more patient condition metrics (paragraph 0069, “The clinical web client 141 may be configured to present a worklist of unreported findings for a certain user and/or reading group. ”).
Das is directed toward, “generating, by the system, a training data cohort comprising the datasets for training a clinical inferencing model to perform the clinical processing task (abstract).” Glottmann is directed toward, “Systems and methods for the improved analysis and generation of medical imaging reports (abstract).” As can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention, Das and Glottmann are directed toward similar methods of endeavor of medical image analysis methods. Further, one of ordinary skill in the art would be aware when a computer algorithm determines a diagnosis, there should be a clear and concise way of reporting the outputs for user review. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Glottmann in order to provide a simple output containing key information relevant to the patient and the determination made.
Regarding dependent claim 9, the rejection of claim 8 is incorporated herein. Additionally, Das further discloses wherein the one or more key images provide a largest representation of the one or more key structures within the set of input images, or wherein the key structure segmentation comprises segmentation for two or more bodily structures (paragraph 0031, “For example, the medical imaging processing/analysis task can include (but is not limited to): disease/condition classification, disease region segmentation, organ segmentation, disease quantification, disease/condition staging, risk prediction, temporal analysis, anomaly detection, anatomical feature characterization, medical image reconstruction, and the like. The terms “medical image inferencing model,” “medical image processing model,” “medical image analysis model,” and the like are used herein interchangeably unless context warrants particular distinction amongst the terms.”).
Regarding dependent claim 11, the rejection of claim 8 is incorporated herein. Additionally, Das further discloses wherein the report or the entry comprise:
identifying information for the patient (paragraph 0029, “Some example types of clinical data that may be included in a pool of multimodal clinical data from which a data cohort may be generated includes (but is not limited to): medical images and associated metadata (e.g., acquisition parameters), radiology reports, clinical laboratory data, patient EHR data, patient physiological data, pharmacy information, pathology reports, hospital admission data, discharge and transfer data, discharge summaries, and progress notes.”);
a potential patient condition based upon the one or more patient condition metrics (paragraph 0031, “ the medical imaging processing/analysis task can include (but is not limited to): disease/condition classification, disease region segmentation, organ segmentation, disease quantification, disease/condition staging, risk prediction, temporal analysis, anomaly detection, anatomical feature characterization, medical image reconstruction, and the like.”);
identifying information for the one or more key images (paragraph 0046, “The scanner localization data can include information regarding the position, orientation and field of view of the imaging scanner relative to one or more anatomical features/planes of reference. In some implementations, the scanner localization data may also include information identifying the relative size, shape and/or position of anatomical features and/or scan-planes. For example, the scanner localization data can provide the relative position and dimensions of the bounding box used to define the capture region, the relative position and orientation of the prescription scan-plane used to capture the image data (e.g., relative to the bounding box and/or one or more anatomical landmarks).”); and/or
one or more recommended practitioner actions based upon the potential patient condition (NOTE: not required based on “and/or”).
Claim(s) 10 and 12-14 are rejected under 35 U.S.C. 103 as being unpatentable over Das further in view of Glottmann as applied to claim 8 above, and further in view of U.S. Patent No. 12,224,047 to Zeljko Velichkovich et al. (hereinafter Zeljko).
Regarding dependent claim 10, the rejection of claim 8 is incorporated herein. Additionally, Das and Glottman in the combination fail to explicitly disclose wherein the report or the entry indicate whether the one or more patient condition metrics satisfy one or more thresholds or conditions, or wherein the instructions are executable by the one or more processors to configure the system to generate the report or generate the entry at the one or more practitioner worklists after determining that the one or more patient condition metrics satisfy one or more thresholds or conditions.
However, Zeljko discloses wherein the report or the entry indicate whether the one or more patient condition metrics satisfy one or more thresholds or conditions, or wherein the instructions are executable by the one or more processors to configure the system to generate the report or generate the entry at the one or more practitioner worklists after determining that the one or more patient condition metrics satisfy one or more thresholds or conditions (column 8, line 59, “The radiology report data processing system 220 may be used to classify or generate an indication of one or more medical conditions (and medical condition attributes) from prior findings. Details for these medical conditions may include a confidence level for the presence or absence of certain conditions (e.g., a score or measurement that corresponds to a level of recognition or confidence of whether certain conditions are or are not indicated), identification of specific features or areas in the image in which certain conditions are detected or likely to occur, identification of images or areas of images in the prior study in which certain conditions are detected or likely to occur, and similar identifications and indications.”).
As noted above, Das and Glottmann are directed toward similar methods of endeavor of medical image analysis methods. Zeljko is directed toward, “Systems and methods for identifying and presenting prior findings data in a radiology review workflow, based on natural language processing (NLP) of unstructured radiology report text, are disclosed (abstract).” As can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention, Das, Glottmann and Zeljko are directed toward similar methods of endeavor of medical data analysis. Further, one of ordinary skill in the art before the effective filing date of the claimed invention would easily understand when determining if there is a disease presence or not, there is some value that must be met in order to determine that “yes a disease is present” or “no a disease is not present.” Said differently, cancer may be diagnosed or staged based on a threshold of abnormal cells compared to abnormal cells. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Zeljko in order to ensure there are quantifiable measures used to determine disease classification.
Regarding dependent claim 12, the rejection of claim 8 is incorporated herein. Additionally, Das and Glottmann in the combination fail to explicitly disclose wherein the instructions are executable by the one or more processors to configure the system to determine a potential patient condition based upon whether the one or more patient condition metrics satisfy one or more thresholds or conditions.
However, Zeljko discloses wherein the instructions are executable by the one or more processors to configure the system to determine a potential patient condition based upon whether the one or more patient condition metrics satisfy one or more thresholds or conditions (column 8, line 59, “The radiology report data processing system 220 may be used to classify or generate an indication of one or more medical conditions (and medical condition attributes) from prior findings. Details for these medical conditions may include a confidence level for the presence or absence of certain conditions (e.g., a score or measurement that corresponds to a level of recognition or confidence of whether certain conditions are or are not indicated), identification of specific features or areas in the image in which certain conditions are detected or likely to occur, identification of images or areas of images in the prior study in which certain conditions are detected or likely to occur, and similar identifications and indications.”).
As noted above, Das and Glottmann are directed toward similar methods of endeavor of medical image analysis methods. Zeljko is directed toward, “Systems and methods for identifying and presenting prior findings data in a radiology review workflow, based on natural language processing (NLP) of unstructured radiology report text, are disclosed (abstract).” As can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention, Das, Glottmann and Zeljko are directed toward similar methods of endeavor of medical data analysis. Further, one of ordinary skill in the art before the effective filing date of the claimed invention would easily understand when determining if there is a disease presence or not, there is some value that must be met in order to determine that “yes a disease is present” or “no a disease is not present.” Said differently, cancer may be diagnosed or staged based on a threshold of abnormal cells compared to abnormal cells. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Zeljko in order to ensure there are quantifiable measures used to determine disease classification.
Regarding dependent claim 13, the rejection of claim 12 is incorporated herein. Additionally, Das in the combination further discloses wherein the instructions are executable by the one or more processors to configure the system to utilize one or more medical imaging reports as input to a natural language processing module, the one or more medical imaging reports being associated with the one or more images of the set of input images, wherein the potential patient condition is further based upon output of the natural language processing module (paragraph 0072, “paragraph 0072, “The feature extraction component 106 can employ a variety of text-based and image-based feature detection/extraction algorithms to identify and extract features from image and/or text data included in the initial multimodal datasets 106. For example, the feature extraction component 302 can employ various natural language processing (NLP) to identify and extract key terms and values from text data included in the multimodal datasets 106, such as text data included in clinical reports (e.g., laboratory reports, radiologist reports, clinical notes, etc.), EMRs, metadata associated with medical imaging studies and the like.””).
Regarding dependent claim 14, the rejection of claim 12 is incorporated herein. Additionally, Das in the combination further discloses wherein the instructions are executable by the one or more processors to configure the system to:
determine whether the potential patient condition comprises an undiagnosed or untreated condition based upon (i) user input provided based on the report or the entry at the one or more practitioner worklists or (ii) output of a natural language processing module provided by processing one or more electronic medical records associated with the patient (paragraph 0072, “For example, the feature extraction component 302 can employ various natural language processing (NLP) to identify and extract key terms and values from text data included in the multimodal datasets 106, such as text data included in clinical reports (e.g., laboratory reports, radiologist reports, clinical notes, etc.), EMRs, metadata associated with medical imaging studies and the like. ” paragraph 0054, “the clinical inferencing model 132/132′ can include a model configured to perform temporal analysis and monitor changes in the disease region over time;” if there is no change in a disease region over time or an increase in the disease region the disease would be considered untreated (i.e. either no treatment applied or treatment isn’t working).
Claim(s) 15-20 are rejected under 35 U.S.C. 103 as being unpatentable over Das, and further in view of U.S. Publication No. 2022/0160290 to Ejaz et al. (hereinafter Ejaz).
Regarding independent claim 15, Das discloses a system (abstract, “In an embodiment, a method comprises accessing, by a system comprising a processor, multimodal clinical data for a plurality of subjects included in one or more clinical data sources.”), comprising:
one or more processors (paragraph 0041, “ The memory can further be operatively coupled to at least one processor (not shown), such that the components (e.g., the multimodal training data generation module 108, the training module 128, the inferencing module 138, and other components described herein), can be executed by the at least one processor to perform the operations described. ”); and
one or more hardware storage devices that store instructions that are executable by the one or more processors (paragraph 0041, “ The memory can further be operatively coupled to at least one processor (not shown), such that the components (e.g., the multimodal training data generation module 108, the training module 128, the inferencing module 138, and other components described herein), can be executed by the at least one processor to perform the operations described. ”) to configure the system to:
determine a potential patient condition based upon natural language processing output of one or more natural language processing modules (paragraph 0072, “The feature extraction component 106 can employ a variety of text-based and image-based feature detection/extraction algorithms to identify and extract features from image and/or text data included in the initial multimodal datasets 106. For example, the feature extraction component 302 can employ various natural language processing (NLP) to identify and extract key terms and values from text data included in the multimodal datasets 106, such as text data included in clinical reports (e.g., laboratory reports, radiologist reports, clinical notes, etc.), EMRs, metadata associated with medical imaging studies and the like.”), the natural language processing output being generated using one or more medical imaging reports or electronic medical records associated with a patient (paragraph 0072, “The feature extraction component 106 can employ a variety of text-based and image-based feature detection/extraction algorithms to identify and extract features from image and/or text data included in the initial multimodal datasets 106. For example, the feature extraction component 302 can employ various natural language processing (NLP) to identify and extract key terms and values from text data included in the multimodal datasets 106, such as text data included in clinical reports (e.g., laboratory reports, radiologist reports, clinical notes, etc.), EMRs, metadata associated with medical imaging studies and the like.”), wherein the potential patient condition is determined to be an undiagnosed or untreated condition based upon (paragraph 0054, “the clinical inferencing model 132/132′ can include a model configured to perform temporal analysis and monitor changes in the disease region over time;” if there is no change in a disease region over time or an increase in the disease region the disease would be considered untreated (i.e. either no treatment applied or treatment isn’t working) (i) user input provided based on a report or worklist entry indicating the potential patient condition (paragraph 0031, “As used herein, a “medical imaging inferencing model” refers to an image inferencing model that is tailored to perform an image processing/analysis task on one or more medical images. For example, the medical imaging processing/analysis task can include (but is not limited to): disease/condition classification, disease region segmentation, organ segmentation, disease quantification, disease/condition staging, risk prediction, temporal analysis, anomaly detection, anatomical feature characterization, medical image reconstruction, and the like. The terms “medical image inferencing model,” “medical image processing model,” “medical image analysis model,” and the like are used herein interchangeably unless context warrants particular distinction amongst the terms.”) or (ii) the natural language processing output (paragraph 0072, “The feature extraction component 106 can employ a variety of text-based and image-based feature detection/extraction algorithms to identify and extract features from image and/or text data included in the initial multimodal datasets 106. For example, the feature extraction component 302 can employ various natural language processing (NLP) to identify and extract key terms and values from text data included in the multimodal datasets 106, such as text data included in clinical reports (e.g., laboratory reports, radiologist reports, clinical notes, etc.), EMRs, metadata associated with medical imaging studies and the like.”); and
Das fails to explicitly disclose as further recited. However, Ejaz discloses determine a recommended digital therapeutic based upon the potential patient condition for the patient (paragraph 0091, “Embodiments can then use this information as input to customize a therapy protocol or generate recommendation data for a therapy protocol.”).
Das is directed toward, “generating, by the system, a training data cohort comprising the datasets for training a clinical inferencing model to perform the clinical processing task (abstract).” Ejaz is directed toward, “Devices, methods and systems related to the assessment and therapy of neurological conditions and dexterous hand function in particular (abstract).” As can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention, Das and Ejaz are directed toward similar methods of endeavor of medical data analysis. Further, one of ordinary skill in the art would easily understand once a diagnosis is made for a patient, the next step is to determine effective treatment. Said differently, diagnosis alone is often not helpful, a patient needs treatments to improve their condition. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Ejaz in order to quickly determine optimal treatment options for a user to review and then implement.
Regarding dependent claim 16, the rejection of claim 15 is incorporated herein. Additionally, Das discloses wherein determining the potential patient condition is further based upon one or more patient condition metrics, the one or more patient condition metrics being determined using image processing output of one or more image processing modules, the image processing output being generated using a set of input images comprising one or more images depicting one or more bodily structures of a patient (paragraph 0031, “For example, the medical imaging processing/analysis task can include (but is not limited to): disease/condition classification, disease region segmentation, organ segmentation, disease quantification, disease/condition staging, risk prediction, temporal analysis, anomaly detection, anatomical feature characterization, medical image reconstruction, and the like.” Paragraph 0060, “ in one implementation in which the clinical inferencing model 132/132′ is adapted to perform an image analysis task on medical images (e.g., a diagnosis/classification task, a segmentation task, a disease quantification task, etc.),” paragraph 0079,” In particular, medical image processing models adapted to perform clinical inferencing tasks on medical images (e.g., diagnosis tasks, anomaly detection tasks, segmentation tasks, risk/staging tasks, etc.) can be very sensitive to appearance variations in the input medical images”).
Regarding dependent claim 17, the rejection of claim 16 is incorporated herein. Additionally, Das discloses wherein the one or more image processing modules comprise one or more of:
a key structure detection module configured to receive image input and provide key structure presence indicator output based upon the image input, a key image localization module configured to receive image input and provide key image indicator output or key image output (paragraph 0046, “The scanner localization data can include information regarding the position, orientation and field of view of the imaging scanner relative to one or more anatomical features/planes of reference. In some implementations, the scanner localization data may also include information identifying the relative size, shape and/or position of anatomical features and/or scan-planes. For example, the scanner localization data can provide the relative position and dimensions of the bounding box used to define the capture region, the relative position and orientation of the prescription scan-plane used to capture the image data (e.g., relative to the bounding box and/or one or more anatomical landmarks).”), or a key structure segmentation module configured to receive image input and provide region of interest output (paragraph 0060, “ in one implementation in which the clinical inferencing model 132/132′ is adapted to perform an image analysis task on medical images (e.g., a diagnosis/classification task, a segmentation task, a disease quantification task, etc.),”).
Regarding dependent claim 18, the rejection of claim 17 is incorporated herein. Additionally, Das discloses wherein the one or more image processing modules comprises at least the key structure segmentation module (paragraph 0054, “The clinical inferencing tasks can include tasks related to triage, such as classification of the medical condition, segmentation of a disease region associated with the medical condition, segmentation of an organ associated with the medical condition or the like.”), and wherein the potential patient condition is based upon one or more patient condition metrics determined using region of interest output of the key structure segmentation module (paragraph 0054, “”For instance, as applied to triage of COVID-19 disease based on chest medical images (e.g., provided in one or more capture modalities), the clinical inferencing models 132/132′ can include a model for classifying the chest images with and without the disease, a model for segmenting the COVID-19 disease region to facilitate further inspection by radiologists, a model for segmenting the entire lung even in the presence of lung consolidation and other abnormalities, and the like. The clinical inferencing tasks can include tasks related to disease quantification, staging and risk prediction.).
Regarding dependent claim 19, the rejection of claim 15 is incorporated herein. Additionally, Das and Ejaz in the combination fail to explicitly disclose wherein the instructions are executable by the one or more processors to configure the system to: associate the patient with a patient care network based upon the potential patient condition. However, Ejaz discloses at paragraph 0091, “Embodiments can then use this information as input to customize a therapy protocol or generate recommendation data for a therapy protocol.” Thus, Ejaz as a whole does generate recommendations as related to therapy; one of ordinary skill in the art before the effective filing date of the claimed invention would be easily aware part of determining a therapy and treatment plan involves determining the patient has optimal support and care needed for the treatment plan. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the teaching of Das and Ejaz to ensure a patient is set up for treatment success by having a care team.
Regarding dependent claim 20, the rejection of claim 15 is incorporated herein. Additionally, Ejaz discloses wherein the instructions are executable by the one or more processors to configure the system to:
add the recommended digital therapeutic to a report or an entry of a practitioner worklist in association with the patient (see Figure 22)
Das and Ejaz in the combination fail to explicitly disclose associate the patient with a patient care network based upon the potential patient condition. However, Ejaz discloses at paragraph 0091, “Embodiments can then use this information as input to customize a therapy protocol or generate recommendation data for a therapy protocol.” Thus, Ejaz as a whole does generate recommendations as related to therapy; one of ordinary skill in the art before the effective filing date of the claimed invention would be easily aware part of determining a therapy and treatment plan involves determining the patient has optimal support and care needed for the treatment plan. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify the teaching of Das and Ejaz to ensure a patient is set up for treatment success by having a care team.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
U.S. Publication No. 2022/0223293 discloses, “Methods enabling prediction, screening, early diagnosis, and recommended intervention or treatment selection of autoimmune conditions using artificial intelligence operating in conjunction with large medical datasets (Abstract).”
Contact
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Courtney J. Windsor whose telephone number is (571)272-3956. The examiner can normally be reached Monday - Friday 8:00 - 4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John Villecco can be reached at 571-272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
COURTNEY J. WINDSOR
Primary Examiner
Art Unit 2661
Prosecution Insights