DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Receipt of Applicant’s Amendment filed April 14, 2025, is acknowledged.
Response to Amendment
Claims 1-39 have been canceled. Claims 40-57 are pending and are provided to be examined upon their merits.
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Drawings
New corrected drawings in compliance with 37 CFR 1.121(d) are required in this application because certain figures are difficult to read (low resolution, light font, small font). In particular Fig’s 4-6C, 8C and 9. Applicant is advised to employ the services of a competent patent draftsperson outside the Office, as the U.S. Patent and Trademark Office no longer prepares new drawings. The corrected drawings are required in reply to the Office action to avoid abandonment of the application. The requirement for corrected drawings will not be held in abeyance.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 40-57 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claims 40-57 are directed to a method or product, which are statutory categories of invention. (Step 1: YES).
The Examiner has identified method Claim 40 as the claim that represents the claimed invention for analysis and is similar to product claim 49.
Claim 40 recites the limitations of:
A method for conducing cancer treatment effects comprising:
outputting, by a deep learning model, a covariate for adjustment, the deep learning model trained on histopathological slides obtained from cancer subjects, and the covariate comprising a prognostic covariate for a cancer patient based on the histopathological slides;
adjusting the prognostic covariate based on a number of events for a time-to-event outcome identified for the cancer patient; and;
outputting an evaluation of a specific treatment effect for the cancer patient.
These above limitations, under their broadest reasonable interpretation, cover performance of the limitation as certain methods of organizing human activity. The claim recites elements, in non-bold above, which covers performance of the limitation as managing personal behavior. Obtaining histopathological slides from cancer patients and adjusting a covariate based on a number of events for the cancer patient is following rules or instructions, and outputting evaluation of a specific treatment for the cancer patient is teaching, therefore, managing personal behavior. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation as managing personal behavior, then it falls within the “Certain Methods of Organizing Human Activity” grouping of abstract ideas. Accordingly, the claim recites an abstract idea. Claim 49 is also abstract for similar reasons. (Step 2A-Prong 1: YES. The claims are abstract)
In as much as adjusting the prognostic covariate based on number of events and outputting an evaluation of a specific treatment can be done in the mind of a person, with pen and paper, the claims also recite abstract elements under Mental Processes grouping of abstract ideas.
This judicial exception is not integrated into a practical application. In particular, the claims only recite: non-transitory machine-readable medium, processing units (Claim 49). The computer hardware is recited at a high-level of generality (i.e., as a generic processor performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer component. The outputting by a deep learning model a covariate and deep learning model trained on histopathological slides is recited at a high level of generality. There is also no indication of an improvement to deep learning model technology. The claims also recite “outputting an evaluation of a specific treatment” which is not providing the patient with a particular treatment for a particular disease or condition (e.g., injecting a patient with a particular drug for treatment of a disease). Further, there is no teaching of a “specific treatment effect” in the specification for a patient and what that specific treatment is and what disease is being treated. Accordingly, these additional elements, when considered separately and as an ordered combination, do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. Therefore claims 40 and 49 are directed to an abstract idea without a practical application. (Step 2A-Prong 2: NO. The additional claimed elements are not integrated into a practical application)
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because, when considered separately and as an ordered combination, they do not add significantly more (also known as an “inventive concept”) to the exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of using a computer hardware amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Accordingly, these additional elements, when considered separately and as an ordered combination, do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. Steps such as outputting (transmitting) are steps that are considered insignificant extra solution activity and mere instructions to apply the exception using general computer components (see MPEP 2106.05(d), II). Thus claims 40 and 49 are not patent eligible. (Step 2B: NO. The claims do not provide significantly more)
Dependent claims 41-48 and 50-57 further define the abstract idea that is present in their respective independent claims 40 and 49 and thus correspond to Certain Methods of Organizing Human Activity and Mental Processes, and hence are abstract for the reasons presented above. The dependent claims do not include any additional elements that integrate the abstract idea into a practical application or are sufficient to amount to significantly more than the judicial exception when considered both individually and as an ordered combination. Claims 43 and 52 recite “applying a first convolutional neural network” which is applying the model at a high level of generality. Claims 44 and 53 recite “computing an artificial intelligence (AI) risk score…using a machine learning model” which is applying a generic machine at a high level of generality. Claims 45 and 54 recite “computing a clinical risk score” and “using a clinical model, the clinical model trained using one or more subject attributes” where computing and trained are recited as a high level of generality. Therefore, the claims 41-48 and 50-57 are directed to an abstract idea. Thus, the claims 40-57 are not patent-eligible.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 40-57 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim 40 recites “outputting an evaluation of a specific treatment effect for the cancer patient” where there is no teaching of outputting an evaluation of a specific treatment effect for a cancer patient, only conduct a study to evaluate a treatment effect in cancer patients (plural).
The specification teaches:
“In some embodiments, the time-to-event outcome is overall survival, disease free survival, or time to disease relapse. In some embodiments, the RCT is conducted to evaluate a treatment effect in cancer patients. In some embodiments, the cancer is hepatocellular carcinoma, mesothelioma, pancreatic cancer, lung cancer, or breast cancer.” [0035]
Therefore, a trial is conducted to evaluate a treatment. This is not the same as outputting an evaluation of a specific treatment effect for a cancer patient. For examination purposes this is interpreted as outputting some kind of treatment for a patient. Claim 40 has a similar problem.
Claims 41-48 and 50-57 are further rejected as they depend from their respective independent claim.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 40-57 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 40 recites “adjusting the prognostic covariate based on a number of events for a time-to-event outcome identified for the cancer patient” where adjusting based on a number of events for a time-to event outcome for the patient is indefinite. A patient would have one event (e.g., relapse/death/etc.), not a number of events (e.g., para. [0094] of the specification). It is also indefinite as to adjusting the covariate based on one patient, where the specification teaches number of events based on patients (plural) for statistical power (e.g., para. [0155] of the specification). For examination purposes this is interpreted as adjusting covariate based on events. Claim 49 has a similar problem.
Claims 41-48 and 50-57 are further rejected as they depend from their respective independent claim.
Examiner Request
The Applicant is requested to indicate where in the specification there is support for amendments to claims should Applicant amend. The purpose of this is to reduce potential 35 U.S.C. §112(a) or §112 1st paragraph issues that can arise when claims are amended without support in the specification. The Examiner thanks the Applicant in advance.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 40-57 are rejected under 35 U.S.C. 103 as being unpatentable over Pub. No. US 2023/0360758 to Casale et al. in view of Pub. No. US 2022/0359084 to Curtis et al.
Regarding claims 40 and 49
(claim 40) A method for conducing cancer treatment effects comprising:
outputting, by a deep learning model, a covariate for adjustment, the deep learning model trained on histopathological slides obtained from cancer subjects, and the covariate comprising a prognostic covariate for a cancer patient based on the histopathological slides;
Casale et al. teaches:
Tissue slides from biopsy (histopathological slides)…
“With reference to FIG. 2, an exemplary system (e.g., one or more electronic devices) can obtain a plurality of medical images obtained from a group of clinical subjects. The medical images depict a state of a disease of interest. In some embodiments, the plurality of medical images comprises a plurality of biopsy images of biopsy samples from the group of clinical subjects. For example, in a biopsy, one or more tissue slides can be obtained from a subject, and one or more digital images can be taken to capture each tissue slide.” [0237]
Biopsy of cancer patients…
“In an exemplary workflow depicted in FIG. 3A, biopsies 1-n have been performed. Biopsies 1-n may correspond to multiple subjects (e.g., cancer patients) and/or multiple visits (e.g., screening visit, follow-up visit). In the depicted example, the disease of interest is non-alcoholic steatohepatitis (NASH) and the biopsies are H&E stained liver biopsies from a number of clinical trials (however similar workflows may be implemented for other dieses of interest). Each biopsy (e.g., biopsy 1) results in one or more biopsy images (e.g., medical image(s) 302 for biopsy 1). Thus, biopsy 1-n result in a plurality of medical images including medical image(s) 302, medical image(s) 352, etc.” [0238]
Predictive (prognostic) images for determining covariate (prognostic covariate)…
“In some embodiments, predictive images can be identified for determining a covariate of interest. In some embodiments, predictive features may be generated at the tile-level in histopathology data. This may be accomplished using various techniques. For example, a mean aggregate tile embedding may be obtained to generate biopsy embeddings. A linear model may be fit using this data such that covariates can be predicted from biopsy embeddings. A linear model may also be applied to the tile embeddings to generate tile-level scores. As another example, multiple instances of a machine learning model may be fit directly on tile embeddings. For instance, a model may be fit that predicts both a score and a weight for each tile and then performs a weighted average of the scores. Both the score and the weight(s) may be considered as two-dimensional predictive features.” [0216]
Generator and discriminator are neural networks (deep learning model)…
“In some embodiments, the GAN model is a conditional GAN model. For instance, the generator may be configured to receive an embedding, as the condition, and a noise and output a simulated image. The discriminator may be configured to receive an input image, which may be a simulated image or a real image, and classify the input image as simulated or real. During training, the generator generates simulated images, and the simulated images and real images are provided to the discriminator for classification. Based on the outputs of the discriminator, the generator and the discriminator can be updated accordingly to minimize loss. In some embodiments, the generator and the discriminator are neural networks.” [0341]
Cancer patients…
“In an exemplary workflow depicted in FIG. 3A, biopsies 1-n have been performed. Biopsies 1-n may correspond to multiple subjects (e.g., cancer patients) and/or multiple visits (e.g., screening visit, follow-up visit). In the depicted example, the disease of interest is non-alcoholic steatohepatitis (NASH) and the biopsies are H&E stained liver biopsies from a number of clinical trials (however similar workflows may be implemented for other dieses of interest). Each biopsy (e.g., biopsy 1) results in one or more biopsy images (e.g., medical image(s) 302 for biopsy 1). Thus, biopsy 1-n result in a plurality of medical images including medical image(s) 302, medical image(s) 352, etc.” [0238]
See Covariate below.
adjusting the prognostic covariate based on a number of events for a time-to-event outcome identified for the cancer patient; and;
Discover associations with covariates…
“The embodiments described herein are merely exemplary and the discovery platform can be applied to discover associations between any phenotype of interest and a covariate. In some examples described herein, the phenotypic data comprises medical images; the phenotype of interest is a disease of interest (e.g., NASH), which can be represented by a medical diagnosis score (e.g., fibrosis score); the covariate of interest is a genetic variant of interest. However, it should be understood that the techniques described herein can be applied to discover associations between another phenotype of interest and another covariate. Exemplary phenotypic data include, but are not limited to, which is not to suggest that other listings are limiting, in vivo medical images (e.g., MRI, X-ray, CT scan), medical images generated from biopsy samples, such as histopathology data (e.g., H&E stained, Trichrome), clinical biomarker data (e.g., blood test measurements, including proteomic and cfDNA, cognitive/psychiatric assessment scores, microbiome assessment, etc.) and genomic biomarker data (e.g., bulk RNA-seq, methylation data, genomic sequence data, epigenetic sequence data, etc.). Exemplary phenotypes include, but are not limited to, which is not to suggest that other listings are limiting, a disease of interest, gene expression, metabolomics, proteomics, transcriptomics, or lipidomics, etc. Exemplary covariates include, but are not limited to, which is not to suggest that other listings are limiting, demographic information (e.g., age, sex), clinical covariates (e.g., a disease state, a clinical score or a blood biomarkers), genomic data (e.g., genetic data, expression data, methylation data, etc.), etc.” [0205]
See Adjusting below.
See Covariate below.
outputting an evaluation of a specific treatment effect for the cancer patient.
Example of define (outputting) treatment effects…
“The correlation metric for each genetic variant can be compared against a predefined threshold to determine whether there is an association between the genetic variant and the embeddings. In one exemplary implementation, the system uses a Bonferroni-adjusted P value threshold of 0.05 to define significant treatment histological effects and assess whether the treatment has an effect on progression embeddings. In some embodiments, only treatments with significant P values are further studied, for example, using the process 2000 in FIG. 20.” [0381]
Another example of assess (outputting) treatment (singular therefore specific) efficacy, where it is restricted to patients with a cluster (therefore applicable to a patient within the cluster)…
“The system can also assess treatment efficacy within different clusters of patients, for example by using the linear model testing procedure described with respect to model 316 in FIG. 3B to test for association between treatment and the clinical endpoint considering only patients in a given cluster. For example, the system can fit a cluster-specific model that receives a binary indicator for treatment vs placebo and outputs a clinical endpoint (e.g. fibrosis progression). The analysis may then be restricted to patients within a specific cluster.” [0450]
Covariate
Casale et al. teaches covariate and cancer. They do not teach adjust covariate.
Curtis et al. also in the business of covariate and cancer teaches:
Covariate and time to event…
“Cox proportion hazard models are statistical survival models that relate the time that passes to an event and the covariates associated with that quantity in time (See D. R. Cox, J. R. Stat. Soc. B 34, 187-220 (1972), the disclosure of which is herein incorporated by reference). To utilize Cox proportional hazards models, in some embodiments, clinical, molecular, and integrative subtype features are included. In some embodiments, features can be linear and/or polynomial transformed and interaction can include variable selection. In some embodiments, to further simplify the model, stepwise variable selection can be incorporated into the cross validation scheme. Any appropriate computational package can be utilized and/or adapted, such as (for example), the RMS package (https://www.rdocumentation.org/packages/rms).” [0170]
Covariate with neural network (deep learning)…
“To perform the analysis, genomic copy number from a SNP6 array consisting of 1,191,855 segments spanning the entire genome was utilized. Each segment denoted the average copy number in that region. In order to both reduce the dimensionality and obtain useful features, the CNRegions function from the iClusterPlus R package were used to merge adjacent regions and obtain a final set of 4794 consistent copy number regions for each sample (of the 1285 patients in the dataset), with adjusted mean copy number values for each region. These were used as features, alongside the clinical covariates such as age at diagnosis, tumor grade, tumor size, and number of tumor-positive lymph nodes in machine learning methods to predict integrative subtype or binary high [IC 1, 2, 6, 9] versus low [IC 3, 4, 7, 8] risk of relapse labels. The performance of various models including logistic regression, support vector machines with a linear kernel, support vector machines with a gaussian kernel, and neural networks were evaluated to determine their ability to accurately predict integrative subtype risk labels from genome-wide copy number data (FIG. 30). While multiple models performed well, the neural network has the strongest performance among the different models, with both the highest AUROC and the highest AUPRC.” [0275]
Outcome associations and time to relapse or death (time to event)…
“The METABRIC dataset was used to generate signatures from gene expression data as detailed in Curtis, et al., (2012), cited supra. Outcome associations, including late relapse, of the METABRIC cohort were also calculated as detailed in Example 1. In this example, the data was limited to ER+/HER2− samples (n=1398). Late relapse is defined as relapse that occurs after 5 years without any previous incidents of relapse after surgery (i.e., relapse free at year 5). Two outcomes were considered, distant relapse free survival and relapse free survival. Distant relapse free survival is defined as time to distance relapse. Relapse free survival is defined as time to distant relapse or disease specific death.” [0290]
Outcome analysis and adjusted clinical covariates based on timepoints…
“To perform outcome analyses, Kaplan Meier plots were generated using the survival packages (model using survfit function) and survminer (plt, using ggsurvplot function. P-values were generated using Logrank test. Hazard ratio was calculated with hazard.ratio function from survcomp package, which was used to measure the effect size of the signature. Concordance Index (C-Index) was calculated using concordance.index from survcomp package. Area under the curve was used to evaluate the prediction performance of the signatures in different time points. Uno's AUROC from AUC.uno function of survAUC package was used to calculate AUROC. To better compare the improvement in prediction with respect to clinical covariates, for each timepoint, the AUC was calculated using a Cox Proportional Hazard model using the risk or the scores along with adjusted clinical covariates. A 20×10-fold cross validation was performed to avoid overfitting in the overestimation of the AUC.” [0291]
It would have been obvious to one of ordinary skill in the art before the effective filing date to include in the method and system of Casale et al. the ability to adjust covariates as taught by Curtis et al. since the claimed invention is merely a combination of old elements and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable. Further motivation is provided by Curtis et al. who teaches the advantages of improvement in prediction using adjusted covariates.
Regarding claims 41 and 50
(claim 41) The method of claim 40, wherein the time-to-event outcome is overall survival, disease free survival, or time to disease relapse.
The combined references teach time-to-event and survival.
Curtis et al. also in the business of covariate and cancer teaches:
Covariate and time to event…
“Cox proportion hazard models are statistical survival models that relate the time that passes to an event and the covariates associated with that quantity in time (See D. R. Cox, J. R. Stat. Soc. B 34, 187-220 (1972), the disclosure of which is herein incorporated by reference). To utilize Cox proportional hazards models, in some embodiments, clinical, molecular, and integrative subtype features are included. In some embodiments, features can be linear and/or polynomial transformed and interaction can include variable selection. In some embodiments, to further simplify the model, stepwise variable selection can be incorporated into the cross validation scheme. Any appropriate computational package can be utilized and/or adapted, such as (for example), the RMS package (https://www.rdocumentation.org/packages/rms).” [0170]
Time with relapse…
“Breast cancer has multiple stages of progression (i.e., a multistate disease), with clinically relevant intermediate endpoints such as recurrence in loco-regional or distant locations. These recurrence events are correlated, and individual survival analyses of one endpoint cannot fully capture patterns of recurrence that may be associated with differential prognosis. A patient's prognosis can differ dramatically depending on when and where a relapse occurs, time since surgery, and time since loco-regional or distant relapse. These distinct states and timescales are generally not accounted for and motivate the development of a unified statistical framework, as proposed here.” [0249]
It would have been obvious to one of ordinary skill in the art before the effective filing date to include in the method and system of the combined references the ability to use time-to-event such as survival or relapse as taught by Curtis et al. since the claimed invention is merely a combination of old elements and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable. Further motivation is provided by Curtis et al. who teaches the use of events for evaluating covariates.
Regarding claims 42 and 51
(claim 42) The method of claim 40, wherein the cancer is hepatocellular carcinoma, mesothelioma, pancreatic cancer, lung cancer, or breast cancer.
Casale et al. teaches:
Cancer patients…
“In an exemplary workflow depicted in FIG. 3A, biopsies 1-n have been performed. Biopsies 1-n may correspond to multiple subjects (e.g., cancer patients) and/or multiple visits (e.g., screening visit, follow-up visit). In the depicted example, the disease of interest is non-alcoholic steatohepatitis (NASH) and the biopsies are H&E stained liver biopsies from a number of clinical trials (however similar workflows may be implemented for other dieses of interest). Each biopsy (e.g., biopsy 1) results in one or more biopsy images (e.g., medical image(s) 302 for biopsy 1). Thus, biopsy 1-n result in a plurality of medical images including medical image(s) 302, medical image(s) 352, etc.” [0238]
The combined references teach cancer. They do not teach breast cancer.
Curtis et al. also in the business of cancer teaches:
Breast cancer…
“Various embodiments are directed towards methods treatments for breast cancer based on its molecular characterization. In various embodiments, the molecular subtype of a breast cancer is determined based on its genetics. In various embodiments, a molecular subtype is indicative breast cancer aggressiveness and risk of relapse. In various embodiments, a molecular subtype is indicative of the molecular pathology of a breast cancer. In various embodiments, a breast cancer is treated based upon aggressiveness, risk of relapse, and molecular drivers as determined by its molecular subtype.” [0004]
It would have been obvious to one of ordinary skill in the art before the effective filing date to include in the method and system of the combined references the ability to consider various cancers such as breast cancer as taught by Curtis et al. since the claimed invention is merely a combination of old elements and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable. Further motivation is provided by the combined references that teach cancer, it would be obvious to consider various types.
Regarding claims 44 and 53
(claim 44) The method of claim 40 further comprising:
obtaining a digital histology image of a histology section from the cancer patient;
Casale et al. teaches:
Digital images captured (obtaining) using biopsy of tissue slides (histology section)…
“With reference to FIG. 2, an exemplary system (e.g., one or more electronic devices) can obtain a plurality of medical images obtained from a group of clinical subjects. The medical images depict a state of a disease of interest. In some embodiments, the plurality of medical images comprises a plurality of biopsy images of biopsy samples from the group of clinical subjects. For example, in a biopsy, one or more tissue slides can be obtained from a subject, and one or more digital images can be taken to capture each tissue slide.” [0237]
dividing the digital image into a set of tiles;
Image split (dividing) into image tiles…
“With reference to FIG. 3A, a medical image can be split into a plurality of image tiles. For example, medical image(s) 302 of biopsy 1 can be split to obtain image tiles 306-1, 306-2, . . . , 306-M.sub.1; medical image(s) 352 of biopsy n can be split to obtain image tiles 356-1, 356-2, . . . , 356-M.sub.n. In some embodiments, image tiles can be extracted from a medical image using a predefined grid and stored as image tiles of a uniform size. In one exemplary implementation, the image tiles are extracted using a predefined grid with tile dimensions 192 m×192 m and the image tiles are saved as images sized 224 pixels×224 pixels.” [0243]
extracting a plurality of feature vectors from the set of tiles, or a subset thereof; and
Translating (extracting) input image into embedding, where embedding can be a vector representation (extracting vector) of input image tile…
“FIG. 4A illustrates an exemplary unsupervised machine-learning model used in block 202. With reference to FIG. 4A, an unsupervised machine-learning model 404 may be configured to receive an input image tile 402 (e.g., one of the image tiles in FIG. 3A) and provide an output tile embedding 406. Tile embedding 406 can be a vector representation of an input image tile 402 (e.g., tile 306-1) in the latent space. Translating an input image into an embedding can significantly reduce the size and dimension of the original data. As an example, an image tile sized 224 pixels×224 pixels can be reduced to a 2,048-dimensional vector. The lower-dimension embedding can be used for downstream processing 476, as described below.” [0246]
computing an artificial intelligence (Al) risk score based on the histology image using a machine learning model, the machine learning model having been trained by processing a plurality of training images to predict a prognosis of the cancer patient,
Image and risk score of likelihood (predict) subject incurring disease of interest…
“In some embodiments, the data associated with a medical image can include genetic data of the subject from whom the biopsy sample is taken. For example, the data (e.g., associated data 304) can include the subject's genetic information related to a plurality of genetic variants (e.g., 100,000 variants, 1 million variants, 10 million variants). For example, the data can indicate whether the subject has each of the plurality of genetic variants. For example, for a genetic variant with two alleles in the population, the medical image may be associated with a genetic variant value of 0, 1, or 2 depending on if the individual has 0, 1, or 2 copies of the least frequent allele. In some embodiments, the genetic data of the subject may be a polygenic risk score, indicating a likelihood of the subject incurring a disease of interest.” [0241]
Example of using unsupervised machine learning (AI) model for score…
“An exemplary method of identifying at least one genetic variant of interest with respect to a disease of interest comprises: inputting a plurality of medical images obtained from a group of clinical subjects into a trained unsupervised machine-learning model to obtain a plurality of embeddings in a latent space, each embedding corresponding to a phenotypic state relative to the disease of interest reflected in one or more of the plurality of medical images; inputting each embedding of the plurality of embeddings into a trained linear regression model to receive a predicted continuous medical diagnosis score for each embedding of the plurality of embeddings to obtain a plurality of predicted medical diagnosis scores, each predicted continuous medical diagnosis score indicative of a state of the disease of interest; associating the plurality of predicted continuous medical diagnosis scores with each candidate genetic variant of a plurality of candidate genetic variants expressed by the group of clinical subjects from whom the plurality of medical images was taken; determining, based on the association, a correlation metric between the disease of interest and each candidate genetic variant to identify the at least one genetic variant of interest from the plurality of candidate genetic variants, the correlation metric indicative of the impact of the candidate genetic variant on the disease of interest.” [0020]
Example of artificial neural network (AI)….
“At block 1610, the system may generate a classification model for determining whether a patient has received the placebo or the treatment based on the plurality of treatment progression embeddings, wherein outputs of the classification model are indicative of a drug response histological phenotype (DRP). With reference to FIG. 17, the system generates a DRP classification model 1730 based on placebo progression embeddings 1722 and treatment progression embeddings 1726. In some embodiments, the classification model (e.g., DRP classification model 1730) is configured to receive an input progression embedding and output a classification result indicating whether a patient has received the placebo or the treatment. The classification model can be implemented, for example, as a logistic regression model, an artificial neural network model, a random forest model, a naïve Bayes model, etc.” [0367]
wherein the Al risk score quantifies the prognosis of the cancer patient.
Example of cancer patients with NASH…
“In an exemplary workflow depicted in FIG. 3A, biopsies 1-n have been performed. Biopsies 1-n may correspond to multiple subjects (e.g., cancer patients) and/or multiple visits (e.g., screening visit, follow-up visit). In the depicted example, the disease of interest is non-alcoholic steatohepatitis (NASH) and the biopsies are H&E stained liver biopsies from a number of clinical trials (however similar workflows may be implemented for other dieses of interest). Each biopsy (e.g., biopsy 1) results in one or more biopsy images (e.g., medical image(s) 302 for biopsy 1). Thus, biopsy 1-n result in a plurality of medical images including medical image(s) 302, medical image(s) 352, etc.” [0238]
Example of predicted (prognosis) score…
“In some embodiments, the plurality of predicted medical diagnosis scores comprises a disease progression score obtained as the difference of predicted medical diagnosis scores at different measurements during the clinical trial.” [0043]
“In block 150, the system may further be configured to evaluate a treatment with respect to progression of a disease of interest. The disease progression can be quantified by continuous medical diagnosis scores. Significant associations between high resolution NASH scores and various treatments are retrieved through an association test. This process can retrieve drug effects on medical diagnosis scores that could not be detected using pathologist-assigned, discrete scores. The continuous scores enable a more precise definition of disease progression, empowering longitudinal expression analysis (e.g., FIG. 26A) and genetic association studies (e.g., FIG. 26B). Details of block 150 are provided herein with reference to FIG. 22.” [0229]
Regarding claims 45 and 54
(claim 45) The method of claim 44, further comprising:
obtaining clinical attributes derived from the cancer patient;
Casale et al. teaches:
Histologically assessed (obtaining) by pathologist (clinical attribute) fibrosis…
“A predicted continuous score has strong advantages over a discrete score assigned by a pathologist. Specifically, the predicted score has a continuous value and thus captures more nuance than a pathologist-assigned, discrete score. The ability to assign continuous scores to the embeddings (and image data) results in higher precision and improved statistical power in downstream analyses, such as obtaining a closer association of each depicted disease state with a genetic variant or variants. For example, the severity of NASH and liver fibrosis is currently histologically assessed by pathologists through the NASH CRN and Ishak stage ordinal scores, such as the Ishak fibrosis score (0-6), the steatosis score (0-3), the lobular inflammation score (0-3), and the ballooning score (0-2). Quantitative analyses of these metrics are challenged by their low-resolution categorization of disease. The linear model can be trained to generate continuous scores from imaging data (e.g., H&E liver biopsy imaging data) that are predictive of pathologist scores. The continuous scores enable a more precise definition of disease progression, empowering longitudinal expression analysis and genetic association studies.” [0198]
computing a clinical risk score using a clinical model, the clinical model trained using one or more subject attributes; and
Generate (computing) using trained linear model and scores…
“A predicted continuous score has strong advantages over a discrete score assigned by a pathologist. Specifically, the predicted score has a continuous value and thus captures more nuance than a pathologist-assigned, discrete score. The ability to assign continuous scores to the embeddings (and image data) results in higher precision and improved statistical power in downstream analyses, such as obtaining a closer association of each depicted disease state with a genetic variant or variants. For example, the severity of NASH and liver fibrosis is currently histologically assessed by pathologists through the NASH CRN and Ishak stage ordinal scores, such as the Ishak fibrosis score (0-6), the steatosis score (0-3), the lobular inflammation score (0-3), and the ballooning score (0-2). Quantitative analyses of these metrics are challenged by their low-resolution categorization of disease. The linear model can be trained to generate continuous scores from imaging data (e.g., H&E liver biopsy imaging data) that are predictive of pathologist scores. The continuous scores enable a more precise definition of disease progression, empowering longitudinal expression analysis and genetic association studies.” [0198]
computing a final risk score for the subject from the Al risk score and the clinical risk score,
Risk score and likelihood of disease…
“In some embodiments, the data associated with a medical image can include genetic data of the subject from whom the biopsy sample is taken. For example, the data can include the subject's genetic information related to a plurality of genetic variants (e.g., 100,000 variants, 1 million variants, 10 million variants). For example, the data can indicate whether the subject has each of the plurality of genetic variants. For example, for a genetic variant with two alleles in the population, the medical image may be associated with a genetic variant value of 0, 1, or 2 depending on if the individual has 0, 1, or 2 copies of the least frequent allele. In some embodiments, the genetic data of the subject may be a polygenic risk score, indicating a likelihood of the subject incurring a disease of interest.” [0298]
Non-linear to predict score from image data using neural network (artificial intelligence)….
“In contrast, a supervised model (e.g., Yr1) refers to a non-linear machine-learning model (e.g., a neural network) configured to receive imaging data and predict a medical diagnosis score. Linear regression models, such as the first four models, are more computationally efficient to train and to apply than the supervised model. As shown in FIG. 14, linear models generated based on embeddings can provide a similar, and in some cases superior, predictive power than supervised machine-learning model, while requiring significantly less resources and time to train and to apply.” [0352] Inherent with similar predictive power than supervised model is that the AI models have been and can be used.
Obtain scores over time…
“In some embodiments, determining the plurality of placebo progression scores and the plurality of treatment progression scores comprises: determining, for each subject in the placebo group, a slope of a linear model fitted at least based on a baseline placebo score and a follow-up placebo score of the subject in the placebo group; and determining, for each subject in the treatment group, a slope of a linear model fitted at least based on a baseline placebo score and a follow-up placebo score of the subject in the treatment group. For example, for a patient, the system can obtain the patient's medical diagnosis scores over time (including the baseline score and the follow-up score) and fit a linear model configured to receive a dosage (or time of treatment) and predict a medical diagnosis score. The progression score for the patient can be the slope of the linear model.” [0417]
Outputs a score or end point (final score)…
“If a specific cluster is associated with progression, the system can then identify genetic and phenotypic biomarkers of that cluster (for example by using the linear model testing procedure described with respect to model 316 in FIG. 3B to test for association between cluster identity and genetics, expression, lab values, etc.). For example, the system fits a model that receives a cluster binary indicator (e.g., patient in cluster coded as 1, not in cluster coded as 0) and outputs a clinical score or end point (e.g., disease progression). The clinical end point can be quantified as a progression score or the clinical endpoint monitored in the clinical trial (e.g., whether patients have a higher or lower fibrosis score based on pathologist assessment).” [0452]
wherein the final risk score quantifies the prognosis of the cancer patient.
“FIG. 14 illustrates the performance of the various linear models configured to predict medical diagnosis scores, in accordance with some embodiments. As shown, five models are trained to: predict fibrosis scores; predict steatosis scores; predict lobular inflammation scores; and predict hepatocyte ballooning scores. For each score type, the colors of the bars are in the same order as the colors in the legend.” [0350]
Predict diagnosis score (quantifies prognosis)…
“In contrast, a supervised model (e.g., Yr1) refers to a non-linear machine-learning model (e.g., a neural network) configured to receive imaging data and predict a medical diagnosis score. Linear regression models, such as the first four models, are more computationally efficient to train and to apply than the supervised model. As shown in FIG. 14, linear models generated based on embeddings can provide a similar, and in some cases superior, predictive power than supervised machine-learning model, while requiring significantly less resources and time to train and to apply.” [0352]
Regarding claims 46 and 55
(claim 46) The method of claim 44, wherein the digital histology image is a whole slide image (WSI).
Casale et al. teaches:
Image from each slide (therefore whole slide image)…
“With reference to FIG. 2, an exemplary system (e.g., one or more electronic devices) can obtain a plurality of medical images obtained from a group of clinical subjects. The medical images depict a state of a disease of interest. In some embodiments, the plurality of medical images comprises a plurality of biopsy images of biopsy samples from the group of clinical subjects. For example, in a biopsy, one or more tissue slides can be obtained from a subject, and one or more digital images can be taken to capture each tissue slide.” [0237]
Regarding claims 47 and 56
(claim 47) The method of claim 44, wherein the histology section has been stained with a dye.
Casale et al. teaches:
Disease of interest is stained using Hematoxylin & eosin (dye)…
“In some embodiments, the discovery platform comprises a plurality of stages. At the first stage, an exemplary system (e.g., one or more electronic devices) generates embeddings based on medical imaging data related to a phenotype of interest such as a disease of interest. An embedding is a mapping of a variable to a vector (an array of numbers). As described herein, an embedding refers to a vector representation of a phenotypic state relative to the disease of interest reflected in the medical imaging data. The embedding captures rich semantic information of the medical imaging data (e.g., features of the microscopic structure of tissues reflected in the image), while excluding information that is not relevant to downstream analyses (e.g., orientation of the image). In an exemplary implementation, the disease of interest is non-alcoholic steatohepatitis (NASH) and the medical images are from hematoxylin & eosin (H&E) stained liver biopsies from a number of clinical trials. The resulting unsupervised embeddings can enable target identification, cross-clinical trial analysis, and enhance interpretability, as described herein.” [0192]
Regarding claims 48 and 57
(claim 48) The method of claim 47, wherein the dye is hematoxylin and eosin (H&E).
Casale et al. teaches:
Disease of interest is stained using Hematoxylin & eosin (dye)…
“In some embodiments, the discovery platform comprises a plurality of stages. At the first stage, an exemplary system (e.g., one or more electronic devices) generates embeddings based on medical imaging data related to a phenotype of interest such as a disease of interest. An embedding is a mapping of a variable to a vector (an array of numbers). As described herein, an embedding refers to a vector representation of a phenotypic state relative to