DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Claim 42 has been newly added.
Claims 21-42 are still pending for consideration.
Response to Arguments
Applicant’s arguments, see “Remarks” filed April 23, 2026 have been fully considered and are persuasive. The rejection previously based on Levy in view of Drake is withdrawn because, after reconsideration of Applicant’s arguments, the prior art of record is not adequately establishing the disputed image-based multi-analyte prediction limitation. The after final response exposed a material weakness in the current Levy/Drake rejection on the limitation requiring prediction of activities of a plurality of molecular analytes from a medical image from the patient without using data of multiple molecular analytes of the patient during model inference. Upon further consideration, examiner has identified newly applied prior art that is more directly teaches prediction of RNA-Seq profiles from whole slide images alone, a learned transcriptomic representation, and multi-gene output from a single model. Accordingly, finality of the office action mailed is withdrawn, persecution is reopened, and the following new ground of rejection is set forth in this non-final Office action.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claims 21- 29, 32-37, and 41 are rejected under 35 U.S.C. 103 as being unpatentable over Schmauch et al. NPL “A deep learning model to predict RNA-Seq expression of tumours from whole slide images” in view of Drake et al. (US 20210210205 A1).
Regarding claim 21, Schmauch et al. teaches implemented using a computer system comprising one or more processors, a memory, and one or more programs stored in the memory (see Abstract; “Deep learning methods for digital pathology analysis are an effective way to address multiple clinical questions, from diagnosis to prediction of treatment outcomes” Note: deep-learning system implemented on computers); the method comprising: receiving, by the compute system, a medical image from the patient; inputting, by the compute system, the medical image from the patient into a machine learning model (see page 13, right col. 3rd para; “Data for MSI prediction. We used the histology images for n=465 patients with colorectal carcinoma (TCGA-COAD and TCGA-READ) (diagnostic slides, FFPE tissue) from the TCGA dataset, together with the corresponding MSI status data obtained from TCGAbiolinks”, see also page 2, last para; “A deep-learning model for the prediction of gene expression. We used matched WSIs and RNA-Seq profiles from TCGA data including 8725 patients and 28 different cancer types, to develop HE2RNA”); predicting, by the computer system, one or more activities of one or more molecular analytes of the patient (see page 2, right col., 5th para; “Longer lists of genes were consistently well-predicted by HE2RNA in smaller subsets of cancer types, and we used ingenuity pathway analysis (IPA) software to identify the corresponding biological networks. We found 156 genes that were well-predicted separately in at least 12 out of 28 different cancer types” page 11, “Discussion”; “HE2RNA also correctly predicts the expression of genes involved in cancer type-specific pathways, such as fibrosis in hepatocellular carcinoma, or CHK gene expression in breast cancer. The ability of our model to detect molecular and cellular modifications within cancer cells was also confirmed by the greater prediction accuracy for defined gene signatures than for lists of random genes”), wherein the machine learning model is trained to predict activities of a plurality of molecular analytes using the medical image from the patient without using data of multiple molecular analytes of the patient during model inference (see Fig. 1, “This transcriptomic representation can be used for: (1) transcriptome prediction from images without associated RNA sequencing”, Abstract; “a model based on the integration of multiple data modes, can be trained to systematically predict RNA-Seq profiles from whole-slide images alone”, see also page 12, left col. 1st para; “our model was able to predict the spatial expression of various genes from the H&E slide alone… the expression levels of which were well-predicted by our model, and which could represent a major tool for medical diagnosis and prediction, by providing virtual multiplexed staining for all WSIs alone”, and page 2, right col. 1st para; “A multilayer perceptron was applied to all supertiles to generate a predicted value per gene”, Note: prediction of RNA-Seq profiles from whole-slide images alone, transcriptome prediction from images without associated RNA sequencing, and predict the spatial expression of various genes corresponding to plurality of molecular analytes with image-only inference). Schmauch et al. does not specifically disclose stratifying the patient regarding the disease of interest based on the predicted one or more activities of the one or more molecular analytes of the patient.
In the same field of endeavor Drake et al. teach a method for stratifying a patient regarding a disease of interest (see para [0183]; “to generate classifiers to stratify individuals or detect disease as described herein”) and stratifying the patient regarding the disease of interest based on the predicted one or more activities of the one or more molecular analytes of the patient (see para [0016]; “methods of incorporating machine learning approaches with analyte data sets to develop and refine classifiers for use in stratifying individual populations and detecting disease such as cancer”). Accordingly, it would have been obvious to one of ordinary skill in the before the effective filing date of the claimed invention to modify A deep learning model to predict RNA-Seq expression of tumours from whole slide images of Schmauch et al. in view of the use a method for a clinical prediction based on molecular and biopsy data using a machine learning model of Drake et al. in order accurately predict patient's response to a cancer treatment (see para [0016]).
Regarding claim 22, the rejection of claim 21 is incorporated herein.
Schmauch et al. in the combination further teach wherein the predicted one or more activities of the one or more molecular analytes comprise: gene expression data; copy number amplification (CNA) data; chromatin accessibility data; DNA methylation data; histone modification; RNA data; protein data; spatial biology data; whole-genome sequencing (WGS) data; somatic mutation data; germline mutation data; genetic sequence data; or any combinations thereof (see Abstract; “We show that HE2RNA, a model based on the integration of multiple data modes, can be trained to systematically predict RNA-Seq profiles from whole-slide images alone, without expert annotation. Through its interpretable design, HE2RNA provides virtual spatialization of gene expression”, see also page 8, right col., 1st para; “The analysis of gene expression prediction, restricted to MSI-H patients from TCGA-COAD (81 samples), revealed that a surprisingly high number of genes were sig nificantly well-predicted by HE2RNA on this subset (1027 genes well-predicted under HS correction)”, see also page3, 2nd para “We found that, in 50% of cancer types for angiogenesis, and 54% for hypoxia, DNA repair, and cell-cycle pathways, signatures were significantly better predicted by HE2RNA than random lists of genes”).
Regarding claim 23, the rejection of claim 21 is incorporated herein.
Drake et al. in the combination further teach wherein the predicted one or more activities of the one or more molecular analytes comprise: a gene expression value (see para [0110]; “FIGS. 36A-36G show cfDNA derived V-plots around TSS regions used to predict gene expression”) comprising an abundance of a transcript; a copy number amplification value; an amplification signature value; abundance of one or more histone modifications comprising a ChIP-seq value abundance of one or more mRNA sequences (see para [0249]; “the measured values relate to the abundance for these cfRNAs. Their transcripts are of a certain size, and each transcript is stored, and the number of cfRNAs found for each can be counted”); abundance of one or more proteins; the presence of one or more somatic mutations; the presence of one or more germline mutations; the presence or absence of one or more specific DNA methylation marks in one or more specific genomic regions; or any combinations thereof (see para [0144]; “Four highly informative classes of molecular biomarkers include: 1) genomic biomarkers based on the analysis of DNA profiles, sequences or modifications; 2) transcriptomic biomarkers based on the analysis of RNA expression profiles, sequences or modifications; 3) proteomic or protein biomarkers based on the analysis of protein profiles, sequences or modifications and 4) metabolomic biomarkers based on the analysis of metabolites abundance”).
Regarding claim 24, the rejection of claim 21 is incorporated herein.
Schmauch et al. in the combination further teach wherein stratifying the patient comprises predicting a patient outcome of the patient based on the plurality of molecular analytes (see Abstract; “The transcriptomic representation learned by HE2RNA can also be transferred on other datasets, even of small size, to increase prediction performance for specific molecular phenotypes”, see also page 2, right col. 5th para; “We found 156 genes that were well-predicted separately in at least 12 out of 28 different cancer types”).
Regarding claim 25, the rejection of claim 24 is incorporated herein.
Schmauch et al. in the combination further teach wherein the patient outcome comprises: a response to treatment, a mortality, a survival, a disease diagnosis, a disease progression, a disease prognosis, a disease risk, or any combinations thereof (see page 2, left col. 1st para; “These tools emerge as exciting opportunities in the clinical and biomedical field7, ultimately improving the prediction of patient survival outcomes and response to treatment…. Predicting gene expression from WSIs would greatly facilitate patient diagnosis and prediction of response to treatment and survival outcome”).
Regarding claim 26, the rejection of claim 21 is incorporated herein.
Schmauch et al. in the combination further teach wherein the disease of interest comprises a cancer, an immune disease, or a fibrosis-associated disease (see page 2, right col. 4th para; “We compared the list of genes well-predicted in each cancer to analyze the consistency of the predictions”).
Regarding claim 27, the rejection of claim 21 is incorporated herein.
Drake et al. in the combination further teach the one or more programs further comprising instructions for: identifying a therapy for the patient (see para [0330]; “After such a classification, treatment may be provided to the subject. Example treatment regimens can include surgical intervention, chemotherapy with a given drug or drug combination, and/or radiation therapy”).
Regarding claim 28, the rejection of claim 27 is incorporated herein.
Drake et al. in the combination further teach wherein the one or more molecular analytes are indicative of an activity level of a target of the identified therapy or a responsiveness to the identified therapy (see para [0378]; “a method for determining the efficacy of a drug designed to treat a disease class, comprising obtaining a sample from an individual having the disease class; subjecting the sample to the drug; assessing the drug-exposed sample for the level of gene expression for at least one gene”).
Regarding claim 29, the rejection of claim 28 is incorporated herein.
Drake et al. in the combination further teach wherein the target of the identified therapy comprises an mRNA or a protein (see para [0423]; “For RNA, assays detecting mRNA or short RNAs can be applied. For proteins, enzyme-linked immunosorbent assay (ELISA) can be used”).
Regarding claim 32, the rejection of claim 27 is incorporated herein.
Drake et al. in the combination further teach wherein the therapy comprises an antibody or bi-specific antibody therapy (see [0153]; “In cancer-patients serum-antibody profiles change, as well as autoantibodies against the cancerous tissue are generated”).
Regarding claim 33, the rejection of claim 29 is incorporated herein.
Drake et al. in the combination further teach wherein the therapy comprises an antibody or bi-specific antibody therapy (see [0153]; “In cancer-patients serum-antibody profiles change, as well as autoantibodies against the cancerous tissue are generated”).
Regarding claim 34, the rejection of claim 21 is incorporated herein.
Drake et al. in the combination further teach the one or more programs further comprising instructions for: identifying a therapeutic that inhibits a gene for the patient (see para [0383]; “the systems and methods described herein that relate to classifying a population based on treatment responsiveness refer to cancers that are treated with chemotherapeutic agents of the classes DNA damaging agents, DNA repair target therapies, inhibitors of DNA damage signaling, inhibitors of DNA damage induced cell cycle arrest and inhibition of processes indirectly leading to DNA damage, but not limited to these classes”).
Regarding claim 35, the rejection of claim 21 is incorporated herein.
Drake et al. in the combination further teach wherein the medical image comprises: one or more histopathology images; one or more magnetic resonance imaging (MRI) images; one or more computerized tomography (CT) scans; or any combination thereof (see para [0352]; “This clinical test may comprise an imaging test, a blood test, a computed tomography (CT) scan, a magnetic resonance imaging (MRI) scan, an ultrasound scan, a chest X-ray, a positron emission tomography (PET) scan, a PET-CT scan, or any combination thereof”).
Regarding claim 36, the rejection of claim 21 is incorporated herein.
Drake et al. in the combination further teach wherein the medical image is unlabeled (see para [0299]; “the computer processing method is an unsupervised machine learning method including, for example, clustering, network, principal component analysis, and matrix factorization” Note: unsupervised machine learning method implies unlabeled image).
Regarding claim 37, the rejection of claim 21 is incorporated herein.
Schmauch et al. in the combination further teach wherein the machine learning model comprises a first module that is trained to receive as input the medical image and provide as output an embedding (see page 12, left col. 2nd para; “CNNs for image recognition make use of an internal representation of the original data that they infer. The features of this latent space encode the statistics of natural images and the information of importance for image recognition. Similarly, the internal transcriptomic representation, learned by HE2RNA during the prediction of RNA-Seq data, may constitute an important step toward understanding the biological descriptors required for clinical classification problems and the link between the information contained at the tissue and molecular levels. We have shown that the lower-dimensional transcriptomic representation learned during the RNA-Seq prediction task can be very powerful when transferred to other datasets used for a different task).
Regarding claim 39, the scope of claim 39 is fully encompassed by the scope of claim 21, accordingly, the rejection of claim 21 is fully applicable here (see also Drake et al. [0050]: “a system comprising one or more computer processors and computer memory coupled thereto. The computer memory comprises machine-executable code that, upon execution by the one or more computer processors, implements any of the methods above or elsewhere herein”).
Regarding claim 40, the scope of claim 40 is fully encompassed by the scope of claim 21, accordingly, the rejection of claim 21 is fully applicable here (see also Drake et al. [0617]; “The software code may be stored as a series of instructions or commands on a computer readable medium for storage and/or transmission. A suitable non-transitory computer readable medium can include random access memory (RAM)”).
Regarding claim 41, the rejection of claim 21 is incorporated herein.
Schmauch et al. in the combination further teach wherein the machine learning model is trained to predict activities of the plurality of molecular analytes without using any molecular analyte data of the patient during model inference (see Fig. 1, “This transcriptomic representation can be used for: (1) transcriptome prediction from images without associated RNA sequencing”, Abstract; “a model based on the integration of multiple data modes, can be trained to systematically predict RNA-Seq profiles from whole-slide images alone”, see also page 12, left col. 1st para; “our model was able to predict the spatial expression of various genes from the H&E slide alone… the expression levels of which were well-predicted by our model, and which could represent a major tool for medical diagnosis and prediction, by providing virtual multiplexed staining for all WSIs alone”, and page 2, right col. 1st para; “A multilayer perceptron was applied to all supertiles to generate a predicted value per gene”) .
Claims 30-31 are rejected under 35 U.S.C. 103 as being unpatentable over Schmauch et al. in view of Drake et al. as applied in claims 21, 27, and 26 above, and further in view of Hegde et al. (US 20190369098 A1).
Regarding claim 30, the rejection of claim 27 is incorporated herein.
The combination of Schmauch et al. and Drake et al. as a whole does not specifically teach wherein the therapy comprises an antibody-drug conjugate (ADC) therapy.
In the same field of endeavor, Hegde et al. teach wherein the therapy comprises an antibody-drug conjugate (ADC) therapy (see para [0512]; “an immunoconjugate is an antibody-drug conjugate (ADC) in which an antibody is conjugated to one or more drugs”). Accordingly, it would have been obvious to one of ordinary skill in the before the effective filing date of the claimed invention to modify a deep learning model to predict RNA-Seq expression of tumours from whole slide images of Schmauch et al. in view of the use a method for a clinical prediction based on molecular and biopsy data using a machine learning model of Drake et al. and diagnostic methods, therapeutic methods, and compositions for the treatment of cancer of Hegde et al. in order to increase therapeutic efficacy of the conjugate in killing the cancer cell to which it binds (see para [0512]).
Regarding claim 31, the rejection of claim 29 is incorporated herein.
Hegde et al. in the combination further teach wherein the therapy comprises an ADC therapy (see Hegde et al. para [0512]; “an immunoconjugate is an antibody-drug conjugate (ADC) in which an antibody is conjugated to one or more drugs”). Accordingly, it would have been obvious to one of ordinary skill in the before the effective filing date of the claimed invention to modify a deep learning model to predict RNA-Seq expression of tumours from whole slide images of Schmauch et al. in view of the use a method for a clinical prediction based on molecular and biopsy data using a machine learning model of Drake et al. and diagnostic methods, therapeutic methods, and compositions for the treatment of cancer of Hegde et al. in order to increase therapeutic efficacy of the conjugate in killing the cancer cell to which it binds (see para [0512]).
Claims 38 and 42 are rejected under 35 U.S.C. 103 as being unpatentable over Schmauch et al. in view of Drake et al. as applied in claims 21, and 37 above, and further in view of Cheng et al. (US 20230377353 A1).
Regarding claim 38, the rejection of claim 37 is incorporated herein.
Schmauch et al. in the combination further teach and provide as output the activities of the plurality of molecular analytes (see page 2, right col., 5th para; “Longer lists of genes were consistently well-predicted by HE2RNA in smaller subsets of cancer types, and we used ingenuity pathway analysis (IPA) software to identify the corresponding biological networks. We found 156 genes that were well-predicted separately in at least 12 out of 28 different cancer types” page 11, “Discussion”; “HE2RNA also correctly predicts the expression of genes involved in cancer type-specific pathways, such as fibrosis in hepatocellular carcinoma, or CHK gene expression in breast cancer. The ability of our model to detect molecular and cellular modifications within cancer cells was also confirmed by the greater prediction accuracy for defined gene signatures than for lists of random genes”). However, the combination of Schmauch et al. and Drake et al. as a whole does not specifically teach and wherein the second module comprises a plurality of heads.
Cheng et al. teaches wherein the machine learning model comprises a second module that is trained to receive as input the embedding, wherein the second module comprises a plurality of heads (see para [0103]; “at step 525, the latent visual embeddings are input into a second module for mapping each visual feature representation into different attention-weighted features through multi-head attentions”). Accordingly, it would have been obvious to one of ordinary skill in the before the effective filing date of the claimed invention to modify a deep learning model to predict RNA-Seq expression of tumours from whole slide images of Schmauch et al. in view of the use a method for a clinical prediction based on molecular and biopsy data using a machine learning model of Drake et al. and a method for performing an automated tagging of features in digital micrographs representing slides with tissue samples of Cheng et al. in order to identify archived whole slide images by matching one or more tag attributes (see para [0103]).
Regarding claim 42, the rejection of claim 21 is incorporated herein.
Schmauch et al in the combination further teach an activity of a respective molecular analyte of the plurality of molecular analytes from the medical image (see page 2, right col., 5th para; “Longer lists of genes were consistently well-predicted by HE2RNA in smaller subsets of cancer types, and we used ingenuity pathway analysis (IPA) software to identify the corresponding biological networks. We found 156 genes that were well-predicted separately in at least 12 out of 28 different cancer types” page 11, “Discussion”; “HE2RNA also correctly predicts the expression of genes involved in cancer type-specific pathways, such as fibrosis in hepatocellular carcinoma, or CHK gene expression in breast cancer. The ability of our model to detect molecular and cellular modifications within cancer cells was also confirmed by the greater prediction accuracy for defined gene signatures than for lists of random genes”).
Cheng et al. in the combination further teach wherein the machine learning model comprises a plurality of prediction heads, each prediction head of the plurality of prediction heads configured to predict (see para [0003]; “a machine learning model to predict multiple slide-level tags from the plurality of non-background patches…. mapping each visual feature representation into different attention-weighted features through multi-head attentions…… a multi-tag attention module configured to construct slide-level tags identifying tag classifications by providing the weighted patch features to a plurality of second attention aggregation mechanisms, one per output head, each performing operations… comprising: mapping each weighted patch feature into different attention-weighted features through multi-head attentions, and outputting a slide-level tag”, see also para [0012; “the tags are selected from the group consisting of.. gene expression”, Note: by definition in digital pathology, "molecular analytes" (e.g., expression levels of certain genes) are precisely the "tags" or classifications being predicted from the visual patterns on a WSI).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WINTA GEBRESLASSIE whose telephone number is (571)272-3475. The examiner can normally be reached Monday-Friday9:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached at 571-270-5180. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WINTA GEBRESLASSIE/Examiner, Art Unit 2677
/ANDREW W BEE/Supervisory Patent Examiner, Art Unit 2677
Prosecution Insights