DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on April 27, 2026 has been entered.
Response to Arguments
Applicant’s arguments have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claims 1, 8-11, and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Cheng et al. (US 20230377353 A1) in view of Schmauch et al. NPL “A deep learning model to predict RNA-Seq expression of tumours from whole slide images”.
Regarding claim 1, Cheng et al. teaches the system comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory (see para [0016]; “provided herein is a system comprising: at least one processor, a memory, and instructions executable by the at least one processor to create a histopathology image”) and configured to be executed by the one or more processors, the one or more programs including instructions for: receiving a medical image of the subject (see para [0016]; “an intake module configured to receive a digital whole slide image”); inputting the medical image from the subject into a first module of a machine learning model to obtain an embedding (see para [0017]; “wherein the machine learning model comprises: a visual feature extraction module configured to extract a visual feature representation of each patch”); and inputting the embedding into a second module of the machine learning model (see para [0106]; “a second module configured to apply one or more neural networks using the visual feature representations of the patches as inputs to produce a heatmap”), wherein the second module of the machine learning model comprises one or more heads (see para [0003]; “mapping each visual feature representation into different attention-weighted features through multi-head attentions”, see also para [0010]; “wherein the machine learning model operates in a multi-scale mode… a multi-tag attention module configured to construct slide-level tags identifying tag classifications …one per output head”). However, Cheng et al. does not teach a system for predicting activities of one or more molecular analytes of a subject, and to predict the activities of the one or more molecular analytes for the subject, wherein the second module of the machine learning model is configured to provide as output activities of a plurality of molecular analytes.
In the same field of endeavor, Schmauch et al. teaches a system for predicting activities of one or more molecular analytes of a subject (see Abstract; “predict RNA-Seq profiles from whole-slide images alone”) and to predict the activities of the one or more molecular analytes for the subject (see Abstract; “We show that HE2RNA, a model based on the integration of multiple data modes, can be trained to systematically predict RNA-Seq profiles from whole-slide images alone”), wherein the second module of the machine learning model is configured to provide as output activities of a plurality of molecular analytes (see page 8 left col. last para; “when HE2RNA transforms each WSI into a vector of P features corresponding to the dimensionality of the last hidden representation of the neural network”, see also page 11, right col. “Discussion”; “HE2RNA robustly and consistently predicted subsets of genes expressed in different cancer types, including genes involved in immune cell activation status and immune cell signaling….. HE2RNA also correctly predicts the expression of genes involved in cancer type-specific pathways, such as fibrosis in hepatocellular carcinoma, or CHK gene expression in breast cancer. The ability of our model to detect molecular and cellular modifications within cancer cells was also confirmed by the greater prediction accuracy for defined gene signatures”). Accordingly, it would have been obvious to one of ordinary skill in the before the effective filing date of the claimed invention to modify the general system for performing an automated tagging of features in digital micrographs representing slides with tissue samples of Cheng et al. in view of a deep learning model to predict RNA-Seq expression of tumours from whole slide images of Schmauch et al. in order to increase prediction performance for specific molecular phenotypes (see Abstract).
Regarding claim 8, the rejection of claim 1 is incorporated here.
Schmauch et al. in the combination further teach wherein the first module of the machine learning model has been trained using a plurality of medical images of a first cohort (see page3, Fig.1; “Hematoxylin & eosin (H&E)-stained histology slides and RNA-Seq data (FPKM UQ values) for 28 different cancer types and 8725 patients were collected from The Cancer Genome Atlas (TCGA) and used to train the neural network HE2RNAto predict transcriptomic profile from the corresponding high-definition whole-slide images (WSI)”).
Regarding claim 9, the rejection of claim 1 is incorporated here.
Cheng et al. in the combination further teach wherein the first module of the machine learning model comprises an embedding module (see para [0087]; “the visual feature extraction module takes as inputs from latent visual embeddings 412 extracted by a ResNet network 415”).
Regarding claim 10, the rejection of claim 8 is incorporated here.
Schmauch et al. in the combination further teach wherein the second module of the machine learning model has been trained using one or more molecular analyte data sets obtained from a second cohort (see page3, Fig.1; “Hematoxylin & eosin (H&E)-stained histology slides and RNA-Seq data (FPKM UQ values) for 28 different cancer types and 8725 patients were collected from The Cancer Genome Atlas (TCGA) and used to train the neural network HE2RNAto predict transcriptomic profile from the corresponding high-definition whole-slide images (WSI)”).
Regarding claim 11, the rejection of claim 10 is incorporated here.
Schmauch et al. in the combination further teach wherein the one or more molecular analyte datasets comprise: gene expression data; copy number amplification (CNA) data; chromatin accessibility data; DNA methylation data; histone modification; RNA data; protein data; spatial biology data; whole-genome sequencing (WGS) data (see page 2, “Result”; “We used matched WSIs and RNA-Seq profiles from TCGA data (https://portal.gdc.cancer.gov/), including 8725 patients and 28 different cancer types, to develop HE2RNA, a deep-learning model based on a multitask weakly supervised approach24 (architecture in the “Methods”). The model was trained to predict normalized gene expression data (logarithmic FPKM-UQ values, see “Methods”) from WSIs”); somatic mutation data; germline mutation data; or any combination thereof (see page 3, 1st para; “prediction of expression levels for genes involved in cell-cycle regulation (cell cycle: G2/M DNA damage checkpoint regulation, cell-cycle control of chromosomal replication, mitotic roles of polo-like Kinase), but also for the prediction of expression levels for CHEK2 (known to be mutated in BRCA29 and involved in its progression30) and Cyclin E (known to be overexpressed in BRCA31)”).
Regarding claim 18, the rejection of claim 1 is incorporated here.
Cheng et al. in the combination further teach wherein the medical image comprises one or more histopathology images; one or more magnetic resonance imaging (MRI) images; one or more computerized tomography (CT) scans; or any combination thereof (see para [0003]; “provided herein is a method of automatic multi-tagging of whole slide histopathology”).
Regarding claim 19, the scope of claim 19 is fully encompassed by the scope of claim 1, accordingly, the rejection of claim 1 is fully applicable here
Regarding claim 20, the scope of claim 20 is fully encompassed by the scope of claim 1, accordingly, the rejection of claim 1 is fully applicable here (see also para [0010]; “provided herein is a non-transitory computer-readable storage media encoded with a computer program including instructions executable by a processor” of Cheng et al.).
Claims 2-4 are rejected under 35 U.S.C. 103 as being unpatentable over Cheng et al. in view of Schmauch et al. as applied in clim 1 above, and further in view of Rothrock et al. (US 20230245477 A1).
Regarding claim 2, the rejection of claim 1 is incorporated here. The combination of Cheng et al. and Schmauch et al. as a whole does not teach wherein the predicted activities of the one or more molecular analytes comprise amplification signature data and/or one or more chromosome accessibility scores comprising one or more ATAC-seq peak values.
In the same field of endeavor, Rothrock et al. teach wherein the predicted activities of the one or more molecular analytes comprise amplification signature data and/or one or more chromosome accessibility scores comprising one or more ATAC-seq peak values (see para [0036]; “AI may be used to predict biomarkers (such as the overexpression of a protein and/or gene product, amplification, or mutations of specific genes) from salient regions within digital images of tissues stained using H&E and other dye-based methods”). Accordingly, it would have been obvious to one of ordinary skill in the before the effective filing date of the claimed invention to modify the general system for performing an automated tagging of features in digital micrographs representing slides with tissue samples of Cheng et al. in view of a deep learning model to predict RNA-Seq expression of tumours from whole slide images of Schmauch et al. and creating a prediction model to predict labels for prepared tissue specimens of Rothrock et al. in order to improve patient care, while also being faster and less expensive (see para [0036]).
Regarding claim 3, the rejection of claim 2 is incorporated here.
Rothrock et al. in the combination further teach wherein the amplification signature data is generated based on a plurality of differentially expressed genes with respect to an amplification and a plurality of weights (see para [0035]; “genetic testing of the tissue may be used to confirm if a biomarker is present (e.g., overexpression of a specific protein or gene product in a tumor, amplification of a given gene in a cancer”).
Regarding claim 4, the rejection of claim 2 is incorporated here.
Rothrock et al. in the combination further teach wherein the amplification signature data comprises a gene-specific copy number amplification (CNA) (see para [0035]; “SH and FISH may be employed to assess the number of copies of genes or the abundance of specific RNA molecules, depending on the type of probes employed (e.g. DNA probes for gene copy number and RNA probes for the assessment of RNA expression)”).
Claims 5, and 7 are rejected under 35 U.S.C. 103 as being unpatentable over Cheng et al. in view of Schmauch et al. as applied in clim 1 above, and further in view of Narainet al. (US 20200185063 A1).
Regarding claim 5, the rejection of claim 1 is incorporated here. The combination of Cheng et al. and Schmauch et al. as a whole does not teach wherein the one or more programs further include instructions for: using a third module of the machine learning model to determine a measure of significance or prognostic value of the one or more molecular analytes to dynamically select a subset of molecular analytes for subsequent use.
In the same field of endeavor, Narainet et al. teach wherein the one or more programs further include instructions for: using a third module of the machine learning model to determine a measure of significance or prognostic value of the one or more molecular analytes to dynamically select a subset of molecular analytes for subsequent use (see para [0014]; “wherein the machine learning penalizes possible biomarkers that are strongly correlated with other possible biomarkers and rewards possible biomarkers based on a level of correlation with the clinical outcome, thereby identifying one or more potential biomarkers for the clinical outcome. In some embodiments, the machine learning employed to analyze the possible biomarkers applies logistic regression with the elastic net penalty”). Accordingly, it would have been obvious to one of ordinary skill in the before the effective filing date of the claimed invention to modify the general system for performing an automated tagging of features in digital micrographs representing slides with tissue samples of Cheng et al. in view of a deep learning model to predict RNA-Seq expression of tumours from whole slide images of Schmauch et al. and identifying one or more potential biomarkers for a clinical outcome related to administration of an agent of Narainet et al. in order to identify biological markers to facilitate patient therapy (see para [0014]).
Regarding claim 7, the rejection of claim 5 is incorporated here.
Schmauch et al. in the combination further teach wherein the second module of the machine learning model and/or the third module of the machine learning model have been trained using transfer learning (see page 3, Fig. 1; “The Cancer Genome Atlas (TCGA) and used to train the neural network HE2RNAto predict transcriptomic profile from the corresponding high-definition whole-slide images (WSI). During this task, the neural network learned an internal representation encoding both information from tiled images and gene expression levels…..Improving predictive performances for different tasks, in a transfer learning framework, as shown here for a realistic setup, for microsatellite instability (MSI) status prediction from non-annotated WSIs. Scale bar: 5 mm”).
Claims 12, and 16-17 are rejected under 35 U.S.C. 103 as being unpatentable over Cheng et al. in view of Schmauch et al. as applied in clim 1 above, and further in view of Yip al. (US 20200258223 A1).
Regarding claim 12, the rejection of claim 11 is incorporated here. The combination of Cheng et al. and Schmauch et al. as a whole does not teach wherein the one or more molecular analyte datasets comprise: a gene expression value comprising an abundance of a transcript; copy number amplification value; an amplification signature value; abundance of one or more histone modifications comprising a ChIP-seq value; abundance of one or more mRNA sequences; abundance of one or more proteins; the presence of one or more somatic mutations; the presence of one or more germline mutations; the presence or absence of one or more specific DNA methylation marks in one or more specific genomic regions; or any combination thereof.
In the same field of endeavor, Yip et al. teach wherein the one or more molecular analyte datasets comprise: a gene expression value comprising an abundance of a transcript, a copy number amplification value; an amplification signature value; abundance of one or more histone modifications comprising a ChIP-seq value; abundance of one or more mRNA sequences; abundance of one or more proteins; the presence of one or more somatic mutations; the presence of one or more germline mutations; the presence or absence of one or more specific DNA methylation marks in one or more specific genomic regions; or any combination thereof (see para [0126]; “The molecular data 162b may include DNA sequences, RNA sequences, metabolomics data, proteomic/cytokine data….. Proteomic data includes protein composition, structure, and activity; when and where proteins are expressed; rates of protein production, degradation, and steady-state abundance”, see also para [0387]; “there are genetic mutations, currently found through genotype biomarkers, that may be correlated with tissue features”, and para [0391]; “with colorectal cancer (CRC) when training models to predict the consensus molecular subtype (CMS), the CMS class is used to guide targeted treatment, but only using genotype biomarkers, i.e., mutations in RNA data”). Accordingly, it would have been obvious to one of ordinary skill in the before the effective filing date of the claimed invention to modify the general system for performing an automated tagging of features in digital micrographs representing slides with tissue samples of Cheng et al. in view of a deep learning model to predict RNA-Seq expression of tumours from whole slide images of Schmauch et al. and deep learning model for predicting biomarker status and biomarker metrics from histopathology slide images of Yip et al. in order to hypothesize which anti-cancer treatments are likely to be successful in eliminating cancer cells from the patient's body (see para [0126]).
Regarding claim 16, the rejection of claim 1 is incorporated here.
Yip et al. in the combination further wherein the one or more programs further include instructions for: generating an annotation map of the predicted activity of the molecular analyte (see para [0245]; “predicted biomarker status and, at the block 906, an overlay map may be generated showing the predicted biomarker status for display to a clinician or for providing to a pathologist for determining a preferred immunotherapy corresponding to the predicted biomarker”); and overlaying the annotation map on the medical image (see para [0051]; “generate a report containing the digital image and a digital overlay visualizing the predicted presence of the one or more biomarkers”, see also para [0070]; “FIG. 16B displays a probability map overlaid on the H&E image”). Accordingly, it would have been obvious to one of ordinary skill in the before the effective filing date of the claimed invention to modify the general system for performing an automated tagging of features in digital micrographs representing slides with tissue samples of Cheng et al. in view of a deep learning model to predict RNA-Seq expression of tumours from whole slide images of Schmauch et al. and deep learning model for predicting biomarker status and biomarker metrics from histopathology slide images of Yip et al. in order to isolate a maximum amount of tumor or non-tumor tissue from the slide (see para [0249]).
Regarding claim 17, the rejection of claim 16 is incorporated herein.
Yip et al. in the combination further teach wherein the annotation map includes a visualization distinguishing healthy tissue from diseased tissue (see para [0253]; “the overlay map generator 324 may produce a digital overlay of a recommended cutting boundary that separates the image regions classified as tumor and the image regions classified as non-tumor”).
Claims 14-15 are rejected under 35 U.S.C. 103 as being unpatentable over Cheng et al. in view of Schmauch et al. as applied in clim 1 above, and further in view of Murezet al. (US 20190244107 A1)
Regarding claim 14, the rejection of claim 1 is incorporated here. The combination of Cheng et al. and Schmauch et al. as a whole does not teach wherein the one or more programs further include instructions for: receiving a medical image of a new subject; obtaining an embedding by providing the medical image of the new subject to the first module of the machine learning model; mapping the embedding based on domain adaptation.
In the same field of endeavor, Murezet et al. teach wherein the one or more programs further include instructions for: receiving a medical image of a new subject; obtaining an embedding by providing the medical image of the new subject to the first module of the machine learning model; mapping the embedding based on domain adaptation. (see Abstract; “The deep CNN is adapted to a new target image domain without requiring new annotations by determining domain agnostic features that map from the annotated source image domain and a target image domain to a joint latent space, and using the domain agnostic features to map the joint latent space to annotations for the target image domain”). Accordingly, it would have been obvious to one of ordinary skill in the before the effective filing date of the claimed invention to modify the general system for performing an automated tagging of features in digital micrographs representing slides with tissue samples of Cheng et al. in view of a deep learning model to predict RNA-Seq expression of tumours from whole slide images of Schmauch et al. and a system for adapting a deep convolutional neural network (CNN) of Murezet et al. in order to avoid training images to be annotated (see para [0249]).
Regarding claim 15, the rejection of claim 14 is incorporated here.
Murezet et al. in the combination further teach wherein mapping the embedding based on domain adaptation comprises: inputting the embedding obtained based on the medical image of the new subject into a fourth module of the machine learning model (see para [0009]; “The deep CNN is adapted to a new target image domain without requiring new annotations by determining domain agnostic features that map from the annotated source image domain and a target image domain to a joint latent space, and using the domain agnostic features to map the joint latent space to annotations for the target image domain”, see also para [0012]; “the joint latent space is regularized by a plurality of auxiliary networks and loss functions.”).
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Cheng et al. and Schmauch et al. in view of Narainet al. as applied in claims 1, and 5 above, and further in view of Hegde et al. (US 20190369098 A1).
Regarding claim 6, the rejection of claim 5 is incorporated herein.
Schmaush et al. in the combination further teach further teach wherein the patient is a first patient (see page 2, “Result”; “We used matched WSIs and RNA-Seq profiles from TCGA data (https://portal.gdc.cancer.gov/), including 8725 patients… We performed a five-fold cross validation, i.e., patients were randomly assigned to five different sets, and each set was used in turn as the validation set, the other four sets being used for training”), the method further comprising: predicting, using the trained first and second module of the machine learning model, activity of at least one of the subset of molecular analytes from a medical image of a second patient (see page 6, right col. last para; “we applied a model trained to predict the expression level of MKI67 (Table 2) to an independent dataset of 369 slides from 194 patients with LIHC4”). However, the combination of Cheng et al. and Schmauch et al. and Narainet al. as a whole does not teach identifying, based on the predicted activity, an Antibody-Drug Conjugate (ADC) therapy for the second patient.
In the same field of endeavor Hegde et al. teach identifying, based on the predicted activity, an Antibody-Drug Conjugate (ADC) therapy for the second patient (see para [0167]; “a sample from an individual is compared, e.g., to make a predictive, diagnostic, prognostic, and/or therapeutic determination”, see also para [0512]; “an immunoconjugate is an antibody-drug conjugate (ADC) in which an antibody is conjugated to one or more drugs”). Accordingly, it would have been obvious to one of ordinary skill in the before the effective filing date of the claimed invention to modify the general system for performing an automated tagging of features in digital micrographs representing slides with tissue samples of Cheng et al. in view of a deep learning model to predict RNA-Seq expression of tumours from whole slide images of Schmauch et al. and identifying one or more potential biomarkers for a clinical outcome related to administration of an agent of Narainet et al. and diagnostic methods, therapeutic methods, and compositions for the treatment of cancer of Hegde et al. in order to conjugated to a radioactive atom to form a radioconjugate (see para [0167]).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WINTA GEBRESLASSIE whose telephone number is (571)272-3475. The examiner can normally be reached Monday-Friday9:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached at 571-270-5180. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WINTA GEBRESLASSIE/Examiner, Art Unit 2677
Prosecution Insights