DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 10-11 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 10 recites the limitation " wherein the one or more machine-learning models were trained by" in line 1. There is insufficient antecedent basis for this limitation in the claim. The one or more machine-learning models were introduced in claim 8 not claim 1, from which claim 10 depends
Claim 11 is rejected by virtue of dependency from claim 10.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-8, 12-14, 16, 18-22 are rejected under 35 U.S.C. 103 as being unpatentable over Coudray (US 20200184643) in view of Sue (US 20220092782).
Regarding claim 1:
Coudray discloses: a method (¶7), comprising:
generating, using one or more processors, image patch data from at least one image patch derived from an image of at least one sample from an individual (Coudray teaches obtaining histopathology whole slide images and dividing each image into nonoverlapping 215 by 512 pixel tiles for processing by convolutional neural network (¶15) using a processor (¶11));
determining, using the one or more processors, a first genomic characteristic of the at least one sample from the image patch data (Coudray teaches that the convolutional neural network processes the image tiles to predict whether genes such as STK11, EGFR, FAT1, SETBP1, KRAS, and TP53 are mutated and produces probabilities indicating whether a mutation in present or absent (¶¶ 11, 17, 51-54));
performing an assay on the at least one sample; based on the assay, determining, using one or more processors, a second genomic characteristic of the at least one sample (Coudray evaluates the image derived mutation prediction using samples having mutation status established or validated through sequencing, including next generation sequencing, and immunohistochemistry (¶¶ 47, 56-57). Performing the sequencing assay to determine EGFR mutation status would have been obvious to obtain the assay derived genomic characteristic used to evaluate the image derived prediction);
Coudray does not expressly teach generating, using the one or more processors, a score by comparing the first genomic characteristic to the second genomic characteristic; and when the score is greater than a threshold, validating, using the one or more processors, the second genomic characteristic.
However, in the same field of endeavor, Sue teaches generating, using the one or more processors, a score by comparing the first genomic characteristic to the second genomic characteristic (Sue teaches a computerized quality assurance process that compares a machine learning designation derived from an electronic image with an external designation associated with the same specimen (¶¶ 80, 82-84). Sue further teaches generating comparison result identifying similarities, differences, or degrees of similarities or differences (scores) between the machine learning designation and the external designation (¶¶ 85-87));
and when the score is greater than a threshold, validating, using the one or more processors, the second genomic characteristic (Sue teaches producing the comparison result as a match or no match or degree of agreement or disagreement, and when the comparison indicated a match, clearing and using the external designation as an authoritative designation (¶¶ 106-109, 116, FIG. 6). Sue additionally teaches applying a threshold to the degree of similarity to determine whether the results should be accepted or subjected to further review or redesignation (¶¶ 110-111). It would have been obvious to implement Sue’s match dependent clearance using a numerical agreement score and validating the external designation when the score exceeds a threshold because this numerical representation predictably implements the disclosed match dependent clearance decision).
Therefore, it would have been obvious to a person of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified Coudray to incorporate the teachings of Sue by including: the comparison and quality assurance process in order to confirm findings, identify discrepancies, flag possible errors, and determine whether further review is needed as taught by Sue in ¶¶ 85-87, 109-111. Implementing the comparison as a numerical agreement score with an above threshold validation decision would preserve Sue’s disclosed clearance of matching external designations and predictably produce a quality assured assay derived genomic determination
Regarding claim 2:
Coudray further teaches: wherein the image comprises a whole-slide image (WSI) (Coudray ¶¶ 6-7, and 15).
Regarding claim 3:
Coudray further teaches: receiving, using the one or more processors, the image, wherein the image comprises an image of a tissue sample (Coudray teaches receiving image data of a region of interest (¶¶ 8, 64). Coudray further teaches that the received images are histopathology images of tissue formalin-fixed paraffin-embedded tissues, frozen tissues, and biopsies (¶¶ 6-8, 15, 33-34; FIGS. 1A-1B)).
Regarding claim 4:
Coudray further teaches: wherein the image comprises a plurality of patches, and wherein each patch of the plurality of patches comprises a plurality of pixels corresponding to one or more regions of the image (Coudray teaches dividing each whole slide image into a plurality of nonoverlapping 512 by 512 pixel tiles or patches to process by a convolutional neural network (¶¶ 15, 33-34; FIG. 1Biii; claims 7 and 13)).
Regarding claim 5:
Coudray further teaches: wherein the image comprises a histological stain image, a fluorescence in situ hybridization (FISH) image, an immunofluorescence (IF) image, or a hematoxylin and eosin (H&E) image (¶¶ 19, 33, 35; FIG. 5).
Regarding claim 6:
Coudray further teaches: wherein the first genomic characteristic of the at least one sample comprises a phenotypic call (Coudray teaches analyzing each image window to identify phenotypic information and using that phenotypic information to predict genotypic information (¶ 11). Coudray explains that its phenotype to determine a genotype or gene mutation (¶ 34). The system produces image derived classifications and probabilities indicating whether a particular mutation associated phenotype is present or absent (¶ 17, 42, 46, 54-55)).
Regarding claim 7:
Coudray further teaches: wherein the second genomic characteristic of the at least one sample comprises a genotypic call (Coudray teaches determining whether a sample is mutation positive or wild type based on molecular testing including next generation sequencing. Coudray evaluates whole slide images from samples having known EGFR mutational status, including samples whose mutation status was established or validated using sequencing (¶ 17, 47, 56-57)).
Regarding claim 8:
Coudray further teaches: wherein determining the first genomic characteristic of the at least one sample further comprises:
receiving, using the one or more processors, the image of the at least one sample (Coudray teaches receiving image data regarding a region of interest (¶ 11));
segmenting, using the one or more processors, the image into a plurality of patches (Coudray divides each whole slide image into numerous nonoverlapping 512 by 512 pixel tiles obtained from corresponding patches of the whole slide image (¶¶ 15, 38, 40, FIG. 1Biii));
inputting, using the one or more processors, the image patch data from the at least one image patch of the plurality of patches into one or more machine-learning models trained to generate a prediction of a genotype of the at least one sample based on the image patch data (Coudray trains, validates, and tests an inception v3 convolutional neural network using the image tiles and trains the network to predict the mutational status of genes including STK11, EFGR, FAT1, SETBP1, KRAS, and TP53 from tiles data (¶¶ 6-7, 15, 34, 38, 40-41)); and
outputting, using the one or more processors, the prediction of the genotype of the at least one sample (Coudray outputs for each tile or slide a probability indicating whether a particular gene mutation is present or absent and classifies the sample as mutated or wild type based on that probability (¶¶ 17, 45-46, 54-55)).
Regarding claim 12:
Sue further teaches: wherein validating the second genomic characteristic comprises determining a match between the first genomic characteristic and the second genomic characteristic (Sue teaches comparing a machine learning designation generated from an electronic specimen image with an external designation associated with the same specimen (¶¶ 82-85). Sue teaches outputting the comparison as a match/no match, or degree of agreement (¶¶ 106-109, 116). When a match is determined, the external designation may be cleared and used as an authoritative designation (¶ 109)).
Regarding claim 13:
Coudray further teaches: wherein the first genomic characteristic comprises a phenotypic call and the second genomic characteristic comprises a genotypic call (Coudray analyzes histopathology image patches to identify phenotypic information and uses that information to predict mutation status (¶¶ 11, 34)), the method further comprising:
determining, using the one or more processors, a first indication of whether the phenotypic call is phenotype positive or phenotype negative (Coudray teaches the image model produces a mutation probability and classifies the image derived result as mutation positive when the probability is at least 0.5 or wild type when the probability is below 0.5 (¶¶ 17, 45-46, 54-55));
determining, using the one or more processors, a second indication of whether the genotype call is genotype positive or genotype negative (Coudray teaches that sequencing establishes whether a sample contains a specified gene mutation or is wild type, including whether the sample is EGFR mutant or EGFR wild type (¶¶ 47, 56-57)); and
generating, using the one or more processors, the score based on the first indication and the second indication (Sue teaches generating a comparison result based on the machine learning designation and the external designation. The comparison may identify similarities, differences, or degrees of similarity or difference and may be expressed as a match or no match, or degree of agreement (¶¶ 84-87, 106-109, and 116)).
Regarding claim 14:
wherein the phenotypic call is phenotype negative and the genotype call is genotype positive, the method further comprising: determining, using the one or more processors, that the genotype call is invalid (Coudray teaches that its image based model may produce a mutation negative or wild type indication, while sequencing of the same sample may establish that the mutation is present, thereby producing a genotype positive indication (¶¶ 17, 45-47, 54-57). Sue teaches comparing an image derived machine learning designation with an external designation and identifying when the designation contradicts or do not match (¶¶ 84-87, 109-110). When a mismatch is detected, Sue generates a notification that the external designation does not match the machine learning designation and may initiate further review or redesignation when the degree of disagreement satisfies a threshold (¶¶ 110-111). It would have been obvious to one ordinary skill to utilize Sue's review and redesignation step to determine that the genotype call is invalid, because a secondary review of the external sequencing data (as prompted by Sue) would reveal processing errors, sample contamination, or artifacts in the sequencing pipeline, thereby confirming that the phenotypic image call was correct and the genotype call was invalid).
Regarding claim 16:
wherein determining that the genotype call is invalid comprises determining that the genotype call is to be further analyzed or reevaluated (Sue teaches that a mismatch between the machine learning designation and the external designation may cause the external designation to be subjected to further review or redesignation (¶¶ 85-87, 110-111)).
Regarding claim 18:
wherein the phenotypic call is phenotype negative and the genotype call is genotype negative, or wherein the phenotypic call is phenotype positive and the genotype call is genotype positive, the method further comprising: generating, using the one or more processors, an indication that the genotype call is valid (Coudray teaches image derived and sequencing derived positive or negative mutation determinations (¶¶ 17, 45-47, 54-57). Sue teaches comparing the image based machine learning designation with the external designation and outputting a match/no match or degree of agreement (¶¶ 84-87, 106-109, 116). When a match is determined, the external designation may be cleared and used as an authoritative designation (¶ 109)).
Regarding claim 19:
wherein validating the second genomic characteristic comprises validating an indication of a genetic biomarker of the at least one sample (Coudray identifies gens alternations such as STK11, EFGR, FAT1, SETBP1, KRAS, and TP53 mutations as molecular or genetic biomarkers relevant to cancer diagnosis and treatment (¶¶ 5-6, 12). Coudray further teaches determining the mutation status of such biomarker through sequencing and comparing that assay established mutation status with the mutation status predicted from histopathology images (¶¶17, 54-57). Sue teaches validating an external designation by comparing it with an image derived machine learning designation and, when the comparison indicates a match, clearing and using the external designation as authoritative (¶¶ 84-47, 106-109)).
Regarding claim 20:
wherein the genetic biomarker of the at least one sample comprises an epidermal growth factor receptor (EFGR) gene alteration, an anaplastic lymphoma kinase (ALK) gene alteration, an ROS-1 gene alteration, a tumor gene mutation burden (TMB), neurotrophic tyrosine receptor kinase 3 (NTRK3) gene alteration, a fibroblast growth factor receptor 2 (FGFR2) gene alteration, mesenchymal-epithelial transition (MET) gene alteration, phosphatidylinositol-4,5-bisphosphate 3-Kinase catalytic subunit alpha (PIK3CA) gene alteration, or one or more neurotrophic tyrosine receptor kinase (NTRK) genes 1/2/3 (Coudray identifies gens alternations such as STK11, EFGR, FAT1, SETBP1, KRAS, and TP53 mutations as molecular or genetic biomarkers relevant to cancer diagnosis and treatment (¶¶ 5-6, 12, 17, 56-57)).
Regarding claims 21-22: the claims limitations are similar to those of claim 1; therefore, rejected in the same manner as applied above. Coudray further teaches a system and a CRM in FIG. 15 and ¶ 62.
Claim(s) 9-11 are rejected under 35 U.S.C. 103 as being unpatentable over Coudray (US 20200184643) in view of Sue (US 20220092782) and Jain (US 20210193323).
Regarding claim 9:
wherein determining the first genomic characteristic of the at least one sample further comprises:
receiving, using the one or more processors, the image of the at least one sample; segmenting, using the one or more processors, the image into a plurality of patches (Coudray teaches receiving an image of a tissue sample and segmenting the image into a plurality of image patches for processing by a machine learning model (¶¶38 and 40));
Coudray in view of Sue does not teach: inputting, using the one or more processors, the image patch data from the at least one image patch of the plurality of patches into one or more machine-learning models trained to generate a prediction of a treatment response of the individual based on the image patch data; and outputting, using the one or more processors, the prediction of the treatment response.
However, in the same field of endeavor, Jain teaches: inputting, using the one or more processors, the image patch data from the at least one image patch of the plurality of patches into one or more machine-learning models trained to generate a prediction of a treatment response of the individual based on the image patch data (Jain teaches receiving tissue image slides, dividing the slides into patches, and training artificial intelligence models using image data associated with known patient outcomes (¶¶9 and 66-69).
outputting, using the one or more processors, the prediction of the treatment response (Jain teaches generating patch level scores using a supervised deep learning model trained based on known patient outcomes and combining the patch level scores to generate and output a prediction of the patient s treatment response or outcome (¶¶94, 96, and 99-100; Fig. 8)).
Therefore, it would have been obvious to a person of ordinary skill in the art prior to the effective filing date of the claimed invention to have modified Coudray in view Sue generate a prediction of treatment response as taught by Jain. Jain teaches that morphological patterns appearing in tissue image patches are correlated with whether a patient responds to treatment and can therefore be used to distinguish responders from non-responders and facilitate selection of an appropriate treatment (Jain, ¶¶65 69). The modification would have amounted to applying Jain s known treatment response prediction technique to Coudray in view Sue’s analysis of histopathology image patches, yielding the predictable result of outputting a treatment response.
Regarding claim 10:
wherein the one or more machine-learning models were trained by:
receiving, by the one or more processors, a training image of a tissue sample (Coudray teaches receiving whole slide tissue images for training and dividing the images into training, validation, and testing sets (¶40));
segmenting, by the one or more processors, the training image into a second plurality of patches (Coudray teaches segmenting or tiling each training image into a plurality of image patches (¶¶38 and 40));
inputting, using the one or more processors, second image patch data from at least one image patch of the second plurality of patches into one or more machine-learning models to generate a prediction of a genotype of the tissue sample based on the second image patch data (Coudray teaches labeling the slides and tiles using information obtained from molecular testing or staining and training a neural network using the labeled image tiles (¶38); and inputting the image tiles into a machine learning model trained to generate a prediction of the mutational status (genotype of the tissue sample) (¶41 and Fig. 5)); and
updating, using the one or more processors, the one or more machine-learning models based on a comparison of the prediction of the genotype of the tissue sample and a genotype of the tissue sample determined based on an assay performed on the tissue sample (Jain teaches a supervised training procedure in which the model’s predicted output is compared with known clinical outcome data and, when the predicted output does not match the known result, the weights of the artificial intelligence model are modified to produce an output closer to the known result (¶¶12, 22, and 94; claim 4)).
Regarding claim 11:
wherein each patch of the second plurality of patches comprises a plurality of pixels corresponding to one or more regions of the training image (Coudray teaches training the neural network using nonoverlapping 512 by 512 pixel tiles or patches obtained from corresponding regions of the whole slide training images (¶40 and Fig. 1D)).
Allowable Subject Matter
Claims 15 and 17 objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Yip (US 20220101519) teaches a generalizable and interpretable deep learning model for predicting biomarker status and biomarker metrics from histopathology slide images.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WASSIM MAHROUKA whose telephone number is (571)272-2945. The examiner can normally be reached Monday-Thursday 8:00-5:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Stephen Koziol can be reached at (408) 918-7630. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WASSIM MAHROUKA/Primary Examiner, Art Unit 2665
