Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Applicant’s claim for the benefit of prior-filed application 63/594713 filed 10/31/2023 under 35 U.S.C. 119(e) or under 35 U.S.C. 120, 121, 365(c), or 386(c) is acknowledged.
Information Disclosure Statement
The information disclosure statements (IDSs) submitted on 10/30/2024 and 02/27/2025 are being considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim 1
Step 1 – YES
Claim 1 discloses a process, and thus falls in one of the statutory categories.
Step 2A, Prong One – YES
Claim 1 recites an abstract idea. Claim 1 recites “parsing the received first image data into a plurality of tiles, each of the plurality of tiles representing a respective portion of the received first image data; identifying, using a first trained machine learning model, at least one cytological feature for each of the plurality of tiles; determining, using a second trained machine learning model, at least one statistic based on the identified at least one cytological feature for each of the plurality of tiles.” All of these steps may be performed in the mind by one of ordinary skill in the art. Under the broadest reasonable interpretation, “parsing” image data into tiles includes mentally examining and dividing an image into tiles, but also may be performed by one of ordinary skill in the art provided with image data representing a biological sample using a physical aid. Identifying cytological feature(s) and determining a statistic based on cytological feature(s) may also be performed in the mind; for example, one of ordinary skill in the art would be able to identify mitotic cells (cytological features) and a number of mitotic cells (a statistic based on cytological features) by visual observation and judgment of image data. The cited limitations also recite first and second trained machine learning models, discussed below.
Step 2A, Prong Two – NO
Claim 1 does not recite additional elements that integrate the judicial exception into a practical application. Claim 1 recites “receiving first image data representing a biological sample” and “outputting the determined at least one statistic.” These steps are merely data collection and outputting steps, and thus amount to insignificant extrasolution activity. Although the claim recites “outputting,” the outputting is generic and does not integrate the abstract ideas of parsing image data, identifying features, and determining statistics into a practical application.
Claim 1 also recites “using a first trained machine learning model” and “using a second trained machine learning model.” These elements are recited at a high level and amount to mere instructions to “apply it” (the abstract ideas) on a computer. That is, “using” a first and second trained machine learning models are mere instructions to apply a first trained ML model to an identifying step and to apply a second trained ML model to a determining step, and lack any specifications of the ML models or their training. The claim fails to recite details of how identifying and determining are accomplished and merely invokes computers as a tool to perform an existing process of visually identifying cytological features and determining statistics of cytological features.
Step 2B – NO
Claim 1 does not recite additional elements that amount to significantly more than the judicial exception. As stated above, Claim 1 recites “receiving first image data representing a biological sample,” “outputting the determined at least one statistic,” “using a first trained machine learning model,” and “using a second trained machine learning model.” None of these elements amount to significantly more than the judicial exception, but rather amount to mere extrasolution activity and instructions to apply the abstract ideas on a computer.
Thus, Claim 1 is not eligible subject matter.
Claim 2 recites “wherein identifying the at least one cytological feature for each of the plurality of tiles includes identifying at least one of a number of mast cells, a number of mast cell nuclei, a number of mitotic figures, and a number of multinucleated cells.” However, the limitation of the identifying step may also be performed in the mind by observation, evaluation, and judgment, and thus the identifying step of Claim 2 is an abstract idea.
Claim 3 recites “wherein the determined at least one statistic includes a ratio of a total number of mitotic figures identified for the plurality of tiles to a total number of mast cells identified for the plurality of tiles.” However, the limitation of the determining step may also be performed in the mind by observation, evaluation, and judgment, and thus the determining step of Claim 3 is an abstract idea.
Claim 4 recites “wherein the determined at least one statistic further includes a correlation based at least in part on the ratio and a histopathologic grade.” However, the limitation of the determining step may also be performed in the mind by observation, evaluation, and judgment; under the broadest reasonable interpretation, a correlation may be any relationship between a ratio and a histopathologic grade, and thus may be determined in the mind. Thus, the determining step of Claim 4 is an abstract idea.
Claim 5 recites “wherein the determined at least one statistic includes a ratio of a total number of multinucleated cells identified for the plurality of tiles to a total number of mast cells identified for the plurality of tiles.” However, the limitation of the determining step may also be performed in the mind by observation, evaluation, and judgment, and thus the determining step of Claim 5 is an abstract idea.
Claim 6 recites “The computer-implemented method of claim 5, wherein the determined at least one statistic further includes a correlation based at least in part on the ratio and a histopathologic grade.” However, the limitation of the determining step may also be performed in the mind by observation, evaluation, and judgment; under the broadest reasonable interpretation, a correlation may be any relationship between a ratio and a histopathologic grade, and thus may be determined in the mind. Thus, the determining step of Claim 6 is an abstract idea.
Claim 7 recites the additional elements “wherein the first trained machine learning model includes at least one of a convolutional neural network sub-model and a deformable detection transformer sub-model, and wherein the second trained machine learning model includes a logistic regression model.” However, these elements are also recited at a high level of generality and also amount to mere instructions to apply the abstract ideas to a computer. . The claim fails to recite details of how identifying and determining are accomplished and merely invokes elements of machine learning models as tools to perform an existing process of visually identifying cytological features and determining statistics of cytological features.
Claim 8 recites the additional element “wherein the first trained machine learning model includes a mask regional convolutional neural network sub-model configured to identify a total number of mast cells for each of the plurality of tiles.” However, this element is also recited at a high level of generality and also amount to mere instructions to apply the abstract ideas to a computer. The claim fails to recite details of how identifying is accomplished and merely invokes element of a mask regional convolutional neural network sub-model as a tool to perform an existing process of visually identifying mast cells. A mask regional convolutional neural network is used generally to apply the abstract idea without placing any limits on how the mask R-CNN functions. Rather, the limitation only recites the outcome of “identify a total number of mast cells” and does not include any details about how the identifying is accomplished.
Claim 9 recites the additional element “wherein the first trained machine learning model includes a mask regional convolutional neural network sub-model configured to identify a total number of mast cell nuclei for each of the plurality of tiles.” However, this element is also recited at a high level of generality and also amount to mere instructions to apply the abstract ideas to a computer. The claim fails to recite details of how identifying is accomplished and merely invokes element of a mask regional convolutional neural network sub-model as a tool to perform an existing process of visually identifying mast cell nuclei. A mask regional convolutional neural network is used generally to apply the abstract idea without placing any limits on how the mask R-CNN functions. Rather, the limitation only recites the outcome of “identify a total number of mast cells nuclei” and does not include any details about how the identifying is accomplished.
Claim 10 recites the additional element “wherein the first trained machine learning model includes a mask regional convolutional neural network sub-model configured to identify a total number of mitotic figures for each of the plurality of tiles.” However, this element is also recited at a high level of generality and also amount to mere instructions to apply the abstract ideas to a computer. The claim fails to recite details of how identifying is accomplished and merely invokes element of a mask regional convolutional neural network sub-model as a tool to perform an existing process of visually identifying mitotic figures. A mask regional convolutional neural network is used generally to apply the abstract idea without placing any limits on how the mask R-CNN functions. Rather, the limitation only recites the outcome of “identify a total number of mitotic figures” and does not include any details about how the identifying is accomplished.
Claim 11 recites the additional element “wherein the first trained machine learning model includes a deformable detection transformer sub-model configured to identify a total number of multinucleated cells for each of the plurality of tiles.” However, this element is also recited at a high level of generality and also amount to mere instructions to apply the abstract ideas to a computer. The claim fails to recite details of how identifying is accomplished and merely invokes element of a deformable detection transformer sub-model as a tool to perform an existing process of visually identifying a total number of multinucleated cells. A mask regional convolutional neural network is used generally to apply the abstract idea without placing any limits on how the mask R-CNN functions. Rather, the limitation only recites the outcome of “identify a total number of multinucleated cells” and does not include any details about how the identifying is accomplished.
Claims 12 and 13 recite limitations regarding the statistic: “wherein the determined at least one statistic is associated with a tumor,” and “wherein the determined at least one statistic is associated with an infectious agent,” respectively. These limitations do not prevent one of ordinary skill in the art from determining statistics in the mind; for example, a number of tumor cells or a number of infectious agent cells may be determined by observation, evaluation, and judgment by one of ordinary skill in the art provided with image data representing a biological sample.
Claim 14 recites “outputting second image data representing an annotated image.” One of ordinary skill in the art would be able to annotate an image with the help of physical aids. Furthermore, the claim does not recite how the second image data is annotated, or if the annotations are in any way related to the previous steps, or are even an annotated image of the first image, and thus outputting an annotated image is also mere data outputting and amounts to insignificant extrasolution activity.
Claim 15 recites “outputting the determined at least one statistic includes outputting a report.” Under the broadest reasonable interpretation, a report includes outputting a report of only the at least one statistic and does not require any further information. Thus, the limitation of Claim 15 is also insignificant extrasolution activity and amounts to mere data outputting.
Claim 16 recites the additional element “generating, using a third trained machine learning model, a report based on the determined at least one statistic, wherein outputting the determined at least one statistic includes outputting the report.” However, this element is also recited at a high level of generality and also amount to mere instructions to apply the abstract ideas to a computer. The claim fails to recite details of how identifying is accomplished and merely invokes element of a deformable detection transformer sub-model as a tool to perform an existing process of generating a report based on a statistic (e.g. writing down a statistic using a physical aid of a pen and paper). A third trained machine learning model is used generally to apply the abstract idea without placing any limits on how the machine learning model functions. Rather, the limitation only recites the outcome of generating “a report” and does not include any details about how the identifying is accomplished. Additionally, under the broadest reasonable interpretation, a report includes outputting a report of only the at least one statistic and does not require any further information. Thus, the limitation of Claim 16 reciting “wherein outputting the determined at least one statistic includes outputting the report” is also insignificant extrasolution activity and amounts to mere data outputting.
Claim 17 recites the additional element “a generative machine learning model configured to generate the report based on data representing an intended reader of the report.” However, this element is also recited at a high level of generality and also amount to mere instructions to apply the abstract ideas to a computer. The claim fails to recite details of how identifying is accomplished and merely invokes element of a generative machine learning model as a tool to perform an existing process of generating a report based on data representing an intended reader of the report (e.g. simply writing the statistic and the name of an intended reader with the aid a of a pen and paper). A generative machine learning model is used generally to apply the abstract idea without placing any limits on how the generative ML model functions. Rather, the limitation only recites the outcome of “generate the report based on data representing an intended reader” and does not include any details about how the identifying is accomplished.
Claims 18 and 19 recite a system with elements corresponding to Claims 1 and 2, respectively, and thus contain the abstract ideas and additional elements stated above. Additionally, Claims 18 and 19 recite “at least one memory storing instructions; and at least one processor configured to execute the instructions to perform operations.” However, these elements are generic computer components and do not amount to significantly more than the judicial exceptions, or integrate the judicial exceptions into a practical application.
Claim 20 recites a non-transitory computer-readable medium with elements corresponding to Claim 1, and thus contain the abstract ideas and additional elements stated above. Additionally, Claim 20 recites “A non-transitory computer-readable medium storing instructions that, when executed by at least one processor, cause the processor to perform operations.” However, these elements are generic computer components and do not amount to significantly more than the judicial exceptions, or integrate the judicial exceptions into a practical application.
Thus, Claims 1-20 are rejected under 35 U.S.C. 101
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 2, 12, and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Cireşan et al. (Mitosis Detection in Breast Cancer Histology Images with Deep Neural Networks, published 2013), in view of Camus et al. (Cytologic Criteria for Mast Cell Tumor Grading in Dogs with Evaluation of Clinical Outcome, published 2016).
Regarding Claim 1, Cireşan teaches “A computer-implemented method for cytological grading, comprising:
receiving first image data representing a biological sample” (Cireşan, Section 2, paragraph 1 discloses “Given an input RGB image I, the problem is to find a set D = {d1,d2,...,dN} of detections, each reporting the centroid coordinates for a single mitosis”; where an input RGB image I is fist image data representing a biological sample);
“parsing the received first image data into a plurality of tiles, each of the plurality of tiles representing a respective portion of the received first image data” (Cireşan, Section 2, paragraph 1 discloses “For any given pixel p, the DNN predicts its class using raw RGB values in a square image window centered on p (Figure 1). Windows of class mitosis contain a visible mitosis around the window’s center. Others contain off-center or no mitosis”; where windows are a plurality of tiles);
“identifying, using a first trained machine learning model, at least one cytological feature for each of the plurality of tiles” (Cireşan, Section 2, paragraph 1 discloses “The problem is solved by training a detector on training images with given ground truth information about the centroid of each visible mitosis. Each pixel is assigned one of two possible classes, mitosis or non-mitosis, the former to pixels at (or close to) mitosis centroids, the latter to all other pixels. Our detector is a DNN-based pixel classifier”; where a DNN-based pixel classifier is a first trained machine learning model; where a set D of detections of centroid coordinates of mitosis is at least one cytological feature);
Cireşan does not explicitly teach “determining, using a second trained machine learning model, at least one statistic based on the identified at least one cytological feature for each of the plurality of tiles; and
outputting the determined at least one statistic.”
However, in an analogous field of endeavor, Camus teaches “determining, using a second trained machine learning model, at least one statistic based on the identified at least one cytological feature for each of the plurality of tiles” (Camus, Materials and Methods; Development of the Grading Scheme and Statistical Analysis, paragraph 5 discloses “Multiple Cox proportional hazards regression was used to evaluate the relationship of various factors to survival probability individually and together. Factors included patient age, margins, the 4 subcomponents of the 2-tier histologic grade (mitotic figures, multinucleation, bizarre nuclei, and karyomegaly) and the 6 subcomponents of the median calculated cytologic grade (granularity, mitotic figures, binucleation, multinucleation, nuclear pleomorphism, and anisokaryosis)”; where relationship between survival probability and various factors is at least one statistic; where Cox proportional hazards regression is a second trained machine learning model; where factors including mitotic figures are at least one cytological feature); “and
outputting the determined at least one statistic” (Camus, Table 2 and Results, paragraph 6 discloses “Histologic and cytologic high grade tumors were each associated with significantly decreased probability of survival (Table 2)”; where outputting mean survival times in relation to 2-tier histology grade, which is based on mitotic figures, is outputting the at least one statistic).
PNG
media_image1.png
427
437
media_image1.png
Greyscale
Table 2 of Camus
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Cireşan to incorporate the teachings of Camus by determining relationships between survival probability and various factors, factors including mitotic figures. The prior art Cireşan contained a ‘base’ method upon which the claimed invention can be seen as an ‘improvement.’ Cireşan discloses a method of generating a set of detections of mitosis. The claimed invention can be seen as an ‘improvement’ because the claim performs a determining of a statistic based on cytological features, thus performing an additional step that Cireşan does not disclose. The prior art contained a known technique that is applicable to the base method. Camus teaches a technique of statistical analysis of features including mitotic figures. One of ordinary skill in the art would have recognized that applying the known technique of Camus to Cireşan would have yielded predictable results and resulted in an improved system. Camus also discloses in Results, paragraph 3 discloses “Selected cytologic grading criteria are presented in Figs. 1–4. The cytologic characteristic most associated with 2-year survival was granularity (R2 ¼ .40, P < .001), followed by anisokaryosis (R2 ¼ .27, P < .001), multinucleated cells (R2 ¼ .21, P <.001),binucleated cells(R2 ¼ .21, P <.001),and mitotic figures (R2 ¼ .16, P < .001).” One of ordinary skill in the art would be motivated to combine the Cireşan and Camus references in order to develop and improve the cytological grading scheme for mast cell tumors: Camus, introduction, paragraph 5 discloses “The purpose of this prospective, multi-institutional study was to use the 2-tier grading criteria as a guide to develop an accurate and reproducible cytologic grading scheme for MCTs that is predictive of patient outcome.” Accordingly, the combination of Cireşan and Camus discloses the invention of Claim 1.
Regarding Claim 2, the combination of Cireşan and Camus teaches “The computer-implemented method of claim 1, wherein identifying the at least one cytological feature for each of the plurality of tiles includes identifying at least one of a number of mast cells, a number of mast cell nuclei, a number of mitotic figures, and a number of multinucleated cells” (Cireşan, Section 2, Processing a Testing Image, paragraph 2 discloses “This yields a (possibly empty)set DI (depending on threshold t) containing the detected centroids of all mitosis in image I, as well as their respective score”; where obtaining a set containing detected centroids is identifying a number of mitotic figures).
Regarding Claim 12, the combination of Cireşan and Camus teaches “The computer-implemented method of claim 1, wherein the determined at least one statistic is associated with a tumor” (Camus, Abstract discloses “The cytologic grading scheme that best correlated with histology (kappa ¼ 0.725 + 0.085) classified a tumor as high grade if it was poorly granulated or had at least 2 of 4 findings: mitotic figures, binucleated or multinucleated cells, nuclear pleomorphism, or >50% anisokaryosis.”)
Regarding Claims 18-19, Claims 18-19 recite a system with elements corresponding to the steps recited in Claims 1-2, respectively. Therefore, the recited elements of this claim are mapped to the proposed combination in the same manner as the corresponding steps in its corresponding method claim. Additionally, the rationale and motivation to combine the Cireşan and Camus references, presented in rejection of Claim 1, apply to this claim. Finally, the combination of Cireşan and Camus references discloses “A computer system for cytological grading, the computer system comprising: at least one memory storing instructions; and at least one processor configured to execute the instructions to perform operations” (Cireşan, Section 3, Building the Detector, paragraph 5 discloses “Training each network requires one day of computation with an optimized GPU implementation”; where a GPU is a computer system comprising a memory and at least one processor).
Regarding Claim 20, Claim 20 recites a computer-readable storage medium storing a program with instructions corresponding to the steps recited in Claim 1. Therefore, the recited programming instructions of this claim are mapped to the proposed combination in the same manner as the corresponding steps in its corresponding method claim. Additionally, the rationale and motivation to combine the Cireşan and Camus references, presented in rejection of Claim 1, apply to this claim. Finally, the combination of Cireşan and Camus references discloses a computer readable storage medium “A non-transitory computer-readable medium storing instructions that, when executed by at least one processor, cause the processor to perform operations for cytological grading” (Cireşan, Section 3, Building the Detector, paragraph 5 discloses “Training each network requires one day of computation with an optimized GPU implementation”; where a GPU is a computer system comprising a memory and at least one processor).
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Cireşan et al. (Mitosis Detection in Breast Cancer Histology Images with Deep Neural Networks, published 2013), in view of Camus et al. (Cytologic Criteria for Mast Cell Tumor Grading in Dogs with Evaluation of Clinical Outcome, published 2016), further in view of Verma et al. (US 2024/0428939 A1).
Regarding Claim 10, the combination of Cireşan and Camus does not explicitly teach the method of Claim 10.
However, in an analogous field of endeavor, Verma teaches “The computer-implemented method of claim 1, wherein the first trained machine learning model includes a mask regional convolutional neural network sub-model configured to identify a total number of mitotic figures for each of the plurality of tiles” (Verma, [0026] discloses “In some embodiments, the machine-learning model applies a model selected from the group consisting of Support Vector Machines (SVM), Random Forests (RF), Artificial Neural Network (ANN), Convolutional Neural Network (CNN), K-means, ResNet, DenseNet, eXtreme Gradient Boosting (XGBoost), VGG, swin-transformer, Faster-RCNN, Mask-RCNN, and CentralNet++. In some embodiments, the machine-learning model uses a uses a probability metric for a predicted classification of the mitotic event and/or wherein the annotated events with confidence above a specified threshold are used to compute a chromosomal instability metric”).
It would have been obvious to one of ordinary skill in the art before the effective filing
date of the claimed invention to have modified the combination of Cireşan and Camus to incorporate the teachings of Verma by implementing a Mask-RCNN to predict a classification of mitotic events. One of ordinary skill in the art would be motivated to combine the Cireşan, Camus, and Verma references in order to improve accuracy of specimen grading: Verma, [0033] discloses “However, aspects of the present disclosure provide processes for more efficiently analyzing an entire digital microscopic image, and more accurately grading a specimen depicted in the image. Moreover, embodiments described herein provide for a more detailed and complete cytological analysis of a specimen, and may thereby support improved clinical decision-making.” Accordingly, the combination of Cireşan, Camus, and Verma discloses the invention of Claim 10.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Cireşan et al. (Mitosis Detection in Breast Cancer Histology Images with Deep Neural Networks, published 2013), in view of Camus et al. (Cytologic Criteria for Mast Cell Tumor Grading in Dogs with Evaluation of Clinical Outcome, published 2016), further in view of Buzaglo et al. (US 2015/0213599 A1).
Regarding Claim 13, the combination of Cireşan and Camus does not explicitly teach the method of Claim 13.
However, in an analogous field of endeavor, Buzaglo teaches “The computer-implemented method of claim 1, wherein the determined at least one statistic is associated with an infectious agent” (Buzaglo, [0016] discloses “The CNN, having previously been trained with a database of images, has acquired the ability to recognize and report with some precision the presence of bacteria in the tissue image”; where bacteria is an infectious agent).
It would have been obvious to one of ordinary skill in the art before the effective filing
date of the claimed invention to have modified the combination of Cireşan and Camus to incorporate the teachings of Buzaglo by using a CNN to identify bacteria in a tissue image. One of ordinary skill in the art would be motivated to combine the Cireşan, Camus, and Buzaglo references in order to overcome difficulties of extracting cell and bacteria regions: Buzaglo, [0011] discloses “Some structures, such as cells, may appear connected, blurred or occluded by other tissue elements in the image. Furthermore, the cell architecture and bacteria may be presented in various 3D orientations due to the method of slicing that exacerbate the challenge of image analysis. Those complexities have made histological slides difficult to extract cell regions and bacteria by traditional image segmentation approaches.” Accordingly, the combination of Cireşan, Camus, and Buzaglo discloses the invention of Claim 13.
Claims 14-17 are rejected under 35 U.S.C. 103 as being unpatentable over Cireşan et al. (Mitosis Detection in Breast Cancer Histology Images with Deep Neural Networks, published 2013), in view of Camus et al. (Cytologic Criteria for Mast Cell Tumor Grading in Dogs with Evaluation of Clinical Outcome, published 2016), further in view of Yoo et al. (US 2022/0036971 A1).
Regarding Claim 14, the combination of Cireşan and Camus does not explicitly teach the method of Claim 14.
However, in an analogous field of endeavor, Yoo teaches “The computer-implemented method of claim 1, further comprising: outputting second image data representing an annotated image” (Yoo, [0140] discloses “The user terminal may output a pathology slide image, which may be displayed as a mask for a target item in units of areas, and displayed as a center point of a cell nucleus or a bounding box for a target item in units of cells”; where a mask or bounding box is an annotated image).
It would have been obvious to one of ordinary skill in the art before the effective filing
date of the claimed invention to have modified the combination of Cireşan and Camus to incorporate the teachings of Yoo by displaying a mask or bounding box over a target item. One of ordinary skill in the art would be motivated to combine the Cireşan, Camus, and Yoo references in order to provide intuitive information to a user: Yoo, [0030] discloses “According to some embodiments, the user can visually and intuitively be provided with results generated in the process of predicting responsiveness to immune checkpoint inhibitor.” Accordingly, the combination of Cireşan, Camus, and Yoo discloses the invention of Claim 14.
Regarding Claim 15, the combination of Cireşan, Camus, and Yoo teaches “The computer-implemented method of claim 1, wherein outputting the determined at least one statistic includes outputting a report” (Yoo, [0143] discloses “FIGS. 11 to 15 are diagrams illustrating examples of outputting results generated in a process of predicting response or non-response to an immune checkpoint inhibitor according to another embodiment. In an embodiment, the information processing system (e.g., at least one processor of the information processing system) may generate and provide reports 1100, 1200, 1300, 1400, and 1500 including the results generated in the process of predicting response or non-response to the immune checkpoint inhibitor. The generated reports 1100, 1200, 1300, 1400, and 1500 may be provided as a file, data, text, image, and the like that can be output through the user terminal and/or output devices.”) The proposed combination as well as the motivation for combining the Cireşan, Camus, and Yoo references presented in the rejection of claim 14, apply to Claim 15 and are incorporated herein by reference. Thus, the apparatus recited in Claim 15 is met by Cireşan, Camus, and Yoo.
PNG
media_image2.png
464
710
media_image2.png
Greyscale
Fig. 10 of Yoo
Regarding Claim 16, the combination of Cireşan, Camus, and Yoo teaches “The computer-implemented method of claim 1, further comprising: generating, using a third trained machine learning model, a report based on the determined at least one statistic, wherein outputting the determined at least one statistic includes outputting the report.” (Yoo, [0146] discloses “For example, as illustrated in FIG. 11, the report 1100 may include a pathology slide image, patient information, basic information, and/or prediction results (whether responder or non-responder). In addition, as illustrated in FIG. 12, the report 1200 may include graphs, numerical values, and information on TIL density (e.g., distribution of densities, and the like) indicating the ratio of immune phenotypes. In addition, as illustrated in FIG. 13, the report 1300 may include statistical results (e.g., TCGA PAN-CARCINOMA STATISTICS) and/or graphs indicating analysis results, clinical notes, and the like. As illustrated in FIG. 14, the report 1140 may include information on reference documents (e.g., academic references), and the like. As illustrated in FIG. 15, the report 1150 may include results (e.g., immune phenotype map images, feature statistics, and the like) generated in the prediction process and/or information used in the prediction process, and the like”; it would be obvious to one of ordinary skill in the art that multiple types of results may be output in the report, and thus the mitosis detections of Cireşan and Camus may be simply substituted for prediction results). The proposed combination as well as the motivation for combining the Cireşan, Camus, and Yoo references presented in the rejection of claim 14, apply to Claim 16 and are incorporated herein by reference. Thus, the apparatus recited in Claim 16 is met by Cireşan, Camus, and Yoo.
Regarding Claim 17, the combination of Cireşan, Camus, and Yoo teaches “The computer-implemented method of claim 16, wherein the third trained machine learning model includes a generative machine learning model configured to generate the report based on data representing an intended reader of the report” (Yoo, [0067] discloses “As another example, the user may include a doctor, a patient, and the like provided with a prediction result of a response to an immune checkpoint inhibitor (e.g., a prediction result as to whether or not the patient responds to an immune checkpoint inhibitor).” Yoo, [0143] discloses “The generated reports 1100, 1200, 1300, 1400, and 1500 may be provided as a file, data, text, image, and the like that can be output through the user terminal and/or output devices”). The proposed combination as well as the motivation for combining the Cireşan, Camus, and Yoo references presented in the rejection of claim 14, apply to Claim 17 and are incorporated herein by reference. Thus, the apparatus recited in Claim 17 is met by Cireşan, Camus, and Yoo.
Allowable Subject Matter
Claims 3-9 and 11 are rejected under 35 U.S.C. 101 and are objected to as being dependent upon a rejected base claim, but would be allowable if: a) the rejection under 35 U.S.C. 101 is overcome and b) rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding Claim 3, although the cited prior art discloses determining a number of mitotic figures (Cireşan, Section 2, paragraph 1 discloses “The problem is solved by training a detector on training images with given ground truth information about the centroid of each visible mitosis”), the cited prior art does not explicitly teach determining a ratio of total number of mitotic figures to a total number of mast cells. Berlato et al. (Value, Limitations, and Recommendations for Grading of Canine Cutaneous Mast Cell Tumors: A Consensus of the Oncology-Pathology Working Group, published 2021) discloses “In dogs with cutaneous MCTs, the literature reviewed provides evidence that the mitotic count is an independent prognostic indicator. In dogs with cutaneous MCTs, the literature reviewed provides evidence that the mitotic count is an independent prognostic indicator.1,2,7,15,22,23 This was previously referred to as mitotic index, but the appropriateness of the term mitotic index (defined as the number of cells in mitosis divided by the number of cells not in mitosis) has recently come into question and the adoption of the term mitotic count (number of mitotic figures within a given area) has been indicated as more appropriate.11 It has also been recently proposed that the mitotic count (MC) be determined and reported in a standardized area of 2.37 mm2.”) Although Berlato teaches various metrics for mitotic figure quantification, Berlato also does not explicitly teach a ratio between mitotic figures to mast cell count.
Thus, none of the previously cited prior art provide a motivation to teach the ordered combination of “The computer-implemented method of claim 1, wherein the determined at least one statistic includes a ratio of a total number of mitotic figures identified for the plurality of tiles to a total number of mast cells identified for the plurality of tiles.”
Claim 4 depends from Claim 3 and thus contains all allowable subject matter of Claim 3.
Regarding Claim 5, although the cited prior art discloses determining a number of mitotic figures (Cireşan, Section 2, paragraph 1 discloses “The problem is solved by training a detector on training images with given ground truth information about the centroid of each visible mitosis”), the cited prior art does not explicitly teach determining a ratio of a total number of multinucleated cells to a total number of mast cells. Berlato, cited above, teaches various metrics for mitotic figure quantification, but likewise does not explicitly teach a multinucleated cells count, a mast cell count, nor a multinucleated cells to a total mast cell ratio.
Thus, none of the previously cited prior art provide a motivation to teach the ordered combination of “The computer-implemented method of claim 1, wherein the determined at least one statistic includes a ratio of a total number of multinucleated cells identified for the plurality of tiles to a total number of mast cells identified for the plurality of tiles.”
Claim 6 depends from Claim 5 and thus contains all allowable subject matter of Claim 5.
Regarding Claim 7, although the cited prior art teaches using a deep neural network and a Cox proportional hazards regression to detect mitotic figures and determine statistics based on mitotic figures, respectively, (Cireşan, Section 2, paragraph 1 discloses “Our detector is a DNN-based pixel classifier. For any given pixel p, the DNN predicts its class using raw RGB values in a square image window centered on p (Figure 1). Windows of class mitosis contain a visible mitosis around the
window’s center. Others contain off-center or no mitosis”) (Camus, Materials and Methods; Development of the Grading Scheme and Statistical Analysis, paragraph 5 discloses “Multiple Cox proportional hazards regression was used to evaluate the relationship of various factors to survival probability individually and together”), the cited prior art does not explicitly teach an ML model with a CNN and deformable detection transformer submodel, and a logistic regression model. Camus does teach logistic regression (Camus, Materials and Methods; Development of the Grading Scheme and Statistical Analysis, paragraph 6 discloses “Simple logistic regression was performed to test for relationships between risk factors and the probability of mortality”), but does not explicitly teach the model may be used to determine a statistic based on at least one cytological feature determined by the first trained machine learning model.
Thus, none of the previously cited prior art, alone or in combination, provides a motivation to teach the ordered combination of “The computer-implemented method of claim 1, wherein the first trained machine learning model includes at least one of a convolutional neural network sub-model and a deformable detection transformer sub-model, and wherein the second trained machine learning model includes a logistic regression model.”
Regarding Claim 8, although the cited prior art teaches using a deep neural network to detect mitotic figures (Cireşan, Section 2, paragraph 1 discloses “Our detector is a DNN-based pixel classifier. For any given pixel p, the DNN predicts its class using raw RGB values in a square image window centered on p (Figure 1). Windows of class mitosis contain a visible mitosis around the
window’s center. Others contain off-center or no mitosis”), the cited prior art does not explicitly teach a mask R-CNN configured to identify a total number of mast cells.
Thus, none of the previously cited prior art, alone or in combination, provides a motivation to teach the ordered combination of “The computer-implemented method of claim 1, wherein the first trained machine learning model includes a mask regional convolutional neural network sub-model configured to identify a total number of mast cells for each of the plurality of tiles.”
Regarding Claim 9, although the cited prior art teaches using a deep neural network to detect mitotic figures (Cireşan, Section 2, paragraph 1 discloses “Our detector is a DNN-based pixel classifier. For any given pixel p, the DNN predicts its class using raw RGB values in a square image window centered on p (Figure 1). Windows of class mitosis contain a visible mitosis around the
window’s center. Others contain off-center or no mitosis”), the cited prior art does not explicitly teach a mask R-CNN configured to identify a total number of mast cell nuclei.
Thus, none of the previously cited prior art, alone or in combination, provides a motivation to teach the ordered combination of “The computer-implemented method of claim 1, wherein the first trained machine learning model includes a mask regional convolutional neural network sub-model configured to identify a total number of mast cell nuclei for each of the plurality of tiles.”
Regarding Claim 11, although the cited prior art teaches using a deep neural network to detect mitotic figures (Cireşan, Section 2, paragraph 1 discloses “Our detector is a DNN-based pixel classifier. For any given pixel p, the DNN predicts its class using raw RGB values in a square image window centered on p (Figure 1). Windows of class mitosis contain a visible mitosis around the
window’s center. Others contain off-center or no mitosis”), and generic deformable detection transformers are known in the art, the cited prior art does not explicitly teach a deformable detection transformer configured to identify a number of multinucleated cells.
Thus, none of the previously cited prior art, alone or in combination, provides a motivation to teach the ordered combination of “The computer-implemented method of claim 1, wherein the first trained machine learning model includes a deformable detection transformer sub-model configured to identify a total number of multinucleated cells for each of the plurality of tiles.”
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Aubreville et al. (Deep learning algorithms out-perform veterinary pathologists in detecting the mitotically most active tumor region, published 2020) discloses a method of estimating mitotic count using a CNN and compares results of the CNN method and expert determined mitotic count.
Zhang et al. (An Efficient Transformer-based Approach for Joint Nuclei Detection and Segmentation in Whole Slide Tissue Images, published 2022) discloses a method of adapting a deformable transformer encoder-decoder scheme to detect and segment nuclei.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CAROLINE TABANCAY DUFFY whose telephone number is (703)756-1859. The examiner can normally be reached Monday - Friday 8:00 am - 5:30 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached at 5712723382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CAROLINE TABANCAY DUFFY/Examiner, Art Unit 2662
/AMANDEEP SAINI/Supervisory Patent Examiner, Art Unit 2662
Prosecution Insights