DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 12/13/2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-2, 4, 6-13, 15, 17, 18 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Gaire et al. (WO 2020083970 A1, as cited by applicant in IDS filed on 12/13/2024) in view of Yoo et al. US PG-Pub(US 20220036971 A1).
Regarding Claim 1, Gaire teaches a method(Figure 2 is a flow chart of an image analysis method for determining the biomedical state of a tissue sample) comprising: accessing, by a digital pathology image processing system, a digital pathology image that depicts a section of a biological sample(Page 38, Lines 29-30 – Page 39, Line 1, “The method can be implemented in and performed for example by an image analysis system 100 as depicted in figure 1. In a first step 202, the image analysis system 100 receives a digital image of a tissue sample.”, in this section of the prior art, an image analysis system receives a digital image of a tissue sample for determining tumor and immune cells in an area as depicted in step 204.), wherein the digital pathology image comprises regions displaying reactivity to a plurality of stains(Page 37, Lines 6-9 “The image 118 can be a brightfield microscopy image or a fluorescence image. Typically, the image is a multichannel image obtained by fluorescence microscopy from a tissue sample having been stained with one or more biomarker specific stains having different colors.”, Page 37, Lines 6-9 disclose the tissue sample is stained and on page 37, Lines 12-17 disclose that the identification of cell types within the area are determined by reactivity to the stain); for each of the tiles, calculating, by the digital pathology image processing system, a local-density measurement of each of a plurality of biological object types(Page 40, Lines 26-29, “ In the next step, the identified density of tumor cells and immune cells in the image or in the tumor tissue area is determined and used for simulating a Poisson distribution of simulated tumor cells and simulated immune cells, respectively. Thereby, the density of the simulated tumor cells and simulated immune cells is identical to the observed density of tumor cells and immune cells in the tissue”, in this section of the prior art, the density of the tumor and immune cells are determined and used for simulating a Poisson distribution); generating, by the digital pathology image processing system, one or more spatial- distribution metrics for the biological object types in the digital pathology image based at least in part on the calculated local-density measurements(Page 41, Lines 1-12, “Then, the distances of all simulated tumor cells to their respectively nearest simulated immune cell are determined 208 and represented as a “reference relative distribution”. Then, the proximity score is computed 210 a function of the determined distances. For example, the distance where - on average - every tumor cell comprises one immune cell in its neighborhood can be determined both for the observed relative distribution and for the reference relative distribution and the difference of these two differences can be used as the proximity score. Then, the image analysis system provides 212 a combined score comprising the proximity score and the immune cell density observed in the image 118 and uses 214 the combined score either for automatically determining the biomedical state (e.g. infiltration state) of the tissue and/or for displaying 216 the combined score on a 2-D score plot as depicted,”, in this section of the prior art, once the Poisson distribution is calculated for the immune and tumor cells, a proximity score is computed based on the distance and the proximity score is used to determine the biomedical state of the tissue. ); and determining, by the digital pathology image processing system, a tumor immunophenotype of the digital pathology image based at least in part on the local-density measurements or the one or more spatial-distribution metrics. (Page 40, Lines 5-13, “Next in step 212, the image analysis system combines the proximity score with the density of immune cells measured in the received digital image 118 for providing a “combined score”. Next in step 214, the image analysis system uses the combined score for automatically determining the biomedical state of the tissue sample, e.g. a particular infiltration state. In addition, or alternatively, the image analysis system displays in step 216 the combined score obtained for the tissue sample on a display 102 of the image analysis system to a user. Thereby, the user is enabled to visually assess the current biomedical state of the tissue and make according conclusions regarding the progress of the disease and suitable treatment options.“, as disclosed in this section of the prior art, the local density measurements of the immune cells are measured and a proximity score is generated and used to determine the biomedical state of the tissue whether there is a tumor within the sample and a treatment can be determined for the state of the tissue.)
Gaire does not explicitly teach subdividing, by the digital pathology image processing system, the digital pathology image into a plurality of tiles;
Yoo teaches subdividing, by the digital pathology image processing system, the digital pathology image into a plurality of tiles (¶[0101], “the processor may determine that at least some patches (e.g., patches in which items associated with cancer and immune cells are detected) from among a plurality of patches (e.g., patches of size 1 mm2) generated by dividing the pathology slide image into N grids (where N is any natural number) are the region of interest (e.g., at least some regions in the pathology slide image).”, as disclosed in ¶[0101], the pathological image is divided into N grids which represent regions of interest in the pathological image.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the claimed invention as taught by Gaire with Yoo in order to divide the digital image into a plurality of tiles/regions. One skilled in the art would have been motivated to modify Gaire in this manner in order to objectively analyze the immune environment around the cancer cells, the predictive rate for whether or not a patient will respond to the immune checkpoint inhibitor can be improved. (Yoo, ¶[0027]
Regarding Claim 2, the combination of Gaire and Yoo teach the method of claim 1, where Gaire further teaches wherein each of the local-density measurements comprises a representation of an absolute or relative quantity, an area, or a density. (Page 40, Lines 26-29 discloses the density of the immune and tumors cells are determined.)
Regarding Claim 4, the combination of Gaire and Yoo teach the method of claim 1, Gaire further teaches wherein the one or more spatial-distribution metrics characterize a degree to which a first biological object type is depicted as being interspersed with a second biological object type. (Page 41, Lines 1-6, “Then, the distances of all simulated tumor cells to their respectively nearest simulated immune cell are determined 208 and represented as a “reference relative distribution”. Then, the proximity score is computed 210 a function of the determined distances. For example, the distance where - on average - every tumor cell comprises one immune cell in its neighborhood can be determined both for the observed relative distribution and for the reference relative distribution and the difference of these two differences can be used as the proximity score.”, The term interspersed in light of the specification was defined in ¶[0050] to be object types that are physically proximate to each other and in this section of the prior art, a distance metric is determined between the immune cells and the tumor cells and a proximity score is calculated based on the distance between the immune cells and tumor.)
Regarding Claim 6, the combination of Gaire and Yoo teach the method of claim 1, Gaire further teaches wherein the biological object types comprise tumor cells and immune cells(Page 25, Lines 3-17, “For example, the tumor cells can be identified in a digital image of a tissue sample by staining the tissue sample with one or more tumor marker specific stains (e.g. stains selectively staining cytokeratins, BRAC1 , etc.) and applying image analysis methods for identifying cells expressing said biomarkers as the tumor cells.”), the tumor immunophenotype comprises: excluded when, for one or more of the tiles, the one or more spatial-distribution metrics indicate a spatial separation of the tumor cells and the immune cells(Page 28, Lines 26-30 – Page 29, Lines 1-2, ““excluded”, wherein “excluded” is indicative of an immunological tissue state in which immune cells are present in the tissue sample but are hindered to come into close contact to the tumor cells, whereby the immune cells are concentrated at the invasive margin and/or in the intratumoral stroma but separated from the tumor cells, wherein the higher the observed immune cell density in the image area and the lower the proximity score, the higher the likelihood of a tissue sample of being classified as“excluded”, as disclosed in this section of the prior art a proximity score is calculated between the tumor cells and immune and based on the proximity score the state of excluded is determined.”; or inflamed when, for one or more of the tiles, the one or more spatial-distribution metrics indicate a co-localization of the tumor cells and the immune cells (Page 28, Lines 20-25, “inflamed”, wherein “inflamed“ is indicative of an immunological tissue state in which immune cells have a significantly increased cell density being indicative of a heavy infiltration of the tumor tissue with immune cells in all compartments of the tumor, wherein the higher the observed immune cell density in the image area and the higher the proximity score, the higher the likelihood of a tissue sample of being classified as “inflamed), as disclosed in this section of the prior art cell density and a proximity score is used to classify the tissue sample to be inflamed.).
Regarding Claim 7, the combination of Gaire and Yoo teach the method of claim 1, wherein calculating the local-density measurement of each of the biological object types comprises: for each of the tiles: segmenting the tile into a plurality of regions according to the stains([0128] “In this experiment, for pathological validation of immune phenotype, a H&E-stained pathology slide image of a patient were segmented into multiple patches of 1 mm2, and the immune phenotype was determined based on the tumor-infiltrating lymphocyte (TIL) in the cancer area and the TIL in the cancer stroma in each patch”, ¶[0128] discloses that the stained image was segmented into a plurality of regions/patches.), wherein each of the biological object types is reactive to one of the stains(¶[0083] discloses a detecting one or more target items in the stained pathological image in order to identify object types in the stained image.); classifying each of the regions according to reactivity to the stains; ([0101] “The processor may detect items associated with cancer and immune cells, including a cancer area (e.g., cancer epithelium), cancer stroma and/or tumor cells in the pathology slide image (e.g., entire H&E-stained slide image) (S410). Then, the processor may determine one or more regions of interest in the pathology slide image (S420). For example, the processor may determine that at least some patches (e.g., patches in which items associated with cancer and immune cells are detected) from among a plurality of patches (e.g., patches of size 1 mm2) generated by dividing the pathology slide image into N grids (where N is any natural number) are the region of interest (e.g., at least some regions in the pathology slide image). [0102] The processor may determine an immune phenotype of one or more regions of interest based on the detection result for the items associated with cancer and immune cells in the one or more regions of interest (S430)”, ¶[0101]-¶[0102] disclose classifying the regions of interest in the stained image by determining whether the regions include immune cells or cancer cells. )and calculating the local-density measurement of each of the biological object types located within the tile based on a number of the regions of the tile classified with each of the stains. (¶[0102], “the processor may calculate at least one of the number of, a distribution of, or a density of immune cells in the item associated with cancer in the one or more regions of interest, and determine the immune phenotype of the one or more regions of interest based on at least one of the calculated number, distribution, or density of immune cells. For example, the processor may calculate, in one or more regions of interest, a density of immune cells in the cancer area (lymphocyte in tumor region) and a density of immune cells in the cancer stroma (lymphocyte in stroma region), and determine the immune phenotype of the one or more regions of interest based on at least one of the density of immune cells in the cancer area or the density of immune cells in the cancer stroma”, ¶[0102] discloses determining an immune phenotype based on the density of the immune cells or cancer cells in the image.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the claimed invention as taught by Gaire with Yoo in order to segment the stained image into regions of interest and calculate a density measurement of the immune and cancer cells. One skilled in the art would have been motivated to modify Gaire in this manner in order to predict whether or not a patient will respond to an immune checkpoint inhibitor by using at least one of an immune phenotype determined from a pathology slide image or information associated with the immune phenotype. (Yoo, ¶[0027])
Regarding Claim 8, the combination of Gaire and Yoo teach the method of claim 7, Gaire further teaches wherein each of the regions of the tile is determined based on a stain intensity value of the region, the stain intensity value being based on the reactivity of each of the biological object types to one of the stains. (Page 25, Lines 3-9, “the tumor cells can be identified in a digital image of a tissue sample by staining the tissue sample with one or more tumor marker specific stains (e.g. stains selectively staining cytokeratins, BRAC1 , etc.) and applying image analysis methods for identifying cells expressing said biomarkers as the tumor cells.Optionally, the tissue can be stained with immune-cell specific stains and the image analysis method is performed such that selectively non-immune cells expressing a tumor marker or a proliferation marker are identified as the tumor cells. For example, cells expressing the biomarker KI67 and which do not express the lymphohoid biomarkers CD8 (or CD3) are identified as tumor cells.” As disclosed in this section, the tissue sample is stained and tumor cells are identified based on the reactivity to the stain. Page 40, Lines 18-22, disclose using an image analysis system to analyze the image for immune and tumor cells in sub areas of the received image.)
Regarding Claim 9, the combination of Gaire and Yoo teach the method of claim 7, Gaire further teaches wherein the regions of the tiles are determined as tumor- associated regions and non-tumor-associated regions (Page 40, Lines 18-22, disclose using an image analysis system to analyze the image for immune and tumor cells in sub areas of the received image.).
Regarding Claim 10, the combination of Gaire and Yoo teach the method of claim 9, Gaire further teaches wherein each of the tumor-associated regions and non- tumor-associated regions are determined as immune-cell-associated regions and non- immune-cell-associated regions. (Page 40, Lines 18-22, disclose using an image analysis system to analyze the image for immune and tumor cells in sub areas of the received image and labeling the region to be a tumorous region or immune cell region based on the distribution of the cells.).
Regarding Claim 11, the combination of Gaire and Yoo teach the method of claim 1, Gaire further teaches wherein determining the tumor immunophenotype of the image comprises: projecting a representation of the digital pathology image into a feature space with axes based on the one or more spatial-distribution metrics(Figure 10 depicts a 2D score plot comprising clusters of combined scores, each combined score representing a respective one of a plurality of different tissue samples. As seen in figure 10, the Y axis represents a proximity score based on the distribution of the cells and the X axis represents the Immunophenotype of the cell.); and determining the tumor immunophenotype of the image based on a position of the digital pathology image within the feature space (Figure 10, X-Axis shows the immunophenotype based on the location of the cell distribution in the digital along the axis.)
Regarding Claim 12, the combination of Gaire and Yoo teach the method of claim 11, Gaire further teaches wherein determining the tumor immunophenotype of the image is further based on a proximity of the position of the digital pathology image within the feature space to a position of one or more other digital pathology image representations with assigned tumor immunophenotypes. (Page 27, Lines 29-31 - Page 28, Lines 1-7, “computing a combined score for each of the patients using the respectively received image in accordance with any one of the previous claims; and graphically representing the biomedical state of each patient by a respective, biomedical-state- specific symbol, on a 2D score plot, the position of the symbol of each patient in the plot depending on the B-type cell density and the proximity score computed for the patient. This may be advantageous, as a user is provided with a plot that allows to quickly and intuitively determine the biomedical state of a tissue sample, e.g. the immune cell infiltration state, based on the x and y coordinates of a point in the plot”, as disclosed in this section of the prior art and shown in Figure 10, a score is computed and graphically shown in a 2D plot in which points are shown in a plot such that a biomedical state/tumor immunophenotype can be identified through the 2D plot based on position of the x and y coordinates of the plot.)
Regarding Claim 13, the combination of Gaire and Yoo teach the method of claim 1, Gaire further teaches wherein the biological object types comprise cytokeratin and cytotoxic structures. (Page 25 Lines 3-5, “the tumor cells can be identified in a digital image of a tissue sample by staining the tissue sample with one or more tumor marker specific stains (e.g. stains selectively staining cytokeratins, BRAC1 , etc.) and applying image analysis methods for identifying cells expressing said biomarkers as the tumor cells.”, as disclosed in this section tumor cells are identified by staining cytokeratin )
Regarding Claim 15, the combination of Gaire and Yoo teach the method of claim 1, where Gaire further teaches further comprising: generating, based at least in part on the tumor immunophenotype of the image and the one or more spatial-distribution metrics, a result that corresponds to an assessment of a medical condition of a subject, including a prognosis for outcomes of the medical condition; and generating a display including an indication of the assessment of the medical condition of the subject and the prognosis. (Page 40, Lines 5-13, “Next in step 212, the image analysis system combines the proximity score with the density of immune cells measured in the received digital image 118 for providing a “combined score”. Next in step 214, the image analysis system uses the combined score for automatically determining the biomedical state of the tissue sample, e.g. a particular infiltration state. In addition, or alternatively, the image analysis system displays in step 216 the combined score obtained for the tissue sample on a display 102 of the image analysis system to a user. Thereby, the user is enabled to visually assess the current biomedical state of the tissue and make according conclusions regarding the progress of the disease and suitable treatment options.“, as disclosed in this section of the prior art, the local density measurements of the immune cells are measured and a proximity score is generated and used to determine the biomedical state of the tissue whether there is a tumor within the sample and a treatment can be determined for the state of the tissue that is displayed to the user.)
Regarding Claim 17, the combination of Gaire and Yoo teach the method of claim 1, further comprising: generating, based at least in part on the one or more spatial-distribution metrics, a result that corresponds to a prediction regarding a degree to which a given treatment that modulates immunological response will effectively treat a medical condition of a subject; determining that the subject is eligible for a clinical trial based on the result; and generating a display including an indication that the subject is eligible for the clinical trial. (Page 40, Lines 5-13, “Next in step 212, the image analysis system combines the proximity score with the density of immune cells measured in the received digital image 118 for providing a “combined score”. Next in step 214, the image analysis system uses the combined score for automatically determining the biomedical state of the tissue sample, e.g. a particular infiltration state. In addition, or alternatively, the image analysis system displays in step 216 the combined score obtained for the tissue sample on a display 102 of the image analysis system to a user. Thereby, the user is enabled to visually assess the current biomedical state of the tissue and make according conclusions regarding the progress of the disease and suitable treatment options.“, as disclosed in this section of the prior art, the local density measurements of the immune cells are measured and a proximity score is generated and used to determine the biomedical state of the tissue whether there is a tumor within the sample and a treatment can be determined for the state of the tissue.)
Regarding Claim 18, Gaire teaches a digital pathology image processing system comprising: one or more data processors; and a non-transitory computer readable storage medium communicatively coupled to the one or more data processors, and including instructions which, when executed by the one or more data processors(Figure 1, element 104 discloses a processor and element 106 discloses a memory and element 108 discloses a non-transitory computer readable medium) , cause the one or more data processors to perform one or more operations comprising:
accessing a digital pathology image that depicts a section of a biological sample (Page 38, Lines 29-30 – Page 39, Line 1, “The method can be implemented in and performed for example by an image analysis system 100 as depicted in figure 1. In a first step 202, the image analysis system 100 receives a digital image of a tissue sample.”, in this section of the prior art, an image analysis system receives a digital image of a tissue sample for determining tumor and immune cells in an area as depicted in step 204.), wherein the digital pathology image comprises regions displaying reactivity to a plurality of stains(Page 37, Lines 6-9 “The image 118 can be a brightfield microscopy image or a fluorescence image. Typically, the image is a multichannel image obtained by fluorescence microscopy from a tissue sample having been stained with one or more biomarker specific stains having different colors.”, Page 37, Lines 6-9 disclose the tissue sample is stained and on page 37, Lines 12-17 disclose that the identification of cell types within the area are determined by reactivity to the stain); or each of the tiles, calculating a local-density measurement of each of a plurality of biological object types identified within the tile(Page 40, Lines 26-29, “ In the next step, the identified density of tumor cells and immune cells in the image or in the tumor tissue area is determined and used for simulating a Poisson distribution of simulated tumor cells and simulated immune cells, respectively. Thereby, the density of the simulated tumor cells and simulated immune cells is identical to the observed density of tumor cells and immune cells in the tissue”, in this section of the prior art, the density of the tumor and immune cells are determined and used for simulating a Poisson distribution); generating one or more spatial- distribution metrics for the biological object types in the digital pathology image based at least in part on the calculated local-density measurements(Page 41, Lines 1-12, “Then, the distances of all simulated tumor cells to their respectively nearest simulated immune cell are determined 208 and represented as a “reference relative distribution”. Then, the proximity score is computed 210 a function of the determined distances. For example, the distance where - on average - every tumor cell comprises one immune cell in its neighborhood can be determined both for the observed relative distribution and for the reference relative distribution and the difference of these two differences can be used as the proximity score. Then, the image analysis system provides 212 a combined score comprising the proximity score and the immune cell density observed in the image 118 and uses 214 the combined score either for automatically determining the biomedical state (e.g. infiltration state) of the tissue and/or for displaying 216 the combined score on a 2-D score plot as depicted,”, in this section of the prior art, once the Poisson distribution is calculated for the immune and tumor cells, a proximity score is computed based on the distance and the proximity score is used to determine the biomedical state of the tissue. ); and determining a tumor immunophenotype of the digital pathology image based at least in part on the local-density measurements or the one or more spatial-distribution metrics. (Page 40, Lines 5-13, “Next in step 212, the image analysis system combines the proximity score with the density of immune cells measured in the received digital image 118 for providing a “combined score”. Next in step 214, the image analysis system uses the combined score for automatically determining the biomedical state of the tissue sample, e.g. a particular infiltration state. In addition, or alternatively, the image analysis system displays in step 216 the combined score obtained for the tissue sample on a display 102 of the image analysis system to a user. Thereby, the user is enabled to visually assess the current biomedical state of the tissue and make according conclusions regarding the progress of the disease and suitable treatment options.“, as disclosed in this section of the prior art, the local density measurements of the immune cells are measured and a proximity score is generated and used to determine the biomedical state of the tissue whether there is a tumor within the sample and a treatment can be determined for the state of the tissue.)
Gaire does not explicitly teach subdividing the digital pathology image into a plurality of tiles;
Yoo teaches subdividing the digital pathology image into a plurality of tiles (¶[0101], “the processor may determine that at least some patches (e.g., patches in which items associated with cancer and immune cells are detected) from among a plurality of patches (e.g., patches of size 1 mm2) generated by dividing the pathology slide image into N grids (where N is any natural number) are the region of interest (e.g., at least some regions in the pathology slide image).”, as disclosed in ¶[0101], the pathological image is divided into N grids which represent regions of interest in the pathological image.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the claimed invention as taught by Gaire with Yoo in order to divide the digital image into a plurality of tiles/regions. One skilled in the art would have been motivated to modify Gaire in this manner in order to objectively analyze the immune environment around the cancer cells, the predictive rate for whether or not a patient will respond to the immune checkpoint inhibitor can be improved. (Yoo, ¶[0027]
Regarding Claim 20, Gaire teaches a non-transitory computer-readable medium comprising instructions that, when executed by one or more data processors of one or more computing devices(Figure 1, element 104 discloses a processor and element 106 discloses a memory and element 108 discloses a non-transitory computer readable medium), cause the one or more processors to:
receive a digital pathology image that depicts a section of a biological sample (Page 38, Lines 29-30 – Page 39, Line 1, “The method can be implemented in and performed for example by an image analysis system 100 as depicted in figure 1. In a first step 202, the image analysis system 100 receives a digital image of a tissue sample.”, in this section of the prior art, an image analysis system receives a digital image of a tissue sample for determining tumor and immune cells in an area as depicted in step 204.), wherein the digital pathology image comprises regions displaying reactivity to a plurality of stains(Page 37, Lines 6-9 “The image 118 can be a brightfield microscopy image or a fluorescence image. Typically, the image is a multichannel image obtained by fluorescence microscopy from a tissue sample having been stained with one or more biomarker specific stains having different colors.”, Page 37, Lines 6-9 disclose the tissue sample is stained and on page 37, Lines 12-17 disclose that the identification of cell types within the area are determined by reactivity to the stain); or each of the tiles, calculate a local-density measurement of each of a plurality of biological object types identified within the tile(Page 40, Lines 26-29, “ In the next step, the identified density of tumor cells and immune cells in the image or in the tumor tissue area is determined and used for simulating a Poisson distribution of simulated tumor cells and simulated immune cells, respectively. Thereby, the density of the simulated tumor cells and simulated immune cells is identical to the observed density of tumor cells and immune cells in the tissue”, in this section of the prior art, the density of the tumor and immune cells are determined and used for simulating a Poisson distribution); generate one or more spatial- distribution metrics for the biological object types in the digital pathology image based at least in part on the calculated local-density measurements(Page 41, Lines 1-12, “Then, the distances of all simulated tumor cells to their respectively nearest simulated immune cell are determined 208 and represented as a “reference relative distribution”. Then, the proximity score is computed 210 a function of the determined distances. For example, the distance where - on average - every tumor cell comprises one immune cell in its neighborhood can be determined both for the observed relative distribution and for the reference relative distribution and the difference of these two differences can be used as the proximity score. Then, the image analysis system provides 212 a combined score comprising the proximity score and the immune cell density observed in the image 118 and uses 214 the combined score either for automatically determining the biomedical state (e.g. infiltration state) of the tissue and/or for displaying 216 the combined score on a 2-D score plot as depicted,”, in this section of the prior art, once the Poisson distribution is calculated for the immune and tumor cells, a proximity score is computed based on the distance and the proximity score is used to determine the biomedical state of the tissue. ); and determine a tumor immunophenotype of the digital pathology image based at least in part on the local-density measurements or the one or more spatial-distribution metrics. (Page 40, Lines 5-13, “Next in step 212, the image analysis system combines the proximity score with the density of immune cells measured in the received digital image 118 for providing a “combined score”. Next in step 214, the image analysis system uses the combined score for automatically determining the biomedical state of the tissue sample, e.g. a particular infiltration state. In addition, or alternatively, the image analysis system displays in step 216 the combined score obtained for the tissue sample on a display 102 of the image analysis system to a user. Thereby, the user is enabled to visually assess the current biomedical state of the tissue and make according conclusions regarding the progress of the disease and suitable treatment options.“, as disclosed in this section of the prior art, the local density measurements of the immune cells are measured and a proximity score is generated and used to determine the biomedical state of the tissue whether there is a tumor within the sample and a treatment can be determined for the state of the tissue.)
Gaire does not explicitly teach segment the digital pathology image into a plurality of tiles;
Yoo teaches segment the digital pathology image into a plurality of tiles; (¶[0101], “the processor may determine that at least some patches (e.g., patches in which items associated with cancer and immune cells are detected) from among a plurality of patches (e.g., patches of size 1 mm2) generated by dividing the pathology slide image into N grids (where N is any natural number) are the region of interest (e.g., at least some regions in the pathology slide image).”, as disclosed in ¶[0101], the pathological image is divided into N grids which represent regions of interest in the pathological image.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the claimed invention as taught by Gaire with Yoo in order to divide the digital image into a plurality of tiles/regions. One skilled in the art would have been motivated to modify Gaire in this manner in order to objectively analyze the immune environment around the cancer cells, the predictive rate for whether or not a patient will respond to the immune checkpoint inhibitor can be improved. (Yoo, ¶[0027])
Claims 3 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Gaire et al. (WO 2020083970 A1) in view of Yoo et al. US PG-Pub(US 20220036971 A1) in view of Caron et al. US PG-Pub(US 20220335606 A1).
Regarding Claim 3, while the combination of Gaire and Yoo teach the method of claim 1, where Gaire further teaches wherein the biological object types comprise tumor cells and immune cells(Page 40, Lines 26-29 disclose tumor and immune cell are determined.),
However, Gaire and Yoo do not explicitly teach the tumor immunophenotype comprises: desert when, for each of the tiles, the local-density measurement of the immune cells is less than an immune-cell-density threshold
excluded when, for one or more of the tiles, the local-density measurement of the tumor cells is less than a tumor-cell-density threshold and the local-density measurement of the immune cells is greater than or equal to the immune-cell-density threshold
or inflamed when, for one or more of the tiles, the local-density measurement of the tumor cells is greater than or equal to the tumor-cell-density threshold and the local-density measurement of the immune cells is greater than or equal to the immune-cell-density threshold.
Caron teaches desert when, for each of the tiles, the local-density measurement of the immune cells is less than an immune-cell-density threshold (¶[0118], “If the TILS falls below the TILS threshold value, the computing device determines whether the larger of the NTILS or the NTILS_margin falls below a second threshold value, which may be referred to as the max NTILS-NTILS_margin threshold value. If the larger of the NTILS or the NTILS_margin falls below the max NTILS-NTILS_margin threshold value, the patient is classified into the immune desert phenotype 1104”, ¶[0118] discloses determining the phenotype to be desert when it is below a certain density threshold. ¶[0057] discloses “a density based tumor infiltrating lymphocytes score, a density based non-tumor infiltrating lymphocytes score, and a non-tumor infiltrating lymphocytes at tumor margin score. The tumor infiltrating lymphocytes score (TILS) measures the density of tumor infiltrating lymphocytes within the tumor cells.”, a density score is used to determine the phenotype classification. );
excluded when, for one or more of the tiles, the local-density measurement of the tumor cells is less than a tumor-cell-density threshold and the local-density measurement of the immune cells is greater than or equal to the immune-cell-density threshold (¶[0118], “If the larger of the NTILS or the NTILS_margin meets or exceeds the max NTILS-NTILS_margin threshold value and the TILS falls below the TILS threshold value, the patient is classified into the immune-excluded tumor-immune phenotype 1106.”, as disclosed in ¶[0118] determining the phenotype to be excluded when the tumor cell density is less than a threshold and immune cell density higher a certain density threshold.)
or inflamed when, for one or more of the tiles, the local-density measurement of the tumor cells is greater than or equal to the tumor-cell-density threshold and the local-density measurement of the immune cells is greater than or equal to the immune-cell-density threshold. (¶[0118], “The computing device determines whether the TILS falls below a first threshold value, which may be referred to as a TILS threshold value herein. If the TILS meets or exceeds the TILS threshold value, the patient is classified into the inflamed tumor-immune phenotype”, ¶[0118] discloses the phenotype is inflamed when the tumor cell and immune cell density is greater than a threshold.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the claimed invention as taught by Gaire and Yoo with Caron in order to determine tumor phenotypes by comparing to a threshold. One skilled in the art would have been motivated to modify Gaire and Yoo in this manner in order to classify a patient into a tumor-immune phenotype. (Caron, ¶[0002])
Regarding Claim 19, the combination of Gaire and Yoo teach the digital pathology image processing system of claim 18, where Gaire further teaches wherein the biological object types comprise tumor cells and immune cells(Page 40, Lines 26-29 disclose tumor and immune cell are determined.),
However, Gaire and Yoo do not explicitly teach the tumor immunophenotype comprises: desert when, for each of the tiles, the local-density measurement of the immune cells is less than an immune-cell-density threshold
excluded when, for one or more of the tiles, the local-density measurement of the tumor cells is less than a tumor-cell-density threshold and the local-density measurement of the immune cells is greater than or equal to the immune-cell-density threshold
or inflamed when, for one or more of the tiles, the local-density measurement of the tumor cells is greater than or equal to the tumor-cell-density threshold and the local-density measurement of the immune cells is greater than or equal to the immune-cell-density threshold.
Caron teaches desert when, for each of the tiles, the local-density measurement of the immune cells is less than an immune-cell-density threshold (¶[0118], “If the TILS falls below the TILS threshold value, the computing device determines whether the larger of the NTILS or the NTILS_margin falls below a second threshold value, which may be referred to as the max NTILS-NTILS_margin threshold value. If the larger of the NTILS or the NTILS_margin falls below the max NTILS-NTILS_margin threshold value, the patient is classified into the immune desert phenotype 1104”, ¶[0118] discloses determining the phenotype to be desert when it is below a certain density threshold. ¶[0057] discloses “a density based tumor infiltrating lymphocytes score, a density based non-tumor infiltrating lymphocytes score, and a non-tumor infiltrating lymphocytes at tumor margin score. The tumor infiltrating lymphocytes score (TILS) measures the density of tumor infiltrating lymphocytes within the tumor cells.”, a density score is used to determine the phenotype classification. );
excluded when, for one or more of the tiles, the local-density measurement of the tumor cells is less than a tumor-cell-density threshold and the local-density measurement of the immune cells is greater than or equal to the immune-cell-density threshold (¶[0118], “If the larger of the NTILS or the NTILS_margin meets or exceeds the max NTILS-NTILS_margin threshold value and the TILS falls below the TILS threshold value, the patient is classified into the immune-excluded tumor-immune phenotype 1106.”, as disclosed in ¶[0118] determining the phenotype to be excluded when the tumor cell density is less than a threshold and immune cell density higher a certain density threshold.)
or inflamed when, for one or more of the tiles, the local-density measurement of the tumor cells is greater than or equal to the tumor-cell-density threshold and the local-density measurement of the immune cells is greater than or equal to the immune-cell-density threshold. (¶[0118], “The computing device determines whether the TILS falls below a first threshold value, which may be referred to as a TILS threshold value herein. If the TILS meets or exceeds the TILS threshold value, the patient is classified into the inflamed tumor-immune phenotype”, ¶[0118] discloses the phenotype is inflamed when the tumor cell and immune cell density is greater than a threshold.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the claimed invention as taught by Gaire and Yoo with Caron in order to determine tumor phenotypes by comparing to a threshold. One skilled in the art would have been motivated to modify Gaire and Yoo in this manner in order to classify a patient into a tumor-immune phenotype. (Caron, ¶[0002])
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Gaire et al. (WO 2020083970 A1) in view of Yoo et al. US PG-Pub(US 20220036971 A1) in view of Svekolkin et al. US PG-Pub(US 20210279866 A1).
Regarding Claim 5, while the combination of Gaire and Yoo teach the method of claim 1, they do not explicitly teach wherein the one or more spatial-distribution metrics comprise: a Jaccard index; a Sorensen index; a Bhattacharyya coefficient; a Moran's index; a Geary's contiguity ratio; a Morisita-Horn index; or a metric defined based on a hotspot/coldspot analysis.
Svekolkin teaches wherein the one or more spatial-distribution metrics comprise: a Jaccard index; a Sorensen index; a Bhattacharyya coefficient; a Moran's index; a Geary's contiguity ratio; a Morisita-Horn index; or a metric defined based on a hotspot/coldspot analysis. (¶[0272], “The Jaccard index shows cell detection accuracy and cell shape reproduction accuracy. The value is determined by matching cells between the ground truth data and the cell segmentation prediction using the Hungarian algorithm (where each cell has either zero or one connections). For each matched pair, the Jaccard index is calculated between them, and the final result is computed as a sum of the values divided by the maximum number of cells (the number of cells in the ground truth mask or in the prediction mask).”, ¶[0272] disclose using a Jaccard index as a metric for spatial distribution of cells.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the claimed invention as taught by Gaire and Yoo with Svelkolkin in order to determine a Jaccard index as a spatial distribution metric. One skilled in the art would have been motivated to modify Gaire and Yoo in this manner in order to show cell detection accuracy and cell shape reproduction accuracy. (Svekolkin, ¶[0272])
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Gaire et al. (WO 2020083970 A1) in view of Yoo et al. US PG-Pub(US 20220036971 A1) in view of Stumpe et al. US PG-Pub(US 20210064845 A1).
Regarding Claim 14, the combination of Gaire and Yoo teach the method of claim 1, they do not explicitly teach further comprising: identifying one or more tumor regions in the digital pathology image comprising: providing a user interface for display comprising the digital pathology image and one or more interactive elements; and receiving a selection of the one or more tumor regions through interaction with the one or more interactive elements.
Stumpe teaches further comprising: identifying one or more tumor regions in the digital pathology image comprising: providing a user interface for display comprising the digital pathology image and one or more interactive elements(¶[0010] discloses a workstation with a processing unit and display with user interface tools); and receiving a selection of the one or more tumor regions through interaction with the one or more interactive elements.([0010] “In still another aspect, a workstation is provided which comprises a processing unit and a display. The display is configured to display digital magnified images of a single slide containing a tissue specimen stained with, e.g., a hematoxylin and eosin (H+E) stain, an immunohistochemical (IHC) stain, or some other staining agent. The workstation is configured with either (a) user interface tools by which an operator inspecting the registered digital magnified images on the display may annotate a digital magnified image of the tissue specimen stained with the staining agent so as to form a closed polygon or other shape to mark a region of said image containing region of interest (e.g., tumor cells) to thereby create a mask.” , ¶[0010] discloses a user interface in which a user can annotate and create a mask of tumor regions in the image for processing.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the claimed invention as taught by Gaire and Yoo with Stumpe in order to allow the user to select tumor regions in the image. One skilled in the art would have been motivated to modify Gaire and Yoo in this manner in order to aid in diagnosis, clinical decision support and for making predictions for the patient providing the tissue sample, such as predicting survival, or response to treatment. (Stumpe, ¶[0003])
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Gaire et al. (WO 2020083970 A1) in view of Yoo et al. US PG-Pub(US 20220036971 A1) in view of Jaber et al. US PG-Pub(US 20220375602 A1).
Regarding Claim 16, the combination of Gaire and Yoo teach the method of claim 15, where Gaire further teaches wherein determining the tumor immunophenotype of the image and generating the one or more spatial-distribution metrics (Page 40, Lines 5-13 discloses the local density measurements of the immune cells are measured and a proximity score is generated and used to determine the biomedical state of the tissue whether there is a tumor within the sample and a treatment can be determined for the state of the tissue.)
However, Gaire and Yoo do not explicitly teach using a trained machine- learned model, the trained machine-learned model having been trained using a set of training elements, each of the set of training elements corresponding to another subject having a similar medical condition and for which an outcome of the medical condition is known.
Jaber teaches using a trained machine- learned model, the trained machine-learned model having been trained using a set of training elements, each of the set of training elements corresponding to another subject having a similar medical condition and for which an outcome of the medical condition is known. (¶[0078], “ In some example embodiments, the classifier 600 may be trained to determine one or more outcomes of an individual whose tissue is shown in an image 102 based on the determination of the region of interest 702 in the image 102. For example, a human labeler 602 may provide one or more outcome labels, such as a diagnosis of the individual based on the region of interest 702, a prognosis of the individual based on the region of interest 702, or a treatment of the individual based on the region of interest 702. The classifier 600 may be trained to generate outcomes of the individuals based on the images 102 that match the outcome labels provided by the human labeler 602.”, ¶[0078] discloses training a machine learning model to determine an outcome based on the region of interest in the image.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the claimed invention as taught by Gaire and Yoo with Jaber in order to train a machine learning model to determine a treatment for each region of interest in the image. One skilled in the art would have been motivated to modify Gaire and Yoo in this manner in order to improve the automated provision of medical diagnostic capabilities. (Jaber, ¶[0106])
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HAN D HOANG whose telephone number is (571)272-4344. The examiner can normally be reached Monday-Friday 8-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, JOHN M VILLECCO can be reached at 571-272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HAN HOANG/Examiner, Art Unit 2661