DETAILED ACTION
Applicant’s response, filed 22 April 2026 has been fully considered. The following rejections and/or objections are either reiterated or newly applied. They constitute the complete set presently being applied to the instant application.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims 2-3 and 13 are cancelled. It is noted Applicant’s claim amendments include strike-through text in cancelled claims 2-3, and thus fails to comply with 37 CFR. 1.121, which states no claim text shall be presented for any claim in the claim listing with the status of "canceled" or "not entered”. In the interest of compact prosecution, the claims are being examined herein; however, future claim amendments should not include text for any claim with the status of “canceled”.
Claims 1, 4-12, and 14-20 are pending.
Claims 1, 4-12, and 14-20 are rejected.
Claim 16 is objected to.
Priority
The effective filing date of the claimed invention is 08 May 2020.
Claim Objections
Claim 16 is objected to because of the following informalities. This objection is newly recited and necessitated by claim amendment.
Claim 16 recites “determining, using a trained artificial neural network model…a plurality of regions of interest (ROIs)…based on detection results…from the first trained artificial neural network model”. To increase clarity, the claim should be amended to refer to “using a second trained artificial neural network model”, similar to claims 1 and 15; and
Claim 16 recites “…receiving…; and outputting…” in the last two limitations, which is grammatically incorrect and should recite “receive…; and output…”.
Appropriate correction is required.
Claim Interpretation
Claims 1 and 16 recite “excluding regions other than the plurality of ROIs from the pathology slide image”. Applicant’s remarks at pg. 10, para. 1 state that support for the amendments can be found in at least para. [0020] and [0064]. These paragraphs disclose that “instead of analyzing the entire pathology slide image the information processing system and/or the user terminal may perform processing…only on the ROIs while excluding the regions where analysis is unnecessary”. Therefore, in light of Applicant’s specification, the limitation of “excluding regions other than the plurality of ROIs from the pathology slide image” is interpreted to encompass excluding regions other than the plurality of ROIs from analysis of the pathology slide image and does not require altering the image itself to remove the excluded regions.
Claim 4 recites “…wherein the ROI extraction model is trained to output a reference ROI upon input of at least one of…”, which is a product by process limitation interpreted to describe the process in which the ROI extraction model was previously trained. See MPEP 2113 I. "[E]ven though product-by-process claims are limited by and defined by the process, determination of patentability is based on the product itself.
Claim Rejections - 35 USC § 112(b)
The previous rejection of claims 1-4, 8-12, and 14-20 under 35 U.S.C. 112(b) in the Office action mailed 23 Jan. 2026 has been withdrawn in view of claim amendments and cancellations received 22 April 2026.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 5-7, 9-10, and 12 are rejected under 35 U.S.C. 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, regards as the invention. This rejection is newly recited and necessitated by claim amendment.
Claim 5, and claims dependent therefrom, are indefinite for recitation of “the immune phenotype” in lines 4 and 6. Claim 1, from which claim 5 depends, recites “determining immune phenotypes for the plurality of ROIs”. Therefore, it is unclear if “the immune phenotype” of claim 5 is referring to the overall immune phenotype or one of the immune phenotypes of the immune phenotype. If “the immune phenotype” is intended to refer to one of the immune phenotypes it is further unclear which immune phenotype is being referenced. For purpose of examination, the immune phenotype is interpreted to refer to any determined immune phenotype.
Claims 6-7, 9-10, and 12 are indefinite for recitation of “wherein the outputting…”. Claim 1, from which claims 6-7, 9-10, and 12 ultimately depend, recites “outputting…through the display device, the proportion…” and “outputting a first visual representation…corresponding to the user input”. As a result, it is unclear which outputting step of claim 1, claims 6-7, 9-10, and 12 are intending to refer to and subsequently further limit. Clarification is requested. For purpose of examination, the claims will be interpreted to refer to either outputting step.
Response to Arguments
Applicant's arguments filed 22 April 2026 regarding 35 U.S.C. 112(b) have been fully considered but they are not persuasive.
Applicant remarks that claims 1 and 16 were amended and thus the rejection of the claims under 35 U.S.C. 112(b) should be withdrawn (Applicant’s remarks at pg. 11, para. 6).
This argument is not persuasive. While the amendments to claims 1 and 16 addressed the 112(b) issue regarding “the corresponding ROI”, Applicant did not amend claim 5 to address the 112(b) issue relating to “the immune phenotype”. Thus the rejection of claims 5 and dependent claims 6-7 under 35 U.S.C. 112(b) is maintained.
Claim Rejections - 35 USC § 101
The rejection of claims 2-3 and 13 under 35 U.S.C. 101 in the Office action mailed 23 Jan. 2026 has been withdrawn in view of the cancellation of these claims received 22 April 2026.
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 4-12, and 14-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to one or more judicial exceptions without significantly more. Any newly recited portion is necessitated by claim amendment.
The Supreme Court has established a two-step framework for this analysis, wherein a claim does not satisfy § 101 if (1) it is “directed to” a patent-ineligible concept, i.e., a law of nature, natural phenomenon, or abstract idea, and (2), if so, the particular elements of the claim, considered “both individually and as an ordered combination,” do not add enough to “transform the nature of the claim into a patent-eligible application.” Elec. Power Grp., LLC v. Alstom S.A., 830 F.3d 1350, 1353 (Fed. Cir. 2016) (quoting Alice, 134 S. Ct. at 2355). Applicant is also directed to MPEP 2106.
Step 1: The instantly claimed invention (claims 1 and 15-16 being representative) is directed to a method, product, and system for providing information associated with an immune phenotype. Therefore, the instantly claimed invention falls into one of the four statutory categories. [Step 1: YES]
Step 2A: First it is determined in Prong One whether a claim recites a judicial exception, and if so, then it is determined in in Prong Two if the recited judicial exception is integrated into a practical application of that exception.
Step 2A, Prong 1: Under the MPEP § 2106.04, the Step 2A (Prong 1) analysis requires determining whether a claim recites an abstract idea, law of nature, or natural phenomenon.
Claims 1, 15, and 16 recite the following steps which fall under the mental processes and/or mathematical concept groupings of abstract ideas:
detecting/detect one or more target items including at least one of immune cells, tumor cells, a cancer area and a stroma area in the pathology slide image…by:…detecting…the one or more target items in the pathology slide image…;
determining/determine…a plurality of regions of interest (ROIs) for determining immune phenotypes based on detection results of the one or more target items…and excluding regions other than the plurality of ROIs from the pathology slide image;
determining/determine the immune phenotypes for the plurality of ROIs in the pathology slide image, based on at least one of a density, a number, or a spatial distribution of the detected immune cells within a caner area or a stroma area of each of the plurality of ROIs;
determining/determine a proportion of the immune phenotypes among the plurality of ROIs; and
generating/generate an immune phenotype distribution map representing the immune phenotypes for the plurality of ROIs;
The identified claim limitations falls into one of the groups of abstract ideas of mental processes and/or mathematical concepts for the following reasons. In this case, the step of detecting target items, including particular cell types, encompasses analyzing different stain colors of a pathology slide image to infer cell types, which is a mental process similar to what a histologist would perform. The step of determining a plurality of regions of interest (ROIs) for determining immune phenotypes based on detection results of the one or more target items encompasses analyzing an image and determining boundaries on the image around areas of interest including the detected target items, which is a mental process. Furthermore, excluding regions other than the plurality of ROIs from analysis of the pathology slide image amounts to a mere analysis of information (only the ROIs) which can be practically performed in the mind. Determining immune phenotypes for the ROIs based on a density of detected immune cells within a caner or stroma area of an ROI amounts to a mere analysis of data involving, for example, determining an ROI is immune excluded if density of immune cells is low. Determining a proportion of the immune phenotypes can be practically performed in the mind by, for example, determining a proportion of the regions of interest classified as an immune-excluded phenotype; determining the proportion further recites a mathematical concept of a mathematical calculation (e.g. determining a fraction/proportion). Furthermore, generating an immune phenotype distribution map representing the phenotypes for the ROIs can be practically performed in the mind aided with pen and paper by, for example, organizing phenotype scores into a heat map. Overall, the mental process recited in the claims is analogous a claim to "collecting information, analyzing it, and displaying certain results of the collection and analysis," where the data analysis steps are recited at a high level of generality such that they could practically be performed in the human mind, Electric Power Group v. Alstom, S.A., 830 F.3d 1350, 1353-54, 119 USPQ2d 1739, 1741-42 (Fed. Cir. 2016). That is, other than reciting the above limitations are performed by a computing device or processor, nothing precludes the steps from being practically performed in the mind. Therefore, these limitations recite a mental process. See MPEP 2106.04(a)(2) III.
Dependent claims 4-5, 8, 11, 14, 17, and 20 further recite an abstract idea and/or further limit the abstract idea of claims 1 or 16. Dependent claim 4 further limits the mental process of determining the plurality of ROIs in claim 1 to include the recited shape, and thus is part of the abstract idea of claim 1. Dependent claims 5 and 17 further recite the mental process of generating an image including a visual representation corresponding to the immune phenotype, where the immune phenotype includes at least one of immune inflamed, immune excluded, or immune desert. Dependent claims 8 and 20 further recites the mental process of obtaining one or more immune phenotype scores for the one or more ROIs, wherein the scores include at least one of a score for immune inflamed, a score for immune excluded, or a score for immune desert, and generating the image including a second visual representation corresponding to one or more immune phenotype scores. Dependent claim 11 further recites the mental process of obtaining a feature associated with one or more immune phenotypes for the one or more ROIs, the feature vector associated with the one or more immune phenotypes includes at least one of a statistical value or a vector. Dependent claim 14 further recites the mental process of generating an image indicative of the detection result.
The claims further recite the law of nature of a natural correlation between the presence of immune or tumor cells, including cell densities, distributions or numbers, in a tissue sample and an immune phenotype, similar to the natural relationship between a patient’s CYP2D6 metabolizer genotype and the risk that the patient will suffer QTc prolongation after administration of a medication called iloperidone, Vanda Pharmaceuticals Inc. v. West-Ward Pharmaceuticals. See MPEP 2106.04(b). Therefore, claims 1, 4-12, and 14-20 recite an abstract idea and law of nature.. [Step 2A, Prong 1: YES]
Step 2A: Prong 2: Under the MPEP § 2106.04, the Step 2A, Prong 2 analysis requires identifying whether there are any additional elements recited in the claim beyond the judicial exception(s), and evaluating those additional elements to determine whether they integrate the exception into a practical application of the exception. This judicial exception is not integrated into a practical application for the following reasons.
Dependent claims 4-5, 8, 11, 17, and 20 do not recite any elements in addition to the judicial exception.
The additional elements of claims 1 and 15-16 include:
acquiring, by the at least one computing device, the pathology slide image digitized by a whole-slide imaging scanner (claims 1 and 15);
using a first trained artificial neural network model stored in the memory and executed by the processor …with the first trained artificial neural network model (claims 1 and 15)/ with the first trained artificial neural network model (claim 16);
inputting the pathology slide image into the artificial neural network (i.e. data input);
using a second trained artificial neural network model stored in the memory and executed by the processor…(claims 1 and 15-16);
outputting/output, through the display device, the overall immune phenotype and the immune phenotype distribution map overlaid on the pathology slide image;
receiving/receive, through a user device, user input that selects information associated with the one or more target items;
outputting a first visual representation in a region of the pathology slide image corresponding to the user input;
a computing device comprising a processor, a memory, and display device (claims 1 and 16); and
a non-transitory computer-readable medium (claim 15).
The additional elements of dependent claims 6-7, 9-10, 12, 14, and 18-19 include:
outputting/output the plurality of ROIs in the pathology slide image together with the image including the visual representation (claims 6, 9, 12, 18);
the outputting includes overlaying/overlay the image including the visual representation on the plurality of ROIs in the pathology slide image (claims 7, 10, and 19); and
outputting the image indicative of the detection result for one or more target items (claim 14);
The additional elements of a processor, memory, non-transitory computer-readable medium, inputting data, receiving data, and outputting data or an image (i.e. displaying data) are generic computer components and/or processes. The additional elements also recite an artificial neural network to perform the abstract idea of detecting target items and a neural network for determining the ROIs. The courts have found the use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application. See Affinity Labs v. DirecTV, 838 F.3d 1253, 1262, 120 USPQ2d 1201, 1207 (Fed. Cir. 2016) (cellular telephone); TLI Communications LLC v. AV Auto, LLC, 823 F.3d 607, 613, 118 USPQ2d 1744, 1748 (Fed. Cir. 2016) (computer server and telephone unit). With further regard to the use of the first and second artificial neural networks, the claims do not recite any details that places limits on how the trained neural networks function, and instead these additional elements merely serves to generally link the abstract idea of detecting target items of an image to the technological environment of neural networks, which is not sufficient to integrate the recited judicial exception into a practical application. See MPEP 2106.05(h).
Furthermore, the additional element of outputting information, including a first visual representation in a region of the pathology slide image corresponding to the user input and one or more ROIs overlayed with a visual representation only serves to output data generated by the abstract idea, which amounts to insignificant extra-solution activity.
Last, it is not apparent that the additional elements, alone or in combination with the judicial exception, improves computer technology or any other technology. Applicant’s specification at pg. 2 lines 3-12 disclose a user (e.g. doctor, patient, and the like) may be provided with immune response information through a pathology slide image to aid in predicting the response of a user to an immune checkpoint inhibitor, which reflects an improvement in the abstract idea (e.g. providing better information) rather than an improvement to technology.
Therefore, the additionally recited elements merely invoke computers as a tool, generally link the abstract idea to a technological environment, and/or amount to insignificant extra-solution activity and, as such, the claims as a whole do no integrate the abstract idea into practical application. Thus, claims 1, 4-12, and 14-20 are directed to an abstract idea and law of nature. [Step 2A, Prong 2: NO]
Step 2B: In the second step it is determined whether the claimed subject matter includes additional elements that amount to significantly more than the judicial exception. See MPEP § 2106.05.
The claims do not include any additional steps appended to the judicial exception that are sufficient to amount to significantly more than the judicial exception for the following reasons.
Dependent claims 4-5, 8, 11, 17, and 20 do not recite any elements in addition to the judicial exception. The additional elements of claims 1, 6-7, 9-10, 12, 14-16, and 18-19 are identified above.
First, the additional elements of a processor, memory, non-transitory computer-readable medium, inputting data, receiving data, outputting data or an image (i.e. displaying data), and first and second neural networks are conventional computer components and/or processes. The courts have found the use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not provide significantly more. See Affinity Labs v. DirecTV, 838 F.3d 1253, 1262, 120 USPQ2d 1201, 1207 (Fed. Cir. 2016) (cellular telephone); TLI Communications LLC v. AV Auto, LLC, 823 F.3d 607, 613, 118 USPQ2d 1744, 1748 (Fed. Cir. 2016) (computer server and telephone unit).
Further regarding the first and second neural networks, MPEP 2106.05(h) states, as explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
Furthermore, the use of neural networks in image processing and identifying regions are well-understood, routine, and conventional, as supported by Madabhushi (Image analysis and machine learning in digital pathology: Challenges and opportunities, 2016, Medical Image Analysis, pg. 170-175; previously cited) and Xing et al. (Robust Nucleas/Cell Detection and Segmentation in Digital Pathology and Microscopy Images: A comprehensive Review, 2016, IEE Reviews in Biomedical Engineering, 9, pg. 234-263; newly cited). Madabhushi reviews the use of computational image analysis tools for digital pathology images (Abstract), and discloses over the last decade, digital pathology has transformed computational image research and discloses a more recent class of approaches utilize deep learning techniques of neural networks to process digital pathology images (pg. 171, col. 1, para. 3 to pg. 171, col. 2, para. 3-6; Fig. 1). Xing similarly reviews the digital pathology and computer-aided methods for nucleus/cell detection and segmentation of pathology images (Abstract). Xing discloses that deep neural networks are vary popular in nuclease or cell detection (pg. 242, col. 2, para. 2) and also have attracted significant attention in pathology image segmentation for determining object boundaries (pg. 244, col. 1, para. 2 to col. 2, para. 2; pg. 245, col. 2, para. 2).
The additional element of outputting an image with information overlayed in a pathology slide image is well-understood, routine, and conventional. This position is supported by Parra et al. (State-of-the-Art of Profiling Immune Contexture in the Era of Multiplexed Staining and Digital Analysis to Study Paraffin Tumor Tissues, 2019, Cancers, 11, 247, pg. 1-23; previously cited). Parra reviews the profiling of immune contexture in multiplex staining and digital analysis of images (Abstract). Parra discloses that multiplexed imaging platforms have arisen as important tools to provide information about the cancer microenvironment, citing 5 different studies (pg. 1, para. 1), and further discloses a plurality of commercially available, well-known, image analysis software packages which allow overlaying information over the images (Table 2, e.g. color; pg. 12, para. 1). Parra discloses further discloses multiplex immunofluorescence microphotography with images overlayed with information on different cell types (Figure 4-5), and can align information from various images of regions of interest (pg. 11, para. 3).
Therefore, taken alone, the additional elements do not amount to significantly more than the above-identified judicial exception(s). Even when viewed as a combination, the additional elements fail to transform the exception into a patent-eligible application of that exception. Thus, the claims as a whole do not amount to significantly more than the exception itself. [Step 2B: NO]
Therefore, the instantly rejected claims are not drawn to eligible subject matter as they are directed to an abstract idea and natural correlation without significantly more. For additional guidance, applicant is directed generally to applicant is directed generally to the MPEP § 2106.
Response to Arguments
Applicant's arguments filed 22 April 2026 regarding 35 U.S.C. 101 have been fully considered but they are not persuasive.
Applicant remarks that claim 1 was amended to recite “determining, using a second trained artificial neural network model stored in the memory and executed by the processor, a plurality of regions of interest (ROI)…”, and the claimed features cannot be practically performed in the mind even with a paper and pencil given a human mind cannot practically use a second trained artificial neural network model as claimed (Applicant’s remarks at pg. 13, para. 1-4).
This argument is not persuasive. The step of determining a plurality of regions of interest and excluding regions recite a mental process for the reasons discussed in the above rejection. The limitation regarding “using a second trained artificial neural network model” is treated as an additional element in the above rejection. However, the claims do not recite any details that places limits on how the trained neural networks function, and therefore this limitation merely invokes computers a tool to perform the abstract idea, and merely serves to generally link the abstract idea of detecting target items of an image to the technological environment of neural networks, which is not sufficient to integrate the recited judicial exception into a practical application. See MPEP 2106.05(f) and MPEP 2106.05(h). Applicant is also directed to the example 47 of the AI-related SME examples issued in 2024, which provide examples in which applying a neural network to perform an abstract idea does not integrate the recited judicial exception into a practical application or provide significantly more.
Applicant remarks that claim 1 is directed to a specific technical improvement in computer technology and integrates the abstract idea into a practical application because the claim as a whole when read in light of the specification solves the technical problem related to inefficient processing of massive digital pathology images by detecting one or more target items, detecting one or more regions of interest, and excluding regions other than the plurality of ROIs from the pathology slide image (Applicant’s remarks at pg. 13, para. 4 to 16, para. 1). Applicant further remarks that determining an immune phenotype and/or calculating an immune phenotype score is performed only on the ROIs while excluding other regions, which results in saving computer resources and achieves more accurate results (Applicant’s remarks at pg. 16, para. 2).
This argument is not persuasive. First, it is noted an improvement cannot be provided by the judicial exception alone. See MPEP 2106.05(a). Furthermore, in computer-related technologies, the examiner should determine whether the claim purports to improve computer capabilities or, instead, invokes computers merely as a tool. Enfish, LLC v. Microsoft Corp., 822 F.3d 1327, 1336, 118 USPQ2d 1684, 1689 (Fed. Cir. 2016).
In the instant case, simply analyzing less information by processing regions of interest does not improve computer technology, but rather uses a generic computer to process less information. Therefore, the alleged improvement is provided by the abstract idea of determining the ROIs and generating the immune phenotypes of the ROIs, rather than an additional element of the claim. Regardless, determining regions of interest in a pathology slide image is conventional in the field (see Xing in step 2B above) and thus Applicant’s use of regions of interest does not provide an improvement in even the analysis of pathology slide images.
Applicant remarks the claims are analogous to example 47, in which claim 3 provides improved network security by blocking traffic from a source address, because the instant claims recite a specific improvement in the processing of digital images (Applicant’s remarks at pg. 17, para. 1 to pg. 18, para. 2).
This argument is not persuasive. As discussed above, simply analyzing less information by processing regions of interest does not improve computer technology, but rather uses a generic computer to process less information; the alleged improvement is provided by the abstract idea of determining the ROIs and generating the immune phenotypes of the ROIs, rather than an additional element of the claim. Claim 3 of example 47 in contrast, recited the additional elements of “dropping the one or more malicious network packets in real time; and (f) blocking future traffic from the source address”, which integrate the recited judicial exception of detecting anomalies into a practical application.
Claim Rejections - 35 USC § 103
The rejection of claims 1-12 and 14-20 under 35 U.S.C. 103 as being unpatentable over Gaire (2019) in view of Saltz (2018) in the Office action mailed 23 Jan. 2026 has been withdrawn in view of claim amendments and cancellations received 22 April 2026. However, after further consideration, a new grounds of rejection is set forth below in view of the claim amendments.
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 4-12, and 14-20 are rejected under 35 U.S.C. 103 as being unpatentable over Gaire (2019) in view of Tomita (2019) and Saltz (2018). This rejection is newly recited and necessitated by claim amendment.
Cited references:
Gaire et al., US 2021/0343009 A1, effectively filed 23 Oct. 2019 (previously cited);
Tomita et al., Attention-based deep neural networks for Detection of Cancerous and Precancerous Esophagus Tissue on Histopathological slides, 2019, JAMA Network Open, 2(11), pg. 1-13 and Suppl. (newly cited); and
Saltz et al., US 2020/0388029 A1, filed 30 Nov. 2018 (previously cited).
Regarding claims 1 and 15-16, Gaire discloses a method for providing information on a biomedical state, including immunophenotyping, of a tissue sample using image-based classification (Abstract; [0001]-[0004), a system comprising a memory, processor, and display device for carrying out the method (FIG. 1), and a memory storing instructions for the method (FIG. 1; [0200]), where in the method comprises the following steps:
Gaire discloses acquiring a digital pathology slide image (claim 1).
Gaire discloses detecting immune cells and tumor cells (i.e. the one or more target items) in the pathology slide image using a connected component analysis and edge detection routines in order to identify pixel blobs representing cells ([0202]) (i.e. detecting one or more target items including at least one of immune cells, tumor cells… in the pathology slide image…).
Gaire discloses identifying sub-areas, including tumor areas, of the image (i.e. determining regions of interest), and that the sub-area may be of a pre-defined size or may be automatically detected by annotating the image where the cells are identified (i.e. based on detection results of the one or more target items) ([0140], e.g. sub-area annotated as tumor-region [0216], e.g. sub-area of pre-defined size; [0239], e.g. multiple tumor regions identified). Gaire further discloses that this ensures that tissue outside the tumor area do not have an impact on the analysis result ([0140]), demonstrating regions other than the tumor areas (i.e. regions of interest) are excluded from analysis of the pathology slide image.
Gaire discloses determining, for the tumor regions, spatial distributions of immune cells represented by a number of CD8+ cells closer to any tumor cell than a given radius r for each of multiple r’s (i.e. multiple immune phenotypes) based on a number of immune cells in each circular region around each tumor cell (i.e. a cancer area) defined by the respective radius r in a tumor region (i.e. a number of cells in a cancer area of a corresponding region of interest) ([0140], e.g. only tumor cells in tumor regions analyzed; [0190]; [0223]; FIG. 7A, e.g. a plurality of immune phenotypes at each radius r).
Gaire discloses determining a proximity score from the number circular regions around a tumor cell having an observed number of immune cells within a given radius from the tumor cell greater or less than an expected number of immune cells in a reference distribution (FIG. 7B; [0229]-[0230]). The proximity score is based on a proportion of the spatial distributions of immune cells for each r (i.e. the immune phenotypes) (e.g. “the number of CD8+ T-cells closer than r” for each r in FIG. 8) being different than a reference and is represented as a percentage (FIG. 7B and FIG. 8, e.g. see area-based delta score, which is a proportion of the immune phenotypes; FIG. 9), demonstrating the determination of a proportion of the immune phenotypes (being different than a reference). In the interest of compact prosecution, it is also noted that Satlz additionally discloses determining a proportion of the tumor composed of lymphocytes ([0313]-[0315]; Table 3) and a proportion of TIL-positive image patches (i.e. a proportion of immune phenotypes) ([0301]).
Gaire discloses generating an image indicative of the immune phenotypes (FIG. 1, e.g. plot generation and display; FIG. 3-4; [0202]-[0203]; [0216]), wherein the image visually represents colors depicting the distribution and proximities of different cell types of immune and tumor cells (i.e. the immune phenotype distribution map) (Figure 3, e.g. see cell distributions and images; [0202], e.g. monochromatic images derived with a color for each cell type). Given the image represents the distribution and proximities of the immune and tumor cells, this is considered to represent the immune phenotypes corresponding to the proximities of immune cells to a tumor cell discussed above.
Gaire discloses displaying (i.e. outputting) the generated image on a display device (claim 19; [0141]-[0142]; [0216]), wherein the outputted image includes monochromatic images from the pathology slide image to generate an image specific to a particular cell biomarker for an immune cell ([0202]-[0203]; [0206], e.g. generated images #118 displayed; FIG. 3 and FIG. 9), which shows overlaying the image including the immune phenotype distribution map (e.g. colors for immune vs tumor cell types) on the pathology slide image. Gaire further discloses displaying the the phenotype distribution map overlaid on the slide image (FIG. 3) and also displaying the proportion (Fig. 8, e.g. see area-based proportion of phenotypes).
Regarding claims 1 and 15-16, Gaire does not disclose the following limitations:
Regarding claims 1 and 15-16¸ Gaire does not disclose detecting the one or more target items uses an artificial neural network stored in memory and executed by the processor by: inputting the pathology slide image into the artificial neural network, and detecting, with the artificial neural network, the one or more target items in the pathology slide image. Gaire further does not disclose a second trained artificial neural network model stored in the memory and executed by the processor is used to determine the regions of interest based on the detection results of the one or more target items.
However, as discussed above, Gaire does disclose automatically detecting immune cells and tumor cells (i.e. the one or more target items) in the pathology slide image using a connected component analysis and edge detection routines in order to identify pixel blobs representing cells ([0202]), and then identifying regions of interest based on annotations of the image where cells are identified (i.e. based on the detection results of the one or more target items ([0140]; [0216]; [0239]).
Furthermore, Tomita overviews a method for using neural networks to process a histopathological slide image (Abstract; pg. 4, para. 4), and discloses performing a first step of extracting grid-based features (i.e. target items) from the pathology slide image using a convolution neural network (i.e. a first artificial neural network for detecting target items) (pg. 4, para. 4; Figure 1 steps A-B), and then a second step of inputting the extracted features into a second attention convolution neural network (CNN) for slide classification of regions of interest (i.e. a second artificial neural network for determining ROIs based on the detected target items) (pg. 3, para. 3; pg. 4, para. 4; pg. 6, para. 1, e.g. attention neural network automatically identifies ROIs; Figure 1). Tomita discloses the attention-based neural network model is carried out by a computer comprising memory (pg. 3, para. 3). Tomita further discloses the disclosed attention-based neural network achieved better accuracy and F1 scores in the classification of all classes (pg. 7, para. 4) and is applicable to high-resolution images without resizing owing to its flexible input design (pg. 9, para. 2).
It would have been prima facie obvious, to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the method of Gaire to have used the two-step CNN and attention based neural network model for the detection of target items and regions of interest, respectively, according to the method of Tomita discussed above. One of ordinary skill in the art would have been motivated to combine the methods of Gaire and Tomita in order to achieve better classification results of regions of interest and to facilitate processing of images without resizing, as shown by Tomita (pg. 7, para. 4; pg. 9, para. 2). This modification would have had a reasonable expectation of success given Gaire automatically determines regions of interest based on identified cells, or features, in pathology slide images, Saltz discloses neural networks for feature extraction from neural networks may detect nuclei ([0008]; [0384]), and thus the neural network method of Tomita is applicable to the images Gaire.
Last regarding claims 1 and 15-16, Gaire in view of Tomita does not disclose, receiving, through a user device, user input that selects information associated with the one or more target items, and outputting a first visual representation in a region of the pathology slide image corresponding to the user input.
However, Saltz discloses a method for quantifying immune cells for clinical pathology in pathology images (Abstract). Saltz further discloses that a pathologist may choose a TIL-map editing tool from a user interface, select an image, and pan and zoom to view image regions, in addition to editing a heatmap overlay using lymphocyte sensitivity and using various sliders that allow the pathologist to change the threshold values to determine if a patch should be classified as lymphocyte-infiltrated or not for finer-grain editing of individual patches (i.e. displaying based on received input selection information associated with the one or more target items/detected cells) ([0266]). Saltz further discloses the pathologist can use the “Markup Edit” function to mark up specific patches and label them ([0266]).
It would have been prima facie obvious, to one of ordinary skill in the art, before the effective filing date of the claimed invention to have modified the method of Gaire in view of Tomita to have marked a visual representation of a region of a pathology slide based on selected information for the target items, as shown by Salts, discussed above. One of ordinary skill in the art would have been motivated to combine the methods of Gaire in view of Tomita with Saltz to allow for finer-grain editing of individual patches of the image, as shown by Satlz ([0266]). This modification would have had a reasonable expectation of success because both Gaire and Saltz receive, analyze, and display pathology slide images.
Regarding claim 4, Gaire further discloses the plurality of ROIs may be an image tile of a pre-defined size (i.e. a square, which is a polygon) or may be an automatically detected tumor region (i.e. a contour) ([0126]). Furthermore, Gaire in view of Tomita disclose the ROIs determined by the second artificial neural network are squares (i.e. polygons) (Figure 4; Supplementary Fig. 2).
Regarding claims 5 and 17, Gaire further discloses obtaining information on each region being immune infiltrated, immune excluded, or immune desert (i.e. obtaining the immune phenotype of immune inflamed, immune excluded, or immune desert) (Figure 3; [0222]).
Gaire further discloses generating a second visual representation of the immune phenotype, including CD8A+ cell densities and a separate plot of the immune cell distribution (FIG. 9), in addition to the colors for different cell types of immune and tumor cells and overall immune phenotype (Figure 3, e.g. see cell density graph in addition to distribution overlaid on image [0202], FIG. 9, e.g. CD8A+ density display in addition to image).
Regarding claims 6 and 18, Gaire further discloses displaying the tumor region together with the second visual representation of immune phenotypes (Fig. 3 and FIG. 9).
Regarding claims 7 and 19, while Gaire discloses displaying the CD8A+ cell density directly below the second visual representation of immune phenotypes ([0202]-[0203]; [0206], e.g. generated images #118 displayed; [02016]; FIG. 9 also see “CD8A+ density” next to region images), Gaire does not disclose the second visual representation is overlayed on the one or more ROIs in the pathology slide image.
However, the limitation of overlaying the second visual representation on the ROI is interpreted as a matter of design choice, and Applicant has not disclosed that this feature provides an advantage, is used for a particular purpose, or solves a stated problem when compared to displaying the second visual representation directly below the region, as shown by Gaire (FIG. 3 and 9). Therefore, the image of Gaire would perform equally as well in labeling or providing information on the region of interest and such a modification fails to patentably distinguish over Gaire.
Regarding claims 8 and 20¸ Gaire discloses obtaining an immune phenotype score for a tumor region ([0018]-[0020], e.g. combined score based on distribution of A and B cells; [0021],e.g. see definitions of A and B cells), wherein the score is for immune inflamed, immune excluded, or immune deserted (FIG. 3; [0134], e.g. combined score indicates immune-cell infiltration state).
Gaire discloses generating an image including a second visual representation of the CD8A+ cell density in addition to the proximity score (FIG. 9), which correspond to the immune phenotype score ([0020], e.g. proximity score used in combined score).
Regarding claim 9, Gaire further discloses displaying the tumor region together with the second visual representation of the immune phenotype score (([0202]-[0203]; [0206], e.g. generated images #118 displayed; [02016]; FIG. 9 see proximity score and density below images).
Regarding claim 10¸ while Gaire discloses displaying the tumor region directly below the second visual representation of immune phenotypes ([0202]-[0203]; [0206], e.g. generated images #118 displayed; [02016]; FIG. 9 also see “proximity score” next to region images), Gaire does not disclose the second visual representation is overlayed on the ROIs in the pathology slide image.
However, the limitation of overlaying the second visual representation on the ROI is interpreted as a matter of design choice, and Applicant has not disclosed that this feature provides an advantage, is used for a particular purpose, or solves a stated problem when compared to displaying the second visual representation directly below the regions, as shown by Gaire (FIG. 3 and 9). Therefore, the image of Gaire would perform equally as well in labeling or providing information on the region of interest and such a modification fails to patentably distinguish over Gaire.
Regarding claim 11¸ Gaire discloses obtaining an immune phenotype score (i.e. a feature) for tumor regions ([0018]-[0020], e.g. combined score based on distribution of A and B cells; [0021],e.g. see definitions of A and B cells), wherein the score (i.e. feature) is associated with a delta between an observed and relative distribution of tumor and immune cells (i.e. a statistical value) ([0018], e.g. proximity score representing the delta [0020], e.g. combined score includes proximity score; FIG. 3; [0134], e.g. combined score indicates immune-cell infiltration state).
Regarding claim 12, Gaire further discloses displaying the tumor regions together with image of a proximity score which corresponds to the immune phenotype score associated with the immune phenotypes (FIG. 9; [0020]).
Regarding claim 14, Gaire further discloses generating and outputting an image indicative of the detected immune and tumor cells (FIG. 3, e.g. see stained cells on top row; [0202], e.g. color information in image).
Therefore, the invention is prima facie obvious.
Response to Arguments
Applicant's arguments filed 22 April 2026 regarding 35 U.S.C. 103 have been fully considered but they are not persuasive.
Applicant remarks that the cited references do not teach or suggest “determining, using a second trained artificial neural network model…a plurality of regions of interest (ROIs) for determining immune phenotypes…”, and Saltz does not teach this limitation because the lymphocyte CNN does not use the outputs from the necrosis segmentation CNN and instead, the outputs of each CNN are combined to generate the final predicted TIL map (Applicant’s remarks at pg. 18, para. 7 to pg. 20, para. 2). Applicant further remarks that Gaire does not cure the deficiency of Saltz, and therefore, independent claims 1 and 16 are patentable over the cited references (Applicant’s remarks at pg. 21, para. 2).
This argument is not persuasive because it does not take into account the newly cited reference, Tomita, which discloses the newly amended limitation of “determining, using a second trained artificial neural network model…a plurality of regions of interest (ROIs) for determining immune phenotypes…”, as applied in the above rejection. Neither Salts or Gaire are relied upon to teach this limitation in the new grounds of rejection set forth above.
Applicant remarks that dependent claims 4-12, 14-15, and 17-20 depend from one of independent claims 1 and 16, and therefore, the dependent claims are patentable due to the respective dependencies and the additional features recited therein (Applicant’s remarks at pg. 21, para. 4).
This argument is not persuasive for the same reasons discussed above for claims 1 and 16. Furthermore, Applicant's arguments regarding “the additional features recited therein” fail to comply with 37 CFR 1.111(b) because they amount to a general allegation that the claims define a patentable invention without specifically pointing out how the language of the claims patentably distinguishes them from the references.
Conclusion
No claims are allowed.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Inquiries
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KAITLYN L MINCHELLA whose telephone number is (571)272-6485. The examiner can normally be reached 7:00 - 4:00 M-Th.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Olivia Wise can be reached at (571) 272-2249. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KAITLYN L MINCHELLA/Primary Examiner, Art Unit 1685
Prosecution Insights