DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. KR-10-2021-0139897, filed on 04/18/2024.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 04/18/2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Status
Claims 1-12 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter
Claim(s) 1-4 and 10-12 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Nie et al (U.S. 20210216746 A1; Nie).
Claim(s) 5-6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nie et al (U.S. 20210216746 A1; Nie), in view of Chukka et al (U.S. 20170103521 A1; Chukka).
Claim(s) 7-9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nie et al (U.S. 20210216746 A1; Nie), in view of Chukka et al (U.S. 20170103521 A1; Chukka) , and in further view of Gallagher (U.S. 20100254589 A1).
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-12 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. When reviewing independent claim 1, and based upon consideration of all of the relevant factors with respect to the claim as a whole, claim(s) 1-12 are held to claim an abstract idea without reciting elements that amount to significantly more than the abstract idea and is/are therefore rejected as ineligible subject matter under 35 U.S.C. 101. The Examiner will analyze Claim 1, and similar rationale applies to independent Claim(s) 11 and 12. The rationale, under MPEP § 2106, for this finding is explained below:
The claimed invention (1) must be directed to one of the four statutory categories, and (2) must not be wholly directed to subject matter encompassing a judicially recognized exception, as defined below. The following two step analysis is used to evaluate these criteria.
Step 1: Is the claim directed to one of the four patent-eligible subject matter categories: process, machine, manufacture, or composition of matter?
When examining the claim under 35 U.S.C. 101, the Examiner interprets that the claims is related to a process since the claim is directed to a method. All claims fall within a statutory category.
Step 2a, Prong 1: Does the claim wholly embrace a judicially recognized exception, which includes laws of nature, physical phenomena, and abstract ideas, or is it a particular practical application of a judicial exception?
The Examiner interprets that the judicial exception applies since Claim 1 recites (1) applying a pre-trained neural network to a medical image to generate position information for stained cells, and (2) calculating a staining ratio within the resulting bounding box. Both limitations recite mathematical concepts (application of a trained mathematical model and an arithmetic area ratio calculation) and are considered together as a single abstract idea. See example of organizing information and manipulating information through mathematical correlations, Digitech Image Techs., LLC v. Electronics for Imaging, Inc., 758 F.3d 1344, 1350, 111 USPQ2d 1717, 1721 (Fed. Cir. 2014). The patentee in Digitech claimed methods of generating first and second data by taking existing information, manipulating the data using mathematical functions, and organizing this information into a new form. The court explained that such claims were directed to an abstract idea because they described a process of organizing information through mathematical correlations, like Flook's method of calculating using a mathematical formula. 758 F.3d at 1350, 111 USPQ2d at 1721. If the claim recites a judicial exception (i.e., an abstract idea enumerated in MPEP § 2106.04(a)(2), a law of nature, or a natural phenomenon), the claim requires further analysis in Prong Two.
Step 2a, Prong 2: Does the claim recite additional elements that integrate the judicial exception into a practical application?
The Examiner interprets that Claim 1 limitation does not provide additional elements or combination of additional elements to a practical application. The pre-trained neural network is applied to medical images as a new data domain, which does not reflect an improvement to the technology itself. The staining ratio calculation is straightforward pixel-area arithmetic. Transmitting results to a user terminal is post-solution activity. The specification frames the benefit as faster pathological diagnosis - an improvement to the abstract idea, not to computer functioning or image processing technology. See MPEP 2106.05(a).see MPEP 2106.05(g). or Generally linking the use of the judicial exception to a particular technological environment or field of use – see MPEP 2106.05(h). See, MPEP §2106.04(a), Because a judicial exception is not eligible subject matter, Bilski, 561 U.S. at 601, 95 USPQ2d at 1005-06 (quoting Chakrabarty, 447 U.S. at 309, 206 USPQ at 197 (1980)), if there are no additional claim elements besides the judicial exception, or if the additional claim elements merely recite another judicial exception, that is insufficient to integrate the judicial exception into a practical application. See, e.g., RecogniCorp, LLC v. Nintendo Co., 855 F.3d 1322, 1327, 122 USPQ2d 1377 (Fed. Cir. 2017) ("Adding one abstract idea (math) to another abstract idea (encoding and decoding) does not render the claim non-abstract"). If there are no additional elements in the claim, then it cannot be eligible. In such a case, after making the appropriate rejection (see MPEP § 2106.07 for more information on formulating a rejection for lack of eligibility), it is a best practice for the examiner to recommend an amendment, if possible, that would resolve eligibility of the claim.
Step 2b: If a judicial exception into a practical application is not recited in the claim, the Examiner must interpret if the claim recites additional elements that amount to significantly more than the judicial exception.
The Examiner interprets that the Claims do not amount to significantly more since the Claim(s) is/state Adding insignificant extra-solution activity to the judicial exception, e.g., mere data gathering in conjunction with a law of nature or abstract idea such as a step of obtaining information about credit card transactions so that the information can be analyzed by an abstract mental process, as discussed in CyberSource v. Retail Decisions, Inc., 654 F.3d 1366, 1375, 99 USPQ2d 1690, 1694 (Fed. Cir. 2011) (see MPEP § 2106.05(g)).
Furthermore, the generic computing device comprising a processor, memory, and network unit (claim 12); and a pre-trained neural network model of unspecified architecture (all independent claims). These recited as performing generic computer functions that are well-understood, routine and conventional activities amount to no more than implementing the abstract idea with a computerized system.
Claims 2-10 are depending on the independent claim/s 1 include all the limitation of the independent claim. The Examiner finds that Claim(s) 2-10 do/does not state significantly more since the claim only recites obtain analysis data and output the data information.
Thus, Claims 1-12 recite the same abstract idea and therefore are not drawn to the eligible subject matter as they are directed to the abstract idea without significantly more.
Therefore, the Examiner interprets that the claims are rejected under 35 U.S.C. 101.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-4 and 10-12 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Nie et al (U.S. 20210216746 A1; Nie).
Regarding claim 1, Nie discloses a method for analyzing a medical image based on deep learning, (Abstract: “ automated systems and methods adapted to quickly and accurately train a neural network to detect and/or classify cells and/or nuclei.”) is the method performed by a computing device (Fig.1: Computer system 14) including at least one processor, (Fig.2: one or more processor 209; memory 201) the method comprising:
obtaining position information of a stained cell present in a medical image by using a pre- trained neural network model; (Fig. 2A-2C; Paragraph 70: “Once the neural network 212 is trained, the trained neural network 212 may be used in conjunction with the region proposal network 214 for detecting and/or classifying cells in an input image”; Paragraph 97: “the region proposal network 214 is configured for cell detection. In other words, the region proposal network 214 uses features from the generated feature map to detect the cells in the sample images based on the determined features. In some embodiments, the region proposal network 214 is configured to generate bounding box detection results.”; Paragraph 89-90; Paragraph 149: “the procedures and automated algorithms described herein may be adapted to identify and classify various types of cells or cell nuclei based on features within the input images …Each of these distinct staining patterns may be used as features for identifying cells and/or nuclei.”) and
calculating a staining ratio of the stained cell in a bounding box including the stained cell corresponding to the position information. (Figs. 2A-2C, 9-11 ; Paragraph 62: “the embodiments described herein significantly reduce the burden on the user to annotate cells at the pixel level for the detection network to learn. In other words, the embodiments described herein enable bounding box labeling for detection, which significantly reduces user annotation burden. “; Paragraph 73: “the system 250 also includes a scoring module to score the detected and/or classified cells, e.g. to determine an H-score or a percent positivity.”; Paragraphs 171-173: “derived stain intensity values, counts of specific cells, or other classification results may be used to determine various marker expression scores, such as percent positivity, an Allred score, or an H-Score, using scoring module 260. … based at least in part on the number of biomarker-positive tumor cells/biomarker-positive non-tumor cells, a score (e.g., a whole-slide score, or a score for an annotated area of an image, such as an area annotated by a pathologist or histologist) can be determined. …. the ‘H’ score is used to assess the percentage of tumor cells with cell membrane staining graded as ‘weak,’ ‘moderate’ or ‘strong.’. The grades are summated to give an overall maximum score of 300 and a cut-off point of 100 to distinguish between a ‘positive’ and ‘negative.’ For example, a membrane staining intensity (0, 1+, 2+, or 3+) is determined for each cell in a fixed field of view (or here, each cell in a tumor or cell cluster).”; Paragraphs 174-176, it shows that “the percentage” is read as “ratio”)
Regarding claim 2, Nie discloses the obtaining of the position information of the stained cell present in the medical image includes: obtaining the bounding box including the stained cell and a coordinate value of the bounding box by inputting the medical image into the neural network model. (Paragraphs 96-97: “A region proposal network 214 can be generally defined as a fully convolutional network that detects objects in images and proposes regions corresponding to those detected objects. The proposal network may overlay a sliding window on the feature map generated by one or more convolutional layers generating k anchor boxes. The results of overlaying the sliding window on the feature map may be input to an intermediate layer of the proposal network, which may generate 2 k scores via a cls layer and 4 k coordinates via a reg layer … the region proposal network 214 is configured to generate bounding box detection results.” ; Paragraph 153: “the images received as input are processed such as to detect nucleus centers (seeds) and/or to segment the nuclei. ….along with color intensities in the input image, image gradient information is also used in radial symmetry voting and combined with an adaptive segmentation process to precisely detect and localize the cell nuclei. A “gradient” as used herein is, for example, the intensity gradient of pixels calculated for a particular pixel by taking into consideration an intensity value gradient of a set of pixels surrounding said particular pixel. Each gradient may have a particular “orientation” relative to a coordinate system whose x- and y-axis are defined by two orthogonal edges of the digital image”)
Regarding claim 3, Nie discloses the neural network model is pre-trained based on whether a cell expressed by staining is positive or negative and a medical image in which the bounding box including the cell is labeled. (Figs. 4 and 9-10;Paragraph 178: “During the process of learning to detect and localize cells, Model A learns features that can facilitate clustering of cells. …. n FIG. 9A, blue boxes were placed mainly on Her2 positive cells, while the green boxes were placed on Her2 negative cells, which means that the two types of cells were separated well through clustering using features derived from Faster-RCNN. Even though we did not provide cell class label for fine tuning the Faster-RCNN, the pre-trained Faster-RCNN already provided richer feature representations which were “transferred” to the fine-tuned Faster-RCNN.”)
Regarding claim 4, Nie discloses the staining ratio of the stained cell is a ratio of an area of a positive region of cell expressed by staining in the bounding box to a total area of the bounding box. ( Paragraph 72: “the cell detection and classification module comprises a classifier, e.g. a support vector machine., … the system 250 also includes a scoring module to score the detected and/or classified cells, e.g. to determine an H-score or a percent positivity.”; Paragraph 91: “The proposal regions are cropped out of the image and resized. Then, the CNN classifies the cropped and resized regions. Finally, the region proposal bounding boxes are refined by a support vector machine (SVM) that is trained using CNN features. “; Paragraphs 171: “derived stain intensity values, counts of specific cells, or other classification results may be used to determine various marker expression scores, such as percent positivity, an Allred score, or an H-Score, using scoring module 260. … based at least in part on the number of biomarker-positive tumor cells/biomarker-positive non-tumor cells, a score (e.g., a whole-slide score, or a score for an annotated area of an image, such as an area annotated by a pathologist or histologist) can be determined. … the number of identified cells output corresponds to the total number of biomarker-positive tumor cells detected in a region, as evidenced by the number of tumor cells counted. … the ‘H’ score is used to assess the percentage of tumor cells with cell membrane staining graded as ‘weak,’ ‘moderate’ or ‘strong.’”)
Regarding claim 10, Nie discloses further comprising: transmitting, to a user terminal, (Paragraph 219: “The computing system can include any number of clients and servers. A client and server are generally remote from each other and typically interact through a communication network. …a server transmits data (e.g., an HTML page) to a client device (e.g., for purposes of displaying data to and receiving user input from a user interacting with the client device).”) the position information of the stained cell obtained through the neural network model Fig. 2A-2C; Paragraph 70: “Once the neural network 212 is trained, the trained neural network 212 may be used in conjunction with the region proposal network 214 for detecting and/or classifying cells in an input image”; Paragraph 97: “the region proposal network 214 is configured for cell detection. In other words, the region proposal network 214 uses features from the generated feature map to detect the cells in the sample images based on the determined features. In some embodiments, the region proposal network 214 is configured to generate bounding box detection results.”; Paragraph 89-90; Paragraph 149: “the procedures and automated algorithms described herein may be adapted to identify and classify various types of cells or cell nuclei based on features within the input images …Each of these distinct staining patterns may be used as features for identifying cells and/or nuclei.”) and the calculated staining ratio, (Figs. 2A-2C, 9-11 ; Paragraph 62: “the embodiments described herein significantly reduce the burden on the user to annotate cells at the pixel level for the detection network to learn. In other words, the embodiments described herein enable bounding box labeling for detection, which significantly reduces user annotation burden. “; Paragraph 73: “the system 250 also includes a scoring module to score the detected and/or classified cells, e.g. to determine an H-score or a percent positivity.”; Paragraphs 171-173: “derived stain intensity values, counts of specific cells, or other classification results may be used to determine various marker expression scores, such as percent positivity, an Allred score, or an H-Score, using scoring module 260. … based at least in part on the number of biomarker-positive tumor cells/biomarker-positive non-tumor cells, a score (e.g., a whole-slide score, or a score for an annotated area of an image, such as an area annotated by a pathologist or histologist) can be determined. …. the ‘H’ score is used to assess the percentage of tumor cells with cell membrane staining graded as ‘weak,’ ‘moderate’ or ‘strong.’. The grades are summated to give an overall maximum score of 300 and a cut-off point of 100 to distinguish between a ‘positive’ and ‘negative.’ For example, a membrane staining intensity (0, 1+, 2+, or 3+) is determined for each cell in a fixed field of view (or here, each cell in a tumor or cell cluster).”; Paragraphs 174-176, it shows that “the percentage” is read as “ratio”) wherein the position information includes a coordinate value of the bounding box, Paragraph 153: “the images received as input are processed such as to detect nucleus centers (seeds) and/or to segment the nuclei. ….along with color intensities in the input image, image gradient information is also used in radial symmetry voting and combined with an adaptive segmentation process to precisely detect and localize the cell nuclei. A “gradient” as used herein is, for example, the intensity gradient of pixels calculated for a particular pixel by taking into consideration an intensity value gradient of a set of pixels surrounding said particular pixel. Each gradient may have a particular “orientation” relative to a coordinate system whose x- and y-axis are defined by two orthogonal edges of the digital image”) and wherein the staining ratio is a value of a ratio of an area of a positive region of a cell expressed by staining in the bounding box to a total area of the bounding box. ( Paragraph 72: “the cell detection and classification module comprises a classifier, e.g. a support vector machine., … the system 250 also includes a scoring module to score the detected and/or classified cells, e.g. to determine an H-score or a percent positivity.”; Paragraph 91: “The proposal regions are cropped out of the image and resized. Then, the CNN classifies the cropped and resized regions. Finally, the region proposal bounding boxes are refined by a support vector machine (SVM) that is trained using CNN features. “; Paragraphs 171: “derived stain intensity values, counts of specific cells, or other classification results may be used to determine various marker expression scores, such as percent positivity, an Allred score, or an H-Score, using scoring module 260. … based at least in part on the number of biomarker-positive tumor cells/biomarker-positive non-tumor cells, a score (e.g., a whole-slide score, or a score for an annotated area of an image, such as an area annotated by a pathologist or histologist) can be determined. … the number of identified cells output corresponds to the total number of biomarker-positive tumor cells detected in a region, as evidenced by the number of tumor cells counted. … the ‘H’ score is used to assess the percentage of tumor cells with cell membrane staining graded as ‘weak,’ ‘moderate’ or ‘strong.’”)
Regarding claim 11, Nie discloses A computer program stored in a computer-readable storage medium, wherein the computer program cause one or more processors to execute following operations for analyzing a medical image based on deep learning when the computer program is executed by the one or more processors, (Paragraph 67: “A digital pathology system 200 for imaging and analyzing specimens is illustrated in FIGS. 1 and 2A-2C. … he computer system 14 can include a desktop computer, a laptop computer, a tablet, or the like, digital electronic circuitry, firmware, hardware, memory 201, a computer storage medium (240), a computer program or set of instructions (e.g. where the program is stored within the memory or storage medium), one or more processors (209) (including a programmed processor), and any other hardware, software, or firmware modules or combinations thereof “) the operations comprising:
an operation of obtaining position information of a stained cell present in a medical image by using a pre-trained neural network model; (Fig. 2A-2C; Paragraph 70: “Once the neural network 212 is trained, the trained neural network 212 may be used in conjunction with the region proposal network 214 for detecting and/or classifying cells in an input image”; Paragraph 97: “the region proposal network 214 is configured for cell detection. In other words, the region proposal network 214 uses features from the generated feature map to detect the cells in the sample images based on the determined features. In some embodiments, the region proposal network 214 is configured to generate bounding box detection results.”; Paragraph 89-90; Paragraph 149: “the procedures and automated algorithms described herein may be adapted to identify and classify various types of cells or cell nuclei based on features within the input images …Each of these distinct staining patterns may be used as features for identifying cells and/or nuclei.”)and
an operation of calculating a staining ratio of the stained cell in a bounding box including the stained cell corresponding to the position information. (Figs. 2A-2C, 9-11 ; Paragraph 62: “the embodiments described herein significantly reduce the burden on the user to annotate cells at the pixel level for the detection network to learn. In other words, the embodiments described herein enable bounding box labeling for detection, which significantly reduces user annotation burden. “; Paragraph 73: “the system 250 also includes a scoring module to score the detected and/or classified cells, e.g. to determine an H-score or a percent positivity.”; Paragraphs 171-173: “derived stain intensity values, counts of specific cells, or other classification results may be used to determine various marker expression scores, such as percent positivity, an Allred score, or an H-Score, using scoring module 260. … based at least in part on the number of biomarker-positive tumor cells/biomarker-positive non-tumor cells, a score (e.g., a whole-slide score, or a score for an annotated area of an image, such as an area annotated by a pathologist or histologist) can be determined. …. the ‘H’ score is used to assess the percentage of tumor cells with cell membrane staining graded as ‘weak,’ ‘moderate’ or ‘strong.’. The grades are summated to give an overall maximum score of 300 and a cut-off point of 100 to distinguish between a ‘positive’ and ‘negative.’ For example, a membrane staining intensity (0, 1+, 2+, or 3+) is determined for each cell in a fixed field of view (or here, each cell in a tumor or cell cluster).”; Paragraphs 174-176, it shows that “the percentage” is read as “ratio”)
Regarding claim 12, Nie discloses A computing device for analyzing a medical image based on deep learning, comprising: a processor including at least one core; a memory including program codes executable in the processor; and a network unit receiving a medical image or transmitting a calculation result of the processor to a user terminal, wherein the processor is (Paragraph 67: “A digital pathology system 200 for imaging and analyzing specimens is illustrated in FIGS. 1 and 2A-2C. The digital pathology system 200 may comprise an imaging apparatus 12 (e.g. an apparatus having means for scanning a specimen-bearing microscope slide) and a computer 14, whereby the imaging apparatus 12 and computer may be communicatively coupled together (e.g. directly, or indirectly over a network 20). The computer system 14 can include a desktop computer, a laptop computer, a tablet, or the like, digital electronic circuitry, firmware, hardware, memory 201, a computer storage medium (240), a computer program or set of instructions (e.g. where the program is stored within the memory or storage medium), one or more processors (209) (including a programmed processor), and any other hardware, software, or firmware modules or combinations thereof”) configured to:
obtain position information of a stained cell present in a medical image by using a pre-trained neural network model, (Fig. 2A-2C; Paragraph 70: “Once the neural network 212 is trained, the trained neural network 212 may be used in conjunction with the region proposal network 214 for detecting and/or classifying cells in an input image”; Paragraph 97: “the region proposal network 214 is configured for cell detection. In other words, the region proposal network 214 uses features from the generated feature map to detect the cells in the sample images based on the determined features. In some embodiments, the region proposal network 214 is configured to generate bounding box detection results.”; Paragraph 89-90; Paragraph 149: “the procedures and automated algorithms described herein may be adapted to identify and classify various types of cells or cell nuclei based on features within the input images …Each of these distinct staining patterns may be used as features for identifying cells and/or nuclei.”) and
calculate a staining ratio of the stained cell in a bounding box including the stained cell corresponding to the position information. (Figs. 2A-2C, 9-11 ; Paragraph 62: “the embodiments described herein significantly reduce the burden on the user to annotate cells at the pixel level for the detection network to learn. In other words, the embodiments described herein enable bounding box labeling for detection, which significantly reduces user annotation burden. “; Paragraph 73: “the system 250 also includes a scoring module to score the detected and/or classified cells, e.g. to determine an H-score or a percent positivity.”; Paragraphs 171-173: “derived stain intensity values, counts of specific cells, or other classification results may be used to determine various marker expression scores, such as percent positivity, an Allred score, or an H-Score, using scoring module 260. … based at least in part on the number of biomarker-positive tumor cells/biomarker-positive non-tumor cells, a score (e.g., a whole-slide score, or a score for an annotated area of an image, such as an area annotated by a pathologist or histologist) can be determined. …. the ‘H’ score is used to assess the percentage of tumor cells with cell membrane staining graded as ‘weak,’ ‘moderate’ or ‘strong.’. The grades are summated to give an overall maximum score of 300 and a cut-off point of 100 to distinguish between a ‘positive’ and ‘negative.’ For example, a membrane staining intensity (0, 1+, 2+, or 3+) is determined for each cell in a fixed field of view (or here, each cell in a tumor or cell cluster).”; Paragraphs 174-176, it shows that “the percentage” is read as “ratio”)
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 5-6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nie et al (U.S. 20210216746 A1; Nie), in view of Chukka et al (U.S. 20170103521 A1; Chukka).
Regarding claim 5, Nie discloses the calculating of the staining ratio of the stained cell Paragraph 73: “the system 250 also includes a scoring module to score the detected and/or classified cells, e.g. to determine an H-score or a percent positivity.”; Paragraphs 171-173: “derived stain intensity values, counts of specific cells, or other classification results may be used to determine various marker expression scores, such as percent positivity, an Allred score, or an H-Score, using scoring module 260. … based at least in part on the number of biomarker-positive tumor cells/biomarker-positive non-tumor cells, a score (e.g., a whole-slide score, or a score for an annotated area of an image, such as an area annotated by a pathologist or histologist) can be determined. …. the ‘H’ score is used to assess the percentage of tumor cells with cell membrane staining graded as ‘weak,’ ‘moderate’ or ‘strong.’. The grades are summated to give an overall maximum score of 300 and a cut-off point of 100 to distinguish between a ‘positive’ and ‘negative.’ For example, a membrane staining intensity (0, 1+, 2+, or 3+) is determined for each cell in a fixed field of view (or here, each cell in a tumor or cell cluster).”; Paragraphs 174-176, it shows that “the percentage” is read as “ratio”)
However, Nie does not discloses generating a binary image for the bounding box based on a staining intensity of the medical image, and calculating the staining ratio of the stained cell based on the binary image.
Chukka discloses the calculating of the staining ratio of the stained cell (Paragraph 193: “ once the marker positive (MET positive in this case, HER2 positive in HER2 stained slides) tumor cells are identified in the earlier step, for each cell—the circumferential percentage of the membrane/cytoplasmic staining is computed, along with the average DAB intensity of the staining and all the counter stained tumor cell detections”) includes: generating a binary image for the bounding box based on a staining intensity of the medical image, (Paragraphs 178-180: “the method may begin with generation of a mask (S301), or a “binary mask image” or “binary refinement mask”, from the DAB channel image where the membranous and cytoplasmic region is set to true and all everywhere else false. A low threshold value, selected from a set of training examples, may be used to segment the DAB channel. All pixels with intensity value above the threshold are set to true and otherwise false. … The DAB intensity image is masked out using the binary refinement mask.”) and calculating the staining ratio of the stained cell based on the binary image. (Paragraph 173-174: “the brown mask, i.e., the binary refinement mask, also helps clear out empty regions, i.e. regions without any nuclei or cell membranes. … completeness and intensity computation module 117 and scoring/binning module 118 are invoked to determine how completely circumferential or how well-enclosed each detected (“identified”) nucleus is, and the intensity of detection results, in order to score the field of view and/or the image. … these scores and categories are based on percentages of complete circumferences and intensities of the detection results. The output from these modules provides a results image depicting completely and partially stained nuclei and membranes.”)
Therefore, it would been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Nie by including A refinement module, intensity computation module and scoring/binning module that is taught by Chukka, to make the invention that automatically identifying biomarker-positive tumor cells on a slide; thus, one of ordinary skilled in the art would have been motivated to combine the references since this will improving the accuracy of biomarker-positive tumor cells identification as well as increasing the accuracy of membrane identification in stained tissue slides. (Chukka: Paragraphs 14 and 43)
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention.
Regarding claim 6, Nie, as modified by Chukka, discloses all the claims invention. Chukka further discloses the binary image is generated based on a result of comparing the staining intensity of the bounding box and a first threshold, (Paragraphs 178-180: “the method may begin with generation of a mask (S301), or a “binary mask image” or “binary refinement mask”, from the DAB channel image where the membranous and cytoplasmic region is set to true and all everywhere else false. A low threshold value, selected from a set of training examples, may be used to segment the DAB channel. All pixels with intensity value above the threshold are set to true and otherwise false. … The DAB intensity image is masked out using the binary refinement mask.”) and wherein the first threshold is a staining intensity which becomes a reference for classifying the stained cell into a positive cell. (Paragraph 215: “Using the completeness and intensity, scores are assigned to the each cell, and the image is binned based on a percentage of cells meeting score thresholds. Each “cell” in this context means, for example, each tumor cell contained in the examined area of the slide, and in particular, each biomarker-positive tumor cell. The score thresholds are defined by training data, based on tissue size, stain variability, and staining protocols, and may be subjective depending on different laboratories performing the procedure. Therefore, the thresholds may vary, and range from 0-255. Different scoring criteria exist for different types of biomarkers. The output biomarker score is a four-binned score of (0, 1+, 2+, 3+) which is used to give a clinical diagnostic evaluation of the patient being marker positive or negative (ex: MET positive or MET negative; HER2 positive or HER2 negative)”; Paragraph 168: “biomarker-positive tumor cells having a brown stained nuclei with intensity values equal to or above a minimum threshold level may be kept as biomarker-positive tumor cells. Biomarker-positive tumor cells having a brown stained nuclei with intensity values below said minimum threshold level may be identified as background cells or cytoplasmic blush.”)
Claim(s) 7-9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nie et al (U.S. 20210216746 A1; Nie), in view of Chukka et al (U.S. 20170103521 A1; Chukka) , and in further view of Gallagher (U.S. 20100254589 A1).
Regarding claim 7, Nie, as modified by Chukka, discloses all the claims invention. Chukka further discloses the calculating of the staining ratio of the stained cell based on the binary image (Paragraph 173-174: “the brown mask, i.e., the binary refinement mask, also helps clear out empty regions, i.e. regions without any nuclei or cell membranes. … completeness and intensity computation module 117 and scoring/binning module 118 are invoked to determine how completely circumferential or how well-enclosed each detected (“identified”) nucleus is, and the intensity of detection results, in order to score the field of view and/or the image. … these scores and categories are based on percentages of complete circumferences and intensities of the detection results. The output from these modules provides a results image depicting completely and partially stained nuclei and membranes.”)
However, , Nie, as modified by Chukka, does not disclose extracting a reference region from the binary image based on a result of comparing a size of the binary image and a second threshold, and calculating the staining ratio of the stained cell based on the extracted reference region.
Gallagher discloses the calculating of the staining ratio of the stained cell based on the binary image includes: extracting a reference region from the binary image based on a result of comparing a size of the binary image and a second threshold, (Paragraph 123: “The purpose of watershed segmentation is to create a 3D view of the stains, again to provide a clearer, more defined image of the positively stained candidate objects of interest (as presented in the First Image) … to further differentiate those areas, an (automated) threshold setting technique may be used where a boundary can be set on the areas (in the first and second images) under consideration. When the boundaries are set, the images are formed by eliminating all parts of the image that are below the thresholds which have been set.”) and calculating the staining ratio of the stained cell based on the extracted reference region. (Paragraph 147: “the level of expression of a candidate object of interest in a biological sample is determined by relating the amount and/or degree (such as intensity) of staining of the candidate positive group relative the degree of staining of the candidate positive group and the candidate negative group.”; Paragraphs 226-227: “The DAB and haematoxylin fractions are further analysed independently to identify all nuclei and eliminate non-specific staining. … inally, the percentage of DAB tumour positive nuclei and the mean intensity of DAB positive nuclei is calculated and stored as the final output for the core.”)
Therefore, it would been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Nie and Chukka by including a threshold analysis technique that is taught by Gallagher, to make the invention that a method for determining the level of expression of one or more candidate objects of interest using an automated (computer-aided) image analysis system; thus, one of ordinary skilled in the art would have been motivated to combine the references since this will improving automated image analysis such as quickly and accurately scan large amounts of biological material on a slide and provide an accurate measurement of the level of expression of one or more candidate objects of interest in a biological sample as well as eliminating the need for operator input to locate and identify candidate objects of interest in a biological sample for analysis. (Gallagher: Paragraphs 18-19)
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention.
Regarding claim 8, Nie, as modified by Chukka and Gallagher, discloses all the claims invention. Gallagher further discloses when the size of the binary image is equal to or less than the second threshold, the reference region is a whole region of the binary image. (Paragraph 123: “The purpose of watershed segmentation is to create a 3D view of the stains, again to provide a clearer, more defined image of the positively stained candidate objects of interest (as presented in the First Image) … to further differentiate those areas, an (automated) threshold setting technique may be used where a boundary can be set on the areas (in the first and second images) under consideration. When the boundaries are set, the images are formed by eliminating all parts of the image that are below the thresholds which have been set”; Paragraph 197-198: “The images resulting after the stain separation process are shown as images 1(a) and 1(b). Also shown is image 0(a) in FIG. 6B, which is the slide background following stain separation. … A binary (1 or 0) map is created for each single pixel to indicate whether is part (1) or not (0) of both DAB positive (DAB MASK) and Haematoxylin (H MASK)”)
Regarding claim 9, Nie, as modified by Chukka and Gallagher, discloses all the claims invention. Gallagher further discloses when the size of the binary image is more than the second threshold, the reference region is a partial region of the binary image based on a center of the binary image. (Paragraph 42: “Once the boundaries are set, any parts (eg any objects) of the First Image and/or the Second Image with boundary values which are above or below the thresholds which have been set, are eliminated. Preferably, the "boundary" is selected from the group consisting of a standard deviation (STD) value or a STD range. “; Paragraphs 128-132: “the STD value is used as a boundary value to distinguish between candidate objects of interest in one or more spatially adjacent groups and/or in one or more spatially distant groups. ….The size of the area encompassed by the one or more candidate objects of interest is determined by using the major axis length and the minor axis length of the smallest ellipse containing the candidate object of interest; The distance between the centre of the smallest ellipse containing each candidate object of interest and the centre of the smallest ellipse containing each neighbouring candidate object of interest is measured. Using these measurements, the candidate objects of interest are segmented into at least two groups,”; Paragraphs 197-198)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Dobson et al (U.S. 20110111435 A1), “Detecting Cell Surface Markers”, teaches about a method for detecting an expression level of a cell surface marker in a sample, comprising staining the sample with a reagent that labels the cell surface marker; obtaining an image of the stained sample; and determining a value for continuity of cell surface staining in the image, wherein the value is indicative of the expression level.
Binnig et al (U.S. 20130108139 A1), “Biomarker Evaluation Through Image Analysis”, teaches about a method for determining whether a test biomarker is a stain for a type of cell component, such as membrane or nucleus, involves performing various segmentation processes on an image of tissue stained with the test biomarker. One segmentation process searches for a first cell component type, and another segmentation process searches for a second cell component type by segmenting only stained pixels. The test biomarker is identified as a stain for each component type if the process identifies the component based only on stained pixels. Whether the test biomarker is a membrane stain or nucleus stain is displayed on a graphical user interface.
Chang et al (U.S. 20190042826 A1), “Automatic Nuclei Segmentation in Histopathology Images”, teaches about systems and computer-implemented methods for quantitative analyses of tissue sections (including, histopathology samples, such as immunohistochemically labeled or H&E stained tissue sections), involving automatic unsupervised segmentation of image(s) of the tissue section(s), measurement of multiple features for individual nuclei within the image(s), clustering of nuclei based on extracted features, and/or analysis of the spatial arrangement and organization of features in the image based on spatial statistics.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Duy A Tran whose telephone number is (571)272-4887. The examiner can normally be reached Monday-Friday 8:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ONEAL R MISTRY can be reached at (313)-446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DUY TRAN/Examiner, Art Unit 2674
/ONEAL R MISTRY/Supervisory Patent Examiner, Art Unit 2674
Prosecution Insights